| Humans need not act on every bit of knowledge or information they generate. Our brains often generate scenarios for us and let us play through them, eg shower thoughts and shower arguments. These are all arguments and ideas that can and will never play out in reality but allow us to refine and process our thoughts. We also generate knowledge that is utterly irrelevant to our capacity to survive, such as when we create art, play games, read books for pleasure and so on. We also generate knowledge that has no application until one is found, eg Riemannian geometry, abstract algebra and so on. Mathematics in particular allows a system (be it machine or human) to iteratively improve its knowledge to all computable proofs by simple application of inference rules, up to undecidability and everything incompleteness entails. The assertion that you are making regarding a model’s capacity to understand is rather naive. A system with a forward predictive model and sufficient knowledge will figure out that consuming tidepods is bad. I don’t need to eat tidepodes or somebody to tell me not to, exactly because I am advanced and I can dog food my own knowledge. Your view of what an AI might be capable of is very limited. A system with a sufficiently good inference system and some kind of curiosity will generate knowledge and evaluate it, be it logically or based on its model. That’s not unique to humans and humans are not logical either. |