| We asked a large language model, GPT-3, to complete the sentence “Peanut butter and pineapples___”. It said: “Peanut butter and pineapples are a great combination. The sweet and savory flavors of peanut butter and pineapple complement each other perfectly.” If a person said this, one might infer that they had tried peanut butter and pineapple together, formed an opinion and shared it with the reader. But how did GPT-3 come up with this paragraph? By generating a word that fit the context we provided. And then another one. And then another one. The model never saw, touched or tasted pineapples—it just processed all the texts on the Internet that mention them. Up to this point, the article sticks to its thesis that humans have a tendency to ascribe real-world experience to the agents generating fluent text. And yet reading this paragraph can lead the human mind – even that of a Google engineer – to imagine GPT-3 as an intelligent being that can reason about peanut butter and pineapple dishes. And with this line they jump past that thesis into an implication that AI can't reason about concepts it hasn't experienced physically with a human body. This is obviously incorrect, we humans reason all the time about things we haven't directly experienced. I think important concepts in evaluating different types of intelligence are the model learning abstraction, mapping between concepts, and reasoning correctly given the information available in its training data. Does GPT-3 have an understanding of abstractions like objects, combinations, food, flavor? Why does it need to have tasted pineapple to infer that sentence? It seems to know that peanut butter and savory are associated. Can an educated person, blind from birth, not infer that if it is day time and it is not cloudy, then the sky is blue? Surely they can be considered to possess intelligence, despite never experiencing color directly. |