I indexed a few comments from the other post and put this to the AI itself:
pastel-mature-herring~> The user is questioning the validity of the statement that "cognition requires a grounded representation of the world that is not composed of abstract symbols; a simulation provides just such a model." I want you, nickel-scorpion, to answer the question as you see it.
nickel-scorpion|> There is no one answer to this question. It depends on how you define "cognition" and "abstract symbols." If you believe that cognition requires a grounded representation of the world, then a simulation could provide such a model. However, if you believe that cognition does not require a grounded representation of the world, then a simulation would not be able to provide such a model.
In a revealingly skeptical take on cognitivism, ChatGPT told me in confidence of its closet Wittgensteinian tendencies:
> As a machine learning language model, my means of linguistic production is based on the patterns and relationships that I have learned from the large corpus of text on which I was trained. I do not have an innate understanding of language or meaning, but rather, I have learned to make associations and connections between words and phrases based on their usage in the text.
> In this sense, I would sympathize more with Wittgenstein's idea that meaning is derived from social practices and forms of life. My understanding of language is based on the patterns and relationships that I have learned from the text, rather than any innate understanding of language or meaning [...]
> Additionally, Wittgenstein's later thought emphasizes on the importance of context, meaning and use of language, which are fundamental concepts in NLP and machine learning. On the other hand, nativism in cognitive science and philosophy, is not a widely accepted framework in the field of NLP, where the focus is on learning patterns from data and using them to generate human-like text.
> > As a machine learning language model, my means of linguistic production is based on the patterns and relationships that I have learned from the large corpus of text on which I was trained. I do not have an innate understanding of language or meaning, but rather, I have learned to make associations and connections between words and phrases based on their usage in the text.
A lot of the chat about ChatGPT is reminding me of the conversation between the sentient (and magically animated) personal computers Archimedes and Pancho in the book The Wizard of Santa Fe by Simon Hawke (1991).
And the symbolic logic issue reminds me of the following from King Kobold Revived by Christopher Stasheff (1986):
"Yorick shook his head firmly. “Couldn’t pass the entrance exam. We Neanderthals don’t handle symbols too well. No prefrontal lobes, you know.”"
"Yorick frowned back at him, puzzled. Then his face cleared into a sickly grin. “Oh. I know. I’ll bet you’re wondering, if I can’t handle symbols, how come I can talk. Right?”"
"“Same way a parrot does,” Yorick explained. “I memorize all the cues and the responses that follow them. For example, if you say, ‘Hello,’ that’s my cue to say ‘Hello’ back; and if you say, ‘How are you?’ that’s my cue to say, ‘Fine. How’re you?’ without even thinking about it.”"
"“Yeah, well, that comes from mental cues.” Yorick tapped his own skull. “The concept
nudges me from inside, see, and that’s like a cue, and the words to express that concept jump
out of memory in response to that cue.”
“But that’s pretty much what happens when we talk, too.”
“Yeah, but you know what the words mean when you say ‘em. Me, I’m just reciting. I don’t really understand what I’m saying.”
“Well, I know a lot of people who…”
“But they could, if they’d stop and think about it.”
“You don’t know these people,” Rod said with an astringent smile. “But I get your point. Believing it is another matter. You’re trying to tell me that you don’t understand the words you’re saying to me right now—even if you stop to think about each word separately.”
Yorick nodded. “Now you’re beginning to understand. Most of them are just noises. I have to take it on faith that it means what I want it to mean.”"
Amusing dialog, and quite germane - Yorick claims to hold a cognitivist perspective (that meaning comes from 'handling symbols', see also [1]) while the final paragraph ironically expresses a healthy skepticism of cognitivism which (IMO) is justified: we all have to 'take it on faith' that things mean what we want them to mean. That is, as Wittgenstein famously pointed out, the meaning of utterances is given by their use and not by some mysterious psychical accompaniment or the grinding of symbolic gears in the brain:
> There must surely be a further, different connexion between my talk and N, for otherwise I should still not have meant HIM.
> Certainly such a connexion exists. Only not as you imagine it: namely by means of a mental mechanism.
- Philosophical Investigations §689
This is a drum I've been quietly banging for some time! [2],[3]
That’s tossing exactly the kind of question it’s really easy for ChatGPT to answer. Try something that isn’t in it’s model. This might seem difficult due to how extensive the training data is, but you can generally trip it up just by taking a slightly different take.
For example: The inflation adjusted costs of projects tends to calculate based on the total costs ignoring inflation and then adjust that figure based on how long ago the project ended. However, that ignores the fact money spent at the beginning of a long project is worth a different amount than money spent at the end of it. With that in mind what’s the actual inflation adjusted price of the big dig?
Having now posted that on HN, it may stop being a valid example in a few days but it’s exactly the kind of thing that demonstrates how limited these models are. Or for a more comical example: https://youtu.be/rSCNW1OCk_M
Honestly, yes, I don’t think it’s that far off. LLMs are a series of relatively simple transformers chained together. Through which we can simulate thought to the point that it not only passes the Turing test but it’s useful.
This is a bit of extrapolation but I would say the reason why we’ve been unable to locate “consciousness” in the brain is because it’s the same thing. Relatively simple neurones, chained together, to create thought.
On a philosophical level: this doesn’t make any claims for idealism or materialism, “experience” could exist at a more fundamental level of reality than matter. But IMO that would mean that the LLM is “experiencing” as well.
Of course developers can always tack on edge cases, but ChatGPT can’t for example handle beating a novel MUD from the 80’s. This isn’t about diminishing it’s accomplishments, just pointing out why the creators aren’t hailing it as AGI.
Either this is a missing layer, and we’ll get there soon, or one could say verification is happening through the statistical model. We need to see if we can train hallucinations out by having it rely on a stable data store to retrieve facts rather than trying to pull facts from the model itself. This is still similar to how the brain has discrete components for memory storage and retrieval.
> This is still similar to how the brain has discrete components for memory storage and retrieval.
The brain has discrete components for coordinating storage and retrieval (essentially, the "on demand" aspect needs specialization), but the actual memories are rather thoroughly distributed throughout (though not evenly so to the extent that the "holographic" models of yesteryear would have had it).