[sigh_again]

Unless you have full access to the entirety of their training data, you can try to convince all you want, but you're just grasping at straws.

LLMs are stochastic parrots incapable of thought or reasoning. Even their chains of thoughts are part of the training data.

[xpe]

When combined with intellectual honesty and curiosity, the best LLMs can be powerful tools for checking argumentation. (I personally recommend Claude 3.5 Sonnet.) I pasted in the conversation history and here is what it said:

> Their position is falsifiable through simple examples: LLMs can perform arithmetic on numbers that weren't in training data, compose responses about current events post-training, and generate novel combinations of ideas.

Spot on. It would take a lot of editing for me to speak as concisely and accurately!

[555watch]

Your use of the word stochastic here negates what you are saying.

Stochastic Generative models can generate new and correct data if the distribution is right. Its in the definition

[xpe]

> you can try to convince all you want, but you're just grasping at straws.

After coming back to this to see how the conversation has evolved (it hasn't), I offer this guess: the problem isn't at the object level (i.e. what ML research has to say on this) nor my willingness to engage. A key factor seems to a lack of interest on the other end of the conversation.

[xpe]

Most importantly, I'm happy to learn and/or be shown to be mistaken.

Based on my study (not at the Ph.D. level but still quite intensive), I am confident the comment above is both wrong and poorly framed. Why? Seeing phrases "incapable of thought" and "stochastic parrots" are red flags to me. In my experience, people that study LLM systems are wary of using such brash phrases. They tend to move the conservation away from understanding towards combativeness and/or confusion.

Being this direct might sound brusque and/or unpersuasive. My top concern at this point, not knowing you, is that you might not prioritize learning and careful discussion. If you want to continue discussing, here is what I suggest:

First, are you familiar with the double-crux technique? If not, the CFAR page is a good start.

Second, please share three papers (or high-quality writing from experts): one that supports your claim, one that opposes it, and one that attempts to synthesize.

Third, perhaps we can find a better forum.

[xpe]

I'll try again... Can you (or anyone) define "thought" in way that is helpful?

Some other intelligent social animals have slightly different brains, and it seems very likely they "think" as well. Do we want to define "thinking" in some relative manner?

Say you pick a definition requiring an isomorphism to thoughts as generated by a human brain. Then, by definition, you can't have thoughts unless you prove the isomorphism. How are you going to do that? Inspection? In theory, some suitable emulation of a brain is needed. You might get close with whole-brain emulation. But how do you know when your emulation is good enough? What level of detail is sufficient?

What kinds of definitions of "thought" remains?

Perhaps something related to consciousness? Where is this kind of definition going to get us? Talking about consciousness is hard.

Anil Seth (and others) talks about consciousness better than most, for what it is worth -- he does it by getting more detailed and specific. See also: integrated information theory.

By writing at some length, I hope to show that using loose sketches of concepts using words such as "thoughts" or "thinking" doesn't advance a substantive conversation. More depth is needed.

Meta: To advance the conversation, it takes time to elaborate and engage. It isn't easy. An easier way out is pressing the down triangle, but that is too often meager and fleeting protection for a brittle ego and/or a fixated level of understanding.