[pu_pe]

We don't understand how humans think, and we don't yet understand completely how LLMs work. It may be that similar methods are being used, but they might also be different.

What is certain is that LLMs can perform as if they are doing what we call thinking, and for most intents and purposes this is more than enough.

[tempfile]

I think the evidence is actually pretty strongly against them doing anything similar to "thinking". Certainly they are exhibiting some behaviour that we have traditionally only associated with thinking. But this comes along with lots of behaviour that is fundamentally opposite to thinking ("hallucination" being the major example).

It seems much more likely that they are doing some other behaviour that only sometimes resembles thinking, in the same way that when you press the middle autocomplete button on your phone keyboard it only sometimes resembles conversation.

[groestl]

> "hallucination" ... "behaviour that only sometimes resembles thinking"

I guess you'll find that if you limit the definition of thinking that much most humans are not capable of thinking either.

[Yizahi]

You see, we are here observing a clash in the terminology. Hallucinations in humans is thinking, just not typical. So called "hallucinations" in LLM programs are just noise output, a garbage. This is why using anthropomorphic terms for programs is bad. Just like "thinking" or "reasoning".

[groestl]

I think the answer is somewhere in the middle, not as restrictive as parent, but also not as wide as AI companies want us to believe. My personal opinion is that hallucinations (random noise) are a fundamental building block of what makes human thinking and creativity possible, but we have additional modes of neuroprocessing layered on top of it, which filter and modify the underlying hallucinations in a way so they become directed at a purpose. We see the opposite if the filters fail, in some non-neurotypical individuals, due to a variety of causes. We also make use of tools to optimize that filter function further by externalizing it.

The flip side of this is that fundamentally, I don't see a reason why machines could not get the same filtering capabilities over time by adjusting their architecture.

[tempfile]

I have never in my life met a person who hallucinates in the way ChatGPT etc do. If I did, I would probably assume they were deliberately lying, or very unwell.

[vidarh]

> But this comes along with lots of behaviour that is fundamentally opposite to thinking ("hallucination" being the major example).

I find this an utterly bizarre claim given how prone humans are to make things up and firmly insist they did not.

[tempfile]

Is this really common behaviour? I do not recognise it. Do people lie? Certainly yes. Do people misremember, or get details incorrect? Yes. But when was the last time you saw someone, say, fabricate an entire citation in a paper? People make transcription errors, they misremember dates, and they deliberately lie. But I don't think people accidentally invent entire facts.

[vidarh]

To me, your entire claim here comes across as "hallucination". That is, I simply do not believe that you have not experienced people accidentally inventing entire facts, and so I don't believe you are genuinely unaware of people doing it.

To be clear, I'm not arguing you've made this claim in bad faith at all.

However, going back and examining my own writing, I have more than once found claims that I'm sure I believed at the time of making them, but that I in retrospect realise I had no actual backing for, and which were for that reason effectively pure fabrication.

An enduring memory of my school days was convincing the teacher that she was wrong about a basic fact of geography. I was convinced. I had also totally made up what I told her, and provided elaborate arguments in favour of my position.

To me this is innate human behaviour that I see on a regular basis. People accidentally invent entire "facts" all the time.

[fragmede]

What little of Fox News excerpted I've seen elsewhere doesn't support your claim.

[tempfile]

Fox News just lies. They aren't "hallucinating".

[vidarh]

What do you imagine the difference is?

[falcor84]

Indeed. The mere fact that we ended up with the anthropomorphic term "hallucination", rather than something purely mechanistic like "glitch", indicates that there's something about this AI pattern that feels familiar.

I'm obviously not claiming that "hallucination" is an appropriate term ("delusion" or "confabulation" are probably more apt), but there is something here that is clearly not just a bug, but rather a result of thinking being applied properly but to ungrounded premises. To my eyes, reading an AIs "hallucination" is not unlike reading the writings of a human on drugs, or with a mental condition like schizophrenia, or just of an analytic philosopher taking their made up axioms all the way to an alternate universe.

[throw310822]

> behaviour that is fundamentally opposite to thinking ("hallucination")

Did you just make this up?

[tempfile]

> Did you make this [opinion] up?

Yes! That is how they work.

[throw310822]

Can you please also hallucinate a plausible-sounding justification for this otherwise unsubstantiated statement?

Jokes aside, we do produce plausibile sounding stuff all the time well beyond the limit of what we actually know or can prove. I think there is a continuum between formulating statements about things we don't know for sure and we can't prove, guessing details here and there to fill gaps in our memory, misremembering things that we thought we knew, and making up entire facts that sound plausible but are completely invented. Yes, llms seem to have trouble introspecting what they actually know; but it sounds more like a missing skill rather than a fundamental difference in the way they reason.

[netdevphoenix]

I believe this is not a binary question, there is a spectrum. I think of LLMs as a sophisticated variation of a Chinese room. The LLMs are given statistical rules to apply to the given input and generate an output. The rules encode some of the patterns that we call thinking uses and so, some of their responses can be interpreted as thinking. But then, again, in certain conditions, the responses of mammals, unicellular organisms and even systems unrelated to carbon based life forms can be thought to be performing what we vaguely call thinking.

One problem is that we don't have a clear definition of thinking and my hunch is that we will never have a clear cut one as it falls in the same category of phenomena like alive/death states, altered states and weather systems. One hidden assumption that I often see implied in the usages of this word is that the word "thinking" implies some sort of "agency" which is another vague term normally ascribed to motile life forms.

All in all I think this debate ensues from trying to emulate something that we don't fundamentally understand.

Imagine in a world where medicine has not advanced and we lack any knowledge of human biology, we are trying to create artificial life forms by creating some heat resistant balloon and having it take in and push air. Someone would argue that the globe is alive because there is something in that taking in air and pushing it out that is like what humans do.

[Eisenstein]

The Chinese Room is just a roundabout way of pleading human exceptionalism. To any particular human, all other humans are a Chinese Room, but that doesn't get addressed. Nor does it address what difference it makes if something is using rules as opposed to, what, exactly? It neither posits a reason why rules preclude understanding nor why understanding is not made of rules. All it does is say 'I am not experiencing it, and it is not human, therefore I dismiss it'. It is lazy and answers nothing.

[ASalazarMX]

> The Chinese Room is just a roundabout way of pleading human exceptionalism

Au contraire, LLMs have proven that Chinese Rooms that can casually fool humans do exist.

ELIZA could be considered a rudimentary Chinese Room, Markov chains a bit more advanced, but LLMs have proven that given enough resources, LLMs can be surprisingly convincing Chinese rooms.

I agree that our consciousness might be fully explained by a long string of deterministic electrochemical reactions, so we could be not that different; and until we can fully explain consciousness we can't close the possibility that a statistical calculation is conscious to some degree. It just doesn't seem likely IMO right now.

Food for thought: If I use the weights to blindly calculate the output tokens with pencil and paper, are they thinking, or is it a Chinese Room with a HUGE dictionary?

[Eisenstein]

> ELIZA could be considered a rudimentary Chinese Room, Markov chains a bit more advanced, but LLMs have proven that given enough resources, LLMs can be surprisingly convincing Chinese rooms.

Eliza is not a Chinese room because we know how it works. The whole point of the Chinese Room is that you don't. It is a thought experiment to say 'since we don't know how this is producing output, we should consider that it is just following rules (unless it is human).

> Food for thought: If I use the weights to blindly calculate the output tokens with pencil and paper, are they thinking, or is it a Chinese Room with a HUGE dictionary?

Well, I never conceded that language models are thinking, all I did was say that the Chinese Room is a lazy way of concluding human exceptionalism.

But, I would have to conclude that if you were able to produce output which was coherent and appropriate, and exhibited all signs of what I understand a thinking system to do, then it is a possibility.

[moffkalast]

I don't know about you but we used to joke back in the day that the computer "is thinking" when taking long to process something.

Dictionaries usually provide some kind of useless circular definition. Thinking? The act of making thoughts. Thoughts? The result of thinking. I can't believe people used to pay for these things.

In any case it's something to do with taking input data, doing something with it, and generating new data related to it. That's more or less just recursive inference.

[IAmBroom]

And that is the essence of the Turing Test.