[healthworker]

This is strong evidence against LLMs experiencing qualia. (I know that that topic often gets people laughed out of the room but please don't jump on me for engaging in that debate. When we can collect evidence and be able to show it to people.)

[ordu]

How did you jumped to qualia from consciousness? I can see how this is a strong evidence against LLM being conscious, but to my mind it doesn't imply in any way or form that they do not experience qualia.

Or... well, ok, maybe they can't experience if they are not conscious? I see how this can be argued, but I still do not agree. I'm sure qualia is created not by consciousness (I would notice if it was), and I'm sure it is created not for consciousness specifically, it must have some other uses too.

[stevenhuang]

No disagreements with what you said in your first paragraph.

> I'm sure qualia is created not by consciousness

Whether or not qualia is created by consciousness, I don't see how we necessarily can tell one way or the other. We don't exactly have great introspective tools to do such self analysis, not to mention what we think we feel is often illusory/not reflective of reality.

[ordu]

Yeah, I think you are right. "The content of consciousness" and "consciousness" are different things, and I mixed them up, when writing that sentence. I can watch the content, but not other parts that may be producing it.

Moreover, thinking about it, I come to a conclusion, that if I cannot reflect on qualia creation, then it is a (weak) evidence for qualia created by consciousness. I suppose the consciousness is harder to reflect on than other things, hard to map it into states of the content of consciousness. Like, I can reflect on my vision and see some hints on how I get these wonderful pictures, despite it being definitely not consciousness, I can reflect on how I produce or decode language. And to my mind it is because consciousness was devised to reflect on these things, so I could report on my observations to others. But to reflect on itself is a wholly different matter.

[LoganDark]

We can't even meaningfully prove that human beings have qualia

[finghin]

We can’t meaningfully deny humans have qualia, there would be too much baby and not enough bathwater.

[LoganDark]

I would not deny it. The reason people accept current evidence is, after all, because they can relate to the experience of qualia, even if there's no complete objective understanding of it yet.

[vinceguidry]

We can't even meaningfully define the concept.

[satvikpendem]

How can qualia be experienced if not from a conscious observer? It's the same question as asking if a tree falls in the woods with no one to hear or know about it.

[layer8]

People have different notions of what “qualia” means. To me, the experience of qualia is the perception of qualities, or of the texture, of inner processings of the brain. Not consciously experiencing them doesn’t mean that these qualities aren’t there, just like not consciously experiencing sounds wouldn’t mean that the sounds aren’t there, and may be unconsciously processed.

I don’t think that “experience as such” makes any sense. Experience is always of something. And that in turn implies that the something that is being experienced also exists independently of it being experienced.

[jibal]

Since this is about what happens in humans, not LLMs, it couldn't possibly be evidence of any sort regarding LLMs.

(I think there are overwhelming reasons to think that LLMs don't experience qualia, but this has nothing to do with it.)

[MarkusQ]

Nuts.

You're saying the discovery that humans can process language without being conscious "couldn't possibly" inform the debate about LLMs? When that debate is literal predicated on the assumption that the ability to process language implies consciousness?

This is a counter example to the fundamental assumption of that argument. Without that, you are left with something like "if we ignore their ability to to process language, do we have any reason to suppose that LLMs (as opposed to, say, a spread sheet or stats package) are conscious?"

[jibal]

Sorry to hear that someone rudely thinks that basic logic is "Nuts".

> When that debate is literal (sic) predicated on the assumption that the ability to process language implies consciousness?

This is an incoherent claim. Debates are between people with differing claims and often differing assumptions; they aren't "predicated" on some assumption or another--that's a category mistake.

Someone can easily argue that LLMs are conscious (or have qualia--that was the disputed claim, and they aren't the same thing) without the strong claim that the ability to process language entails consciousness ... perhaps it is the processing of language together with other features that they think indicates consciousness. For instance, George Lemoine and Richard Dawkins didn't base their judgments on consciousness on such an entailment, but rather on the specifics of what the LLMs said to them.

I won't respond about this again.

[MarkusQ]

If LLMs did not process language as well as they do, we would not be having the argument.

The only reason we are having the argument at all is that people see LLMs responding appropriately to language, and _from_that_ conclude that LLMs may be conscious. You even sneak this in yourself when you say "George Lemoine and Richard Dawkins didn't base their judgments on consciousness on such an entailment, but rather on the specifics of what the LLMs said to them" -- in other words, they wouldn't have had judgements in the first place if the LLMs had not "said things to them".

[naasking]

I don't see how that follows. The brain could be experiencing all sorts of things while processing, but simply not record it, and so of course the person will have no recollection of experiencing anything.

[ordu]

An anecdote to demonstrate the point.

I broke my leg recently. Shortly after that I've lost my consciousness. It was very painful, the body reacted with a lot of adrenaline, and after a several minutes when adrenaline was drained away my consciousness was drained too.

I experienced something like this several times, though not to the point of fainting. But this time was special in other way too: I had friends near me, they observed me through all the process and we could compare our observations later. It seems, that my memory stopped recording before I fainted. I was still operating to some extent, but I couldn't remember a thing. When asked something I grunted in answer. When one of my friends insisted that I stand up and come to a better place to sit down, I actually stand up and did several steps before stopping and slowly (and carefully) sank to the ground. (An interesting observation, my controls over my body were weakening, but I was still using them for what they worth. It fits with all other similar experiences: the limbs and all the muscles seem to be losing their strength, and it takes a lot of will to make them work.)

On the light of this, I'm very interested what proponents of the idea, that feelings need consciousness to work, would say about my half-unconscious state. Did I feel myself extremely bad at the time? Or maybe I didn't feel anything? My friends are sure that the former statement is true, but they may be mistaken by my outside looks. I personally don't remember. Up to some point I remember that I felt really bad, but the next thing I remember I look at the sky and I'm surprised by what I see (I was not in a place I expected to be). And at that moment I was pretty ok already, no more adrenaline issues, just my leg was aching.

Was I experiencing qualia is another interesting question. I'm pretty sure I was, but I'd like to hear an argument for the opposite.

[satvikpendem]

Maybe it was a different part of your nervous system experiencing them, akin to a BIOS versus the operating system. The brain is a very complex and fractal thing, it is entirely possible that a more basal part of "you" took over for a very traumatic part of your life, very similar, but not exactly, to those with multiple personality disorder act.

[layer8]

> I'm very interested what proponents of the idea, that feelings need consciousness to work, would say about my half-unconscious state.

I’m not one of these proponents, but to play the devil’s advocate: The fact that you can’t remember it doesn’t necessarily imply that you didn’t fully consciously experience it at the time.

[Geste]

Go on, where is the evidence ? I am actually curious and open-minded about it.

[slopinthebag]

The Chinese room argument suggests that it's impossible anyways.

[finghin]

I don’t disagree with the general point here but the attraction of that argument has surely drifted much in ~5y.

I’d be REALLY curious to see a survey of philosophy 2015 vs 2025 UG entrants on mind-brain connection intuitions

[slopinthebag]

I feel like it might be strengthened by a machine that actually does implement the core of the argument (even passes the turing test?) but is also just matrix multiplication. Previously the idea of a device that could respond to input with human-like output was a fantasy, now it's reality which removes one of the arguments against the Chinese Room on the basis of plausibility.

But yes, I would like to see a modern philosophers take on it.

[finghin]

I'm more interested in whether the uninformed intuitions have changed, whether someone who used an LLM before graduating from school and entering university has a more forgiving stance than an "ignorant" college entrant from ten years ago. From my experience of the field, I would really doubt many philosophers of mind have changed their view based on this. The major questions of interest concern qualia, it ha been so for many years.

[jibal]

Plausibility was never an argument against Searle's Chinese Room Argument ... that's a very deep confusion, since Searle was arguing against the possibility of Strong AI, whereas advocates of Strong AI of course thought it was plausible and that's why they were working to create it. That you conflate the Chinese Room Argument with "the Chinese Room" is one aspect of the confusion.

[finghin]

It’s absolutely about the plausibility of mechanistic intentionality, given the implausibility of intentionality where the person in the experiment doesn’t understand chinese

[jibal]

Searle's argument may well be about the plausibility of "mechanistic intentionalty", whatever exactly that means ("mechanistic" just sounds like bigotry to me ... we are all mechanisms and Searle didn't say otherwise, just that we are meat mechanisms and not purely syntactic mechanisms ... and his argument was intended as a logical proof, not just a plausibility argument), but that's not what the previous comments were about. Apparently you mechanistically see the word "plausibility" and think that any and all statements about it refer to the same thing.

I won't respond further.

[nothinkjustai]

There are arguments made specifically about the implausibility of such a room making the argument itself invalid. Or I guess I should say “we’re”, because that position is now much less tenable with LLMs.

[jibal]

The Chinese Room argument is one of the most fallacious arguments ever offered and is rejected by all experts.

https://www.nybooks.com/articles/1995/12/21/the-mystery-of-c...

"For [John Searle's] part, he has one argument, the Chinese Room, and he has been trotting it out, basically unchanged, for fifteen years. It has proven to be an amazingly popular number among the non-experts, in spite of the fact that just about everyone who knows anything about the field dismissed it long ago. It is full of well-concealed fallacies. By Searle’s own count, there are over a hundred published attacks on it. He can count them, but I guess he can’t read them, [...]"

[slopinthebag]

> one of the most fallacious arguments ever offered and is rejected by all experts

Do you have any sources for the first claim? The second claim is of course trivially wrong.

[nothinkjustai]

Every significant argument has plenty of detractors, that doesn’t mean they’re right. Every argument I’ve heard to the contrary is unconvincing to me, and to the majority:

https://survey2020.philpeople.org/survey/results/5002

[stupidgeek314]

holy no sequitur batman!