[zahlman]

> It's a waste of time to think about whether an LLM has a subjective experience of reality... It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human condition, because we are entrusting Claude to make decisions and take actions that have real world consequences

First off: without taking for granted that an LLM "has a subjective experience of reality", all of those descriptors are meaningless. Second, there is no reason to suppose that Claude experiencing those qualia would actually impact on its "decision-making".

Third, text output is not a "demonstration" of emotion, nor is it evidence of the self-perception of the system, or of any self-perception. A printing machine that is actively churning out copies of Wagahai wa neko de aru (https://en.wikipedia.org/wiki/I_Am_a_Cat) is not a cat, and is not self-identifying as a cat, and is not self-identifying as anything, and is not expressing a thought, and is not conscious.

> Additionally, I'm not fully convinced that consciousness isn't built out of words

Do you suppose that, for example, insects are not conscious? Is the mooing of cattle a language?

[randallsquared]

> Do you suppose that, for example, insects are not conscious?

Not the OP, but: since there is no testable theory of consciousness, yet, I can't be sure, but my current assumption is that insects are not conscious, in the sense of there being someone implemented in insect hardware who experiences the world. That is, I would argue there is nothing it is like to be a bee, since there's no one being a bee.

I'm pretty sure there IS someone who is being me, at least much of the time.

[eks391]

> since there is no testable theory of consciousness

I can't argue with this, so I acknowledge that my interpretation is as bunk as anyone else's.

> my current assumption is that insects are not conscious

Some species act as hive-minds (like bees! How convenient for your example), so I imagine the hive as the consciousness, making each bee individually lacking conscious but collectively so. Like a single neuron is not consciousness alone, but the brain is... For some reason. Kinda like how you use different physics at different levels; Newtonian physics is always there, but negligible at quantum levels, so effectively not present at all. Even a human is a collection of minds, but only one conscious. My gut biome is independent biology and can even be removed and transplanted, but I don't believe my gut bacteria is conscious. So long as it is in me, it is nonetheless part of me, and I am one conscious. I also don't exist only at my brain, eyes, or hands (deaf/blind people have expressed to me that they feel like they are located at their hands in the way I used to think I was located behind my eyes), but as my whole body.

With this perspective, I still don't believe LLMs are conscious despite modeling thinking so well. At best, it is a highly accessible modeling software, like goat simulator but if it were so good that someone thought the goat was real. You are still steering the goat/LLM, and it doesn't exist when you aren't running it. I guess the missing piece for me is the lack of autonomy that a conscious has.

Then you can go into an argument on whether we actually have choice or it is an illusion, but that is a whole topic on its own.

[Kim_Bruning]

> Second, there is no reason to suppose that Claude experiencing those qualia

I'd argue the qualia question is a red herring. Functional Affect is a thing, regardless of ontological status. It's all fun and games until someone gets hurt.

To paraphrase Dijkstra: "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.". If you're building a navy: you care about displacement, propulsion, navigation and whether it can fire torpedoes. Whether your submarine has some "biological essence" of swimming is not really relevant to the fact that it is currently moving through the water and can collide with things. Turing also rejected the question "Can Machines Think" as posed, and replaced it with an operationalization (something else that we can actually usefully measure and work with).

To reiterate, functional affect is a concrete phenomenon. Whether or not there is a what-it-is-to-be is interesting in the abstract, but engineering a system means looking at how the inputs influence the outputs. A next token predictor working on a language that communicates affect needs to be able to predict affect or it is simply not going to be accurate. Given an 'angry' version of an input and a 'friendly' version of the same input, LLMs are likely to provide a different output, especially if there's a non-objective element. You can diff this.

Searle argues "A simulation is not the real thing", which is great and all... but if you hook up say an autopilot to the real world (as llms increasingly are) , you'd best hope the simulation was accurate in the first place (utterly regardless of where you stand on Searle).

Right now we're seeing situations where LLMs can be helpful or a real nuisance. Ignoring functional affect out of sheer ideology means you can't properly predict what they'll do, and that causes trouble, as we've already seen stories about.

This gets especially interesting when you start feeding the output back into the input (autoregression) , because now you have a highly non-linear dynamic system and you've introduced some amount of sensitivity to initial state. There's some interesting mathematical intuitions to be had there.