[gpm]

Computable doesn't really make sense here IMO, as you say, consciousness is not the illusion, it's the audience, it's the thing receiving the output not just an evaluation of a mathematical function.

The better question is why couldn't a consciousness attach itself to (be the audience for) a computation. Since we really don't understand anything significant about it, questions like this are next to impossible to disprove. At the same time since we've never seen anything except human start talking about consciousness spontaneously* it seems like a reasonable guess to me that LLMs/the machines running them are not in fact conscious simply because of their dissimilarity and the lack of other evidence.

* I note LLMs did not do so spontaneously, they did so because they were trained to mimic human output which does so. Because we fully understand the deterministic process by which they started talking about consciousness (a series of mathematical operations), them doing so was an inevitability regardless of whether they are conscious, and as such it is not evidence for their consciousness.

[jstanley]

> it's the thing receiving the output not just an evaluation of a mathematical function.

how do you know it's not just an evaluation of a mathematical function?

[gpm]

The definition is wrong.

A mathematical function is a set, possibly infinite, of pairs of abstract elements (commonly defined via sets) where no two pairs share the same first element. Nothing less, nothing more.

Computation is the act of determining the abstract output (second element in the pair) for a given abstract input (first element in the pair).

Nothing in those definitions is capable of expressing the concept of having perceptions (consciousness). That's not an abstract thing.

This isn't to say the concrete thing doing the computation couldn't in principal be conscious, just that it doesn't definitionally make sense for the math itself to be conscious.

[layer8]

A camera perceives its surroundings (add some image recognition to that). An LLM perceives its inputs. Things like that can be fully mathematically described. The question is, how is consciousness different from that? Introspecting, all there is is perception, arrows pointing to the things being perceived, where “things” includes some of the arrows themselves. Is there anything else? What prevents any of this to be fully mathematically described?

[gpm]

> The question is, how is consciousness different from that

A consciousness experiences perceptions, if you don't, I won't be able to describe this to you. If you do, it should be clear what I mean by that.

We have no evidence that either a camera or a GPU executing an LLM experiences perception. Certainly they react to physical stimuli, but so does an atom, physical reaction is not the definition of experience I am referring to when I say perception. We also have no evidence that they do not, except for the lack of evidence to the contrary.

We have some reason to believe that other people do experience perception, in that they spontaneously describe experiencing things that our similar to our experiences, and it's surprising they do that if they don't also experience things*. When I say "we", I really mean "I", but I'm assuming that you have the same experience I do.

> What prevents any of this to be fully mathematically described?

There's nothing that says you can't, in principle, create an entirely accurate mathematical description of perception (in the experiencing and not the reacting sense) where you define that certain abstract variables correspond to certain perceptions and can entirely predict them. The model would still be that, a model that predicts what perceptions occur, not the perceptions themselves. The same way mathematically describing a particle of hydrogen doesn't create a particle of hydrogen. The common concrete example is that mathematically describing what color someone perceives when looking at something, while basically possible, gives absolutely no insight in to what that experience is like (apart from saying "it's similar to <this experience> had by the same consciousness").

* See my other comment in this thread for why this argument does not apply to GPUs running LLMs.

[layer8]

The problem I'm trying to hint at is that we are unable to specify what the distinguishing properties of "experience" are supposed to be. It always ends up being self-referential. So maybe it is just a self-referential thing, a system perceiving parts of itself (mixed in with other things it perceives). Perceiving perception as such is trivial, e.g. I can have a system monitoring its own image processing pipeline, or I could feed information about the inner intermediate states of an LLM's neural network back into the LLM as part of its token input. The question is, how is "experiencing perception" any different? I'm not sure it is. The fact that I name part of my inner perceptions "feelings" or "qualia" or "texture", is largely immaterial to the question. They are objects of perception just the same, just originating from the inside rather than from the outside. I also disagree about the color example. I don't see how the experience of what a color feels like, or any insight related to that, could not be described mathematically in detail. Maybe it takes a bazillion of synaptic states that would need to be captured in that description, but that doesn't pose any conceptual obstacle.

[jstanley]

I agree with everything you said up to "Nothing in those definitions is capable of expressing the concept of having perceptions (consciousness)".

Do you think the universe is not computable?

If you think the universe is computable, and you think that you exist in the universe, and you think that you are conscious, don't you think it follows that consciousness can exist within mathematical structures?

[gpm]

> Do you think the universe is not computable?

Yes, definitionally not, the universe isn't an abstract object let alone one in the shape of a function.

You might, in principle, be able to precisely predict the future of the universe given perfect information using a precise model of the universe. That model, a mathematical function, would be computable. It would be accurate to say that the model describes the universe, but not that the model is the universe.

The thing about mathematical structures is that they are concepts, not things, I feel confident in saying that concepts aren't conscious.

[jstanley]

But you are inside the universe. The mathematical structure of the universe can contain all of your perceptions and to you they will "feel real". Indeed, to you they are real.

If you had a perfectly accurate universe simulation, do you think the people inside the simulation would not be conscious?

If they're not conscious, it's not a perfectly accurate simulation.

And if it is possible to have a perfectly accurate simulation, then (like you said) all of the contents of the universe were "there" all along inside the giant mathematical structure. You don't need anyone to run the simulator!

All of the contents of the universe, the apparent flow of time, our thoughts and feelings, our consciousness, all lives inside this incomprehensibly large mathematical structure.

This is how I believe reality works. The universe exists inside mathematics the way 42 does. You don't need a calculator to show the number 42 in order for the number 42 to exist. Running the simulator can expose the contents of the universe to someone outside it, but everything on the inside is independent of the simulator.

You might ask "why this reality and not any other?" and I would say they all exist equally well, we just happen to notice this one because we're inside this one.