[my50cents]

> Since the interior state of another being can only be understood through interaction, no objective answer is possible to the question of when an “it” becomes a “who”

A large language model can in theory be understood at an algorithmic level by reverse engineering. If the algorithm turned out to be a giant lookup table, it's an "it". If the algorithm contained an obvious model of self, it's a "who".

[crooked-v]

> an obvious model of self

How do you determine what's an "obvious" model of self?

[messe]

> If the algorithm turned out to be a giant lookup table, it's an "it". If the algorithm contained an obvious model of self, it's a "who".

It seems to be that (at least a naive understanding of those two possibilities) leaves a large middle ground where an language model isn't a lookup table but doesn't have a model of self.

[robbedpeter]

We assume other people are "who" and not "it" because their behavior is predictably similar to our own, and so we model their experiences and perspectives based on our own subjective experience.

Occam's razor suggests that if we know other people have the same basic configuration as ourselves, then the subjective experiences of others will be more or less comparable to our own.

Additional assumptions are necessary to propose that Chalmersian/philosophical zombies could exist, having the same human hardware and behavior but lacking subjective experience. Under the principle of least complexity and the absence of evidence for alternate explanations, I think it's rational to assume the consciousness of other humans and certain animals with nearly 100% confidence.

That means something happening in the neocortex is causing consciousness. We know it's in the cortex because of the history of injury or absence of other parts of the brain, leaving us with empirical evidence. We know that hippocampus injury can result in a person losing the ability to remember more then 5 minutes of their past. We also know these individuals retain their personalities and the long-term memory from before their injury. This suggests that long term memory is encoded in the synaptic structure of the neocortex and that consciousness is an emergent result of neocortical operation.

There might be some involvement of particular brain regions or the thalamus or other organ, but it looks like the answer is part of whatever algorithm is encoded in the structure and processing of the neocortex.

It could be possible that consciousness doesn't require an explicit model of self. This is implied by ego death and other experiences by meditators and psychedelic users. The thing that is having a subjective experience could simply be a consequence of processing a particular configuration of information. The self concept seems to be a separate model that can be perceived as part of the process of awareness, but it seems to be a discrete thing.

To me that presents an interesting ethical question - if gpt-3 has a sense of self, then is it OK to subject it to the incoherent flashes of single moments of consciousness it undergoes each time you run it?

If it has no self concept, it might still have subjective experience, but would have no persistent contiguous experience . If it did have a self concept, then it would have a static past that informed the results of each run, as if you could isolate a single moment of awareness, then instantly reset the brain to a saved state and run it again. The output could be convincingly continuous, but the subjective experiences would be a myriad of similar but unrelated singularities of awareness.

I think gpt-3 lacks consciousness, and that a persistence mechanism is missing that plays a part in whatever is happening that causes awareness.

Hopefully, at some point, a m mathematician or scientist will be able to identify and explain the process of consciousness so that we can be reasonably certain we're not subjecting entities to a really weird tortured existence.