[verygoodname]

Would you say that a "Markov chain"-type (e.g. Dissociated Press) language model is "self-aware" in any way?

If yes, then... how, exactly? It is basically an N x N matrix of values.

Current language models (GPT et al.), are qualitatively nothing fancier than this: probabilistic models which encode regularity in language and from which you can sample to get "plausible content".

If a bunch of values in a matrix is "self-aware", then I guess GPT can be seen as "self-aware"; if not, then it can't.

My problem is trying to imagine an N x N matrix being self-aware (like... what does that even mean in this context?).

Is GPT human-like? Sure... if you stretch the meaning of "human-like" enough (it produces content that is similar to content produced by a human). Is a human GPT-like? That's harder to argue (and I don't see how your argument would support it).

[magusdei]

If you know what a Markov chain is then you must also know that modern language models are nothing like Markov chains. Just as an example, a Markov chain can't do causal reasoning or correctly solve unseen programming puzzles, the way GPT-3 can.

As for self-awareness, your brain is an N x N -matrix in the same sense as an ANN, so surely it must be possible for one to be self-aware? Not claiming that GPT-3 is, of course.

[verygoodname]

> If you know what a Markov chain is then you must also know that modern language models are nothing like Markov chains. Just as an example, a Markov chain can't do causal reasoning or correctly solve unseen programming puzzles, the way GPT-3 can.

First, GPT-3 can hardly do "causal reasoning" beyond exploiting regularities in the content it has been trained on. That's why you get "interesting causal reasonings" such as "the word 'muslim' co-occurs a lot with 'terrorist', so these things must be causally related".

Second, just because a more complicated version of a Markov chain (i.e. a probabilistic language model) can do things that a first-order Markov chain cannot does not mean that it is qualitatively different: both things are nothing more than simple mathematical models (linear algebra + sprinkles). A polynomial model can do things than a linear model cannot, but it is no closer to consciousness and self-awareness, as far as I can tell.

My point is... a mathematical model is just that (a model) and, as such, cannot be "self-aware".

Humans, as cybernetic agents embedded in some environment (from which they get sensory input and with which they can interact), have agency and, as such, can display the property of "self-awareness". Models, by themselves, cannot.

Humans may contain systems within them that resemble a GPT-3 language model (or a Markov chain model), and such "language models" may even be required for self-awareness (it's not obvious, but let's assume it's true).

*Still*, it is not the language model itself that is self-aware, but the agent that is using the model.

A GPT-3 model, by itself, cannot be self-aware, because it not much more than a applying some linear algebra operations on numbers (just like e.g. a Markov chain language model). An agent containing (among many other things) something that could be approximated by a GPT-3 model within, may.

> As for self-awareness, your brain is an N x N -matrix in the same sense as an ANN, so surely it must be possible for one to be self-aware? Not claiming that GPT-3 is, of course.

No, it is not. My brain is an analog computation device and not a bunch of numbers. Perhaps you can approximate some aspects of how it works using numbers, probably using digital computation devices, but that is not what anyone's brain is (neither at the "hardware" level, nor at the "software" level). Also, notice that a brain always exists within a biological agent, and it is the biological agent that is (or may be) self-aware, and not "the brain".

[magusdei]

Sorry that my answer comes so late, but I'll put this here for posterity. I will only address the latter part. My point was that a brain is an N x N -matrix in the same sense as an ANN. An ANN is no more an N x N -matrix "in reality" than a biological brain; in reality it is some collection of analog electric potentials and configurations of matter which can sometimes be represented as a digital N x N -matrix for the convenience of the programmer. Thus the situation is exactly identical to a biological brain, which is also not "in reality" an N x N -matrix but can be represented as one. If we had sufficiently advanced (nano-)technology, we could manipulate human brains through their abstract representation as a matrix just as we can ANNs. Any distinction is purely pragmatic.

In any case I was not saying that being representable as an N x N -matrix is sufficient for consciousness (which I do not believe), simply that it is clearly compatible with consciousness. I agree that a self-aware ANN would probably require a body (possibly simulated) and some notion of agency.