[justinjlynn]

Are you certain that you're not playing with words to arrive at a predetermined conclusion? What is this "I" to which you're referring and how can you demonstrate that "I" does not or cannot exist within systems such as these? Further, if you are to find something which qualifies as an "I" elsewhere, what makes that elsewhere fundamentally different and therefore capable of supporting and being an "I" and is that elsewhere such simply by definition or in and of itself? Further, if the language usage is indistinguishable from the language usage of an "I", is the difference of source meaningful? If so, why?

[skydhash]

The "I" is stemmed from the theory of the mind. We can only access our own mind and thus has no way to infer the thoughts of other. So we observe them and infer based on our own patterns. In a sense, we assume that others have the same mechanism that we possess, and thus we engage in interactions with them. So far, there is no demonstration of reasoning within systems such as these, it's all simulation of the communication channel themselves.

> Further, if the language usage is indistinguishable from the language usage of an "I", is the difference of source meaningful?

Is it indistinguishable? The first thing we look for in communication is consistency so that we can examine for intent. And this is after we determine the other party. Because we know the intent is not ours. But what I've seen of prompt engineering is that the communication intent always come from the person, not the models. Then it goes on to find the most likely continuation of this intent (based on the model training) and then it quickly become an echo chamber. It's search in lexical space and you can see the limits when it became a oscillating loop between the same set of reply. Because there's no "I don't know" damping.