[physicles]

For humans, the notion of “self” is also an illusion. We’re just atoms dancing to physics. But that’s not the right level of abstraction from which to do psychology.

Is the notion of an LLM operating on a document useful for building an intuition about how the LLM operates? That’s the more interesting question.

[jdonaldson]

I’d say it is, although it’s not really a document but a stream of tokens. The intuition is that the model is able to build abstractions from that stream. Some of the abstractions are related to goal states of the training metric. However, it’s easy to imagine how some of the goal states pertain to the model’s survival in the “wild”, and part of the abstraction that the model captures for this goal is a reflection of itself.