[version_five]

> It's like saying a computer system is simply a turing machine executing simplistic instructions from a tape roll when such instructions can form things like games and 3D simulations of entire open worlds.

That's a bad analogy, none of those things are emergent behavior.

We can debate whether what an llm does is "emergent" - it's basically a definition thing though and isn't very interesting.

In reality, what's most surprising is that so much of what we say is explainable as next token prediction. It's not the other way around - we're showing how predictable we are, rather than how smart the AI is. But it's clear to me that it's in the outlying cases where the differences are. AI doesn't extrapolate outside it's training data, and even if it gets (100-\alpha)% of it's output right, there is always some alpha that's not in the training data and differentiates pattern matching or fancy key-value lookup (which is how we know AI works) from whatever intelligence is.

[dontmobile]

The analogy is about abstraction. It is not about emergent properties. A computer program is characterized differently when it's a 3D engine versus a a series of instructions.

Same with LLMs. We can characterize an LLM as a text predictor at the lowest level. But when the LLM gives me a novel response and solves a bug in my code, is text prediction really the only way to characterize that? Obviously there is a higher level analysis that we cannot fully comprehend yet.

In this case yes, the 3D engine is not an emergent property while the novel responses of an LLM are emergent. But this dichotomy is irrelevant to the analogy.