[pizza234]

> Now come back to my first question. An LLM predicts the next word based on all the words before it. That is the whole story. There is no idea sitting underneath.

No!! This is misinformation.

An LLM predicts the next words (tokens) based on the words before it *and* on the internal representations it has learned from training.

Processing language can lead to internal representations that capture patterns and relationships; LLMs can exhibit emergent abilities, including reasoning-like (stress on "-like") behavior, even when they weren't explicitly trained for that.

Making reductionist claims about LLMs, based on next-word prediction, is similar to making such claims about biology, based on amino acids.