[wren6991]

That's true. The fact that an LLM is a pure function of (all previous tokens) -> (next token), with internal state like KV cache only existing for optimisation purposes, is pretty mind-blowing.

I guess it was more the "predictor" part I had issue with. There's a tendency to reach for statistical or probabilistic terminology to describe things that aren't usefully understood in those terms. For example in the "Speed Always Wins" LLM technical survey (https://arxiv.org/pdf/2508.09834):

> The gate is a crucial component to bring sparsity in MoE models. For a batch of input token representations X ∈ RT×D, the gate function G determines the probabilities of dispatching token xi to each expert e

...which is nonsense: the gate simply, directly, selects the experts. There's nothing probabilistic about it.

[didibus]

> which is nonsense: the gate simply, directly, selects the experts. There's nothing probabilistic about it.

Isn't G a learned probability?

[wren6991]

I don't think so. Probability of what?

If you put something through a softmax the output is (trivially) a valid PMF. Does that matter? You're not sampling from it.