[cjtrowbridge]

Entropy is also added via a random seed. The model is only deterministic if you use the same random seed.

[HarHarVeryFunny]

I think you're confusing training and inference. During training there are things like initialization, data shuffling and dropout that depend on random numbers. At inference time these don't apply.

[jampekka]

Decoding (sampling) uses (pseudo) random numbers. Otherwise same prompt would always give the same response.

Computing entropy generally does not.

See e.g. https://huggingface.co/blog/how-to-generate

[HarHarVeryFunny]

Sure - but that's not the output of the model itself, that's the process of (typically) randomly sampling from the output of the model.

[throwaway314155]

Right, sampling from a model, also known as *inference* (for LLM's).

The inference here is perhaps less pure than what you refer to but you're talking to human beings; there's no need for heavy pedantry.