[visarga]

> The same input will always produce the same output

Not guaranteed even with the same seed. If you don't perform all operations in exactly the same order, even a simple float32 sum, if batched differently, will result in different final value. This depends on the load factor and how resources are allocated.

[simonw]

Yeah, the fact that floating point multiplication isn't associative is a real pain for producing deterministic outputs - especially when you're running massively parallel computations on GPUs (or multiple GPUs) making the order of operations even less predictable.

[taneq]

This doesn’t mean LLMs are inherently non-deterministic, just that current common implementations are non-deterministic.