[dist-epoch]

> all the non-determinism comes from the seed

not fully true, when using floating point the order of operations matters, and it can vary slightly due to parallelism. I've seen LLMs return different outputs with the same seed.

[onionisafruit]

That’s an interesting observation. Usually we try to control that, but with LLMs the non-determinism is fine.

It seems like that would make it hard to unit test LLM code, but they seem to be managing.

[kerkeslager]

Oh, that's really interesting. Hadn't thought of that.