[foco_tubi]

Ok, technically you’re right. You can make LLMs deterministic if you make literally every other aspect of execution deterministic as well - self-hosting and open-weights model with identical, reproducible logits, batching, FP math, etc.

[win311fwg]

Yup. Just as you can technically make compilers deterministic if you control all aspects of execution, but it isn't typical to do so. In fairness, reproducible builds have started to become considered increasingly important recently over concerns like supply chain attacks so attitudes are starting to change, which is why most compilers nowadays give you determinism as an option, but the historical view was that the user didn't really care what the compiler spit out, so there was rarely any effort to make them deterministic.

What's old is new again. LLMs are back to believing that the user doesn't really care what gets spit out, preferring performance over determinism, but it is likely that eventually we'll find increasing importance in reproducible builds again — especially when, like we saw with compilers, the hardware/algorithms catch up and the performance cost of determinism isn't there anymore. The industry is highly cyclical. These aren't hard technical limitations, just human choices mixed with a humorous dose of "this time is different". But in hindsight the times never end up being different, do they?