[win311fwg]

> Generally, compilers are deterministic

Not really. Maybe more so than they used to be thanks to deterministic algorithms becoming faster, but most compilers still aren't deterministic by default. There are still advantages to non-determinism in compilers, like not having to worry about thread execution order. To be fair, most compilers these days allow you to optionally enable determinism. Then again, LLMs also allow you to optionally enable determinism.

[KellyCriterion]

Thread-connected non-determinism is not what the author meant by "compilers are deterministic"

[win311fwg]

If you consider hidden inputs to be part of the input space then the statement is obviously true, but then all computation is deterministic. It is fundamentally impossible for computers to be anything but deterministic. Maybe that is what the author meant, but it seems unlikely as it doesn't add anything. Mentioning determinism usually implies that there is an non-deterministic alternative, which in computing can only mean where there are hidden inputs.

[foco_tubi]

> LLMs also allow you to optionally enable determinism

really? Please explain

[win311fwg]

What's in need of explanation? Computers are designed to be deterministic, and LLMs run on computers, so LLMs must also be deterministic.

There caveat to that is where external inputs give the illusion of non-determinism. Thread execution order, like we already discussed, is one such example. Technically still deterministic if you understand the external inputs, but for the sake of discussion we can consider external inputs to be non-deterministic. Which is why compilers usually end up being non-deterministic by default. However, computers are designed to be deterministic so there are ways to avoid introducing those external inputs, albeit often at the cost of performance. LLMs and compilers alike can be run deterministically.

LLMs have one additional property not typically found in a traditional compiler — a call to rand() — but rand() is also deterministic when configured with a constant seed and can also be turned off completely by setting temperature to 0.

[foco_tubi]

Ok, technically you’re right. You can make LLMs deterministic if you make literally every other aspect of execution deterministic as well - self-hosting and open-weights model with identical, reproducible logits, batching, FP math, etc.

[win311fwg]

Yup. Just as you can technically make compilers deterministic if you control all aspects of execution, but it isn't typical to do so. In fairness, reproducible builds have started to become considered increasingly important recently over concerns like supply chain attacks so attitudes are starting to change, which is why most compilers nowadays give you determinism as an option, but the historical view was that the user didn't really care what the compiler spit out, so there was rarely any effort to make them deterministic.

What's old is new again. LLMs are back to believing that the user doesn't really care what gets spit out, preferring performance over determinism, but it is likely that eventually we'll find increasing importance in reproducible builds again — especially when, like we saw with compilers, the hardware/algorithms catch up and the performance cost of determinism isn't there anymore. The industry is highly cyclical. These aren't hard technical limitations, just human choices mixed with a humorous dose of "this time is different". But in hindsight the times never end up being different, do they?