[bandrami]

Right but for an LLM non-determinism is the success mode, not the failure mode. By design you get different outputs when running the same inputs. Which, again, really contradicts the consensus that was widely held even just a year ago about how to do CI.

[XenophileJKO]

I mean you don't have to. With 0 temp, the only source of non-determinism is really the associative math chaning on accumulation of the floating point multiplication. That could be eleminated if we wanted to buy restricting the ordering.

[bandrami]

True but at that point you're not really using "an LLM" in the usual sense, you're using a very resource-hungry database.