[tymscar]

I fully sympathise with this analogy and I think I have used it before myself. But there is a tremendous difference in practice. A compiler doesn’t produce randomly different code each time you run it, while an LLM, no matter how good will. At which point if something breaks, you have to take the reins.

[fzzzy]

Note that openai added the "seed" parameter to get deterministic results in the last release.

[tymscar]

That doesn’t help with the issue I put forward. Even if the seed is identical, the output is not deterministic based on the input. A tiny change in the input could result in no change in the output, a little change in the out, a medium change in the output, or a totally new output.

[marginalia_nu]

This assuming they'll keep providing access to the model you were using in perpetuity.