| > This leads to the point: in general do we care about this non-determinism? > Most of the time, no we don't. well that’s a sweeping generalisation. i think this is a better generalised answer to your question. > It depends on the problem we’re trying solve and the surrounding conditions and constraints. software engineering is primarily about understanding the problem space. are 99% of us building a pacemaker? no. but that doesn’t mean we can automatically make the leap to assuming a set of tools known for being non-deterministic are good enough for our use case. it depends. > Once you accept that the next stage is accepting that most of the time the non-deterministic output of an LLM is good enough! the next stage is working with whatever tool(s) is/are best suited to solve the problem. and that depends on the problem you are solving. |
This seems irrelevant?
Either way hopefully you test the pacemaker code comprehensively!
That's pretty much the best case for llm generated code: comprehensive tests of the desired behaviour.