[this_user]

Not really though, since the result of the prompt is not deterministic. It greatly depends on the model, the version, the harness, even time of day if the provider's infrastructure is currently overloaded and is silently degrading performance. Some things also require multi-turn interactions.

[kylemart]

100% agree with this. Was about to write the same thing.

[doctorpangloss]

this is the unintentional HN humor i come here for

[lelanthran]

> Not really though, since the result of the prompt is not deterministic. It greatly depends on the model, the version, the harness, even time of day if the provider's infrastructure is currently overloaded and is silently degrading performance. Some things also require multi-turn interactions.

All of that are even more reasons to reject AI-generated code: if it cannot be trusted to produce same (or even similar) output just from the prompts, why accept it at all?

[rat9988]

This is why I have been asking to fire product owners. Givven the same input, but different software engineers working with them, they can produce different output (code).

[Volundr]

It's worse than that! Even when the product managers inputs to the engineers are constrained for many problems no two engineers will produce consistent results! In fact recent studies suggest that even the same engineer may produce different results based on mood, if they've had coffee yet, and how near to EOD it is.