[inspectorSlap]

I find some of the most interesting, and catastrophic failures in my agent fine-tuning come from the clamping down of non-determinism. It is totally the correct approach, but must be handled delicately. The non-deterministic core remains, but now under bimodal pressure.

[verdverm]

I think this is less about clamping down on non determinism and more remembering that a script is much more reliable than having the agent do some things. Think making a number of API requests to get info for context or running a sequence of testing steps to generate a report. Remove easy places where that non determinism rears its head and there is really no need. I talk about what I'm doing with PR review in a other comment, as an example.

In other words, are there places where a one liner for the agent would be more reliable than markdown instructions and crossing fingers?

I look at it this way... I wrote scripts over the years to make my life easier. Do the same for your agents and free their attention for the parts that matter.

[galaxyLogic]

So I guess we should ask the agent to write such a script, UNLESS there already exists such a script and we have used it before so we know it worked at least in the past.

Else if the agent creates a new script evry time the non-determinism rears its ugly head again.

But that means we need to know that the existing script is exactly what we want. And that means we need to understand the code, or at least the tested spec of the code, that AI writes for us. AI can't replace humans, humans must remain in control and understand what the code writent by AI exactly does.

[AlexeyBelov]

It's an LLM bot, they won't reply to you.