[shireboy]

A maxim I’ve arrived at working w llm every day is “put deterministic things in code, non deterministic in llm”. I do wish the harnesses would be more helpful in this regard. For example I’ve seen tons of scheduled jobs that people wrote in Claude/copilot/etc that could easily have been scripts. They aren’t scripts because the author doesn’t know how to script and they stop at the point the llm does what they want. It isn’t hard to then tell the llm “make a script to do all of the tool calls this prompt needs and then pass the result back to the llm for this non-deterministic part”.

[agentdev001]

The way I try to illustrate this to my peers, in the context of automating with llms, is to "do as much of the deterministic work as possible before and after involving an agent". Tbf this is largely a restatement of your comment;

Ie instead of

"hey agent, use the api to get X, then do unpredictable mutation Y to it, and email it to me",

it should be;

"Scheduled task runs code to get X then place it in workspace. Task then starts agent in workspace and is prompted to do mutation Y to it. Post-flight code then sends final product."

The difference here being that the pre/post flight stuff is called deterministically, rather than called by an llm.

[firasd]

An absurd example is how harnesses like Codex, Claude code have no 'cut/paste this range of chars' affordance so if you want to move a block of code the LLM galaxy brain rewrites it in token space in the new file.

It ~can~ be done using standard bash utils but there just isn't a kind of standard pattern for 'move this text block via CLI tools' so the LLM and human both overlook it

[agentdev001]

Anecdotally, when I see coding agents preform this action- I see them using bash. IMO less tools is better, if the agent has a shell- so not having a dedicated cut/paste tool is good.

[jwally]

Preach!

Dedicated Infrastructure > automation > ad hoc scripts > non deterministic llms from a dependability perspective alone. Using an LLM to do something that prettier could have done is analogous to using a 3d printer to nail up a fence.

[j45]

I’d expect the llm to write the script and run it.

The tooling can remain generic and token spending focused.