[jacobgold]

Making systems fully deterministic ignores the entire purpose of having agents involved.

IMHO the best of both worlds option is agents working with deterministic CLIs. Where the agent does the reasoning (and text generation) but uses CLIs to carry out all of the actions (issuing refunds, unblocking accounts, or whatever).

It's possible to get very reliable and consistent work out of agents when they're using well written prompts with well designed CLIs.

[variety8675]

Isn't this how we end up with things like: https://www.reuters.com/legal/government/high-profile-meta-a...

[jacobgold]

Yes: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

Although you can certainly do a better-and-worse job of preventing these kinds of issues.

[camdenreslink]

I think it just depends on the workflow. A lot of times (probably the majority of the time) a well crafted deterministic process is what people want.

Sometimes you can have a multistep deterministic workflow with a decision that needs made somewhere in the middle, which is where an LLM that can call tools is useful.

I think there are much less repeated tasks where you would want a fully agentic process (but there are many ad hoc tasks where that is exactly what you want).

[bethekidyouwant]

How else would anyone do something like issue a refund if not through a programmatic interface?

[sqquima]

Direct access to the database, and create the "refund program" on the fly. Yes, stuff of nightmares.

[jcgrillo]

yes... ha ha ha... yes!

[bethekidyouwant]

Right thats just head cannon though. Unless of course you believe the lies you read on the Internet.

[jacobgold]

At some level everything an agent does is through a "programmatic interface" (tool calls).

Some people might use skill-based scripts, MCPs, or some kind of raw access to a database. My point is that well designed CLIs are the optimal programmatic interface, for many reasons.

[bethekidyouwant]

Sorry what other option is there? Is it going to create an API call from scratch every time after reading a page of documentation?

Wait raw access to the database? That’s one of the options for issuing a refund?

[cflewis]

Yes, it can do.

At Big Tech Company I Work At the LLM is quite happy to make raw API calls. If it thinks the data is big, then it'll write a Python tool to do it.

The reason crafted backing CLIs are useful is you can guide the LLM towards stuff that is immediately useful rather than hoping the nondetermism can separate the wheat from the chaff.

Take CI: is it interesting to know which tests passed? Maybe, but probably not. What is really interesting is what failed. Instead of having the LLM go out and talk directly to the CI system, write an intermediate CLI that filters out less actionable stuff by default, and have a flag that'll deliver the full dump if necessary.

It's a skill to do this stuff, and it's a lot of hard won experience than something I think is easily teachable. You kind of have to feel out your model and how it "thinks" about solving problems.

And then a new model version comes out and you have to learn it all again!

[kristiandupont]

>Making systems fully deterministic ignores the entire purpose of having agents involved.

That sounds backwards to me. I hope that most places don't see "having agents involved" as the ultimate purpose, but will use agents where it makes sense, i.e. when deterministic systems fall short.