[abshkbh]

Morden software consists of separation of responsibilities between services and a higher plane orchestrating the data flow for business logic.

If you believe there is value in fuzzy tasks being done by LLMs then from that it follows that having separate "agent" services with a higher order orchestrator would be required. Each calling LLMs on their own inside.

[simonw]

I don't buy it. Why would I want my LLM to talk to some other LLM and introduce even more space for weird, non-deterministic bugs when I could have my LLM call a deterministic API to achieve the same thing?

[octopoc]

Isn't an agent just a system prompt + specific tools? Why not just switch out the system prompt and tools in the same context?

[IanCal]

No.

You have anything else that modifies the context, tools, model, and most importantly perhaps the iteration that controls what's going on with those other values.

[IanCal]

Why do you assume there's a deterministic API doing the same thing?

[simonw]

Because if a company built an LLM that can perform actions, they almost certainly did that by building an API first for it to use as a tool.

[IanCal]

But so much more besides that, including the model itself, RAG, the agentic workflow control, moderation, etc. There's also a huge factor of maintenance, that's a key reason why companies have different internal and external APIs - they don't just open up everything internal and hand the code for managing all of that to you. Interface design is really important.

Not to mention the cost being a factor here - who pays for which part.

[simonw]

Offering up an LLM-fronted "agent" for people to send their LLMs to talk to feels a whole lot more expensive and complicated to me than operating a traditional API endpoint.

[IanCal]

A traditional API endpoint wrapping an agent? That's pretty much what this is but as a standard so we don't need to build thousands of them.

[phillipcarter]

Because there often is?