[IanCal]

But so much more besides that, including the model itself, RAG, the agentic workflow control, moderation, etc. There's also a huge factor of maintenance, that's a key reason why companies have different internal and external APIs - they don't just open up everything internal and hand the code for managing all of that to you. Interface design is really important.

Not to mention the cost being a factor here - who pays for which part.

[simonw]

Offering up an LLM-fronted "agent" for people to send their LLMs to talk to feels a whole lot more expensive and complicated to me than operating a traditional API endpoint.

[IanCal]

A traditional API endpoint wrapping an agent? That's pretty much what this is but as a standard so we don't need to build thousands of them.