[bob1029]

> The software you’re using to run your LLM, e.g. Ollama, vLLM, OpenAI, Anthropic, etc., is responsible for running this loop.

There's currently always this same man behind the curtain.

To me these are all effectively the same picture. The differences merely shuffle things around in the code that is invoking the LLM and handling its response.

Until we have something like CLR integration for ChatGPT, I don't see the significance of an inner loop agent.

Has anyone considered this model yet? E.g., shipping a DLL to the LLM provider that contains the actual implementations of the desired tool calls? Imagine how much easier it would be to provide your debugging symbols and XML doc files directly rather than re-documenting everything for some ever-shifting tool calling API surface.

[rahimnathwani]

Right. The core premise of this article makes no sense:

  what if LLMs executed tool calls without going back to the client

An LLM can't execute tools. The only thing an LLM can do is output tokens. It's up to the calling code to decide what to do with those tokens. If an API provider enables its own tools, that doesn't mean the LLM is executing its own tools. It just means the API provider is executing tools instead of the API caller.