Hacker News new | ask | show | jobs
by whoknowsidont 212 days ago
>but isn't this solved by publishing good API docs, and then pointing the LLM to those docs as a training resource?

Yes.

It's not a dumb question. The situation is so dumb you feel like an idiot for asking the obvious question. But it's the right question to ask.

Also you don't need to "train" the LLM on those resources. All major models have function / tool calling built in. Either create your own readme.txt with extra context or, if it's possible, update the API's with more "descriptive" metadata (aka something like swagger) to help the LLM understand how to use the API.
1 comments
riverdweller 211 days ago
You keep saying that major models have "tool calling built in". And that by giving them context about available APIs, the LLM can "use the API".

But you don't explain, in any of your comments, precisely how an LLM in practice is able to itself invoke an API function. Could you explain how?

A model is typically distributed as a set of parameters, interpreted by an inference framework (such as llama.cpp), and not as a standalone application that understands how to invoke external functions.

So I am very keen to understand how these "major models" would invoke a function in the absence of a chassis container application (like Claude Code, that tells the model, via a prompt prefix, what tokens the model should emit to trigger a function, and which on detection of those tokens invokes the function on the model's behalf - which is not at all the same thing as the model invoking the function itself).

Just a high level explanation of how you are saying it works would be most illuminating.

link
cstrahan 211 days ago
The LLM output differentiates between text output intended for the user to see, vs tool usage.

You might be thinking "but I've never seen any sort of metadata in textual output from LLMs, so how does the client/agent know?"

To which I will ask: when you loaded this page in your browser, did you see any HTML tags, CSS, etc? No. But that's only because your browser read the HTML rendered the page, hiding the markup from you.

Similarly, what the LLM generates looks quite different compared to what you'll see in typical, interactive usage.

See for example: https://platform.openai.com/docs/guides/function-calling

The LLM might generate something like this for text:

    {
      "content": [
        {
          "type": "text",
          "text": "Hello there!"
        }
      ],
      "role": "assistant",
      "stop_reason": "end_turn"
    }
Or this for a tool call:

    {
      "content": [
        {
          "type": "tool_use",
          "id": "toolu_abc123",
          "name": "get_current_weather",
          "input": {
            "location": "Boston, MA"
          }
        }
      ],
      "role": "assistant",
      "stop_reason": "tool_use"
    }
The schema is enforced much like end-user visible structured outputs work -- if you're not familiar, many services will let you constrain the output to validate against a given schema. See for example:

https://simonwillison.net/2025/Feb/28/llm-schemas/

https://platform.openai.com/docs/guides/structured-outputs

link