Hacker News new | ask | show | jobs
by jasonjmcghee 491 days ago
I've used "structured output" (with supplied schema) on Google and openai, and function calling / tool use on those, anthropic and others- and afaict they are functionally the same (if you force a specific function / schema). Has someone had a different experience?
2 comments
fzysingularity 491 days ago
They’re slightly nuanced - every model provider has a slightly different Pydantic /JSON schema compatibility (i.e for handling Literals, Unions, nested subtypes etc).

So you end up hitting roadblocks for seemingly simple Pydantic schemas.

link
jasonjmcghee 491 days ago
I meant between "structured output" and "function calling". Afaict one is outputting according to a schema and the other is outputting according to a schema... which will be used as the parameters to a function.

But they seem to be considered disparate concepts. So I'm trying to understand if there's some additional nuance I'm missing.

link
fzysingularity 491 days ago
Ah ok, I misunderstood. As far as I've seen, structured outputs is essentially "json-mode" with some constraints (i.e. guided decoding over a known schema) - so the model effectively emits valid JSON that conforms to the schema. In function calling, the model is asked to emit "code" that conforms to some function parameter spec. You could use json-mode for function-calling, but probably not the other way around.

I've generally found json-mode to be more useful than function-calling, even though the latter is what everyone fixates on because of it's obvious use in agents.

link
jasonjmcghee 491 days ago
I don't understand the difference based on your explanation (or the significance of "code") and have used function calling for outputting json according to a schema.
link
guntars 491 days ago
With function calls the model may or may not output something that matches the schema, with structured output the schema is enforced at the logit level.
link
jasonjmcghee 491 days ago
At least in the case of openai, you can set "strict" to "true" and function calling / tool use must / is enforced to follow the schema too.
link
potatoman22 491 days ago
The model might not use the tools every completion, depending on your setup.
link