[fzysingularity]

They’re slightly nuanced - every model provider has a slightly different Pydantic /JSON schema compatibility (i.e for handling Literals, Unions, nested subtypes etc).

So you end up hitting roadblocks for seemingly simple Pydantic schemas.

[jasonjmcghee]

I meant between "structured output" and "function calling". Afaict one is outputting according to a schema and the other is outputting according to a schema... which will be used as the parameters to a function.

But they seem to be considered disparate concepts. So I'm trying to understand if there's some additional nuance I'm missing.

[fzysingularity]

Ah ok, I misunderstood. As far as I've seen, structured outputs is essentially "json-mode" with some constraints (i.e. guided decoding over a known schema) - so the model effectively emits valid JSON that conforms to the schema. In function calling, the model is asked to emit "code" that conforms to some function parameter spec. You could use json-mode for function-calling, but probably not the other way around.

I've generally found json-mode to be more useful than function-calling, even though the latter is what everyone fixates on because of it's obvious use in agents.

[jasonjmcghee]

I don't understand the difference based on your explanation (or the significance of "code") and have used function calling for outputting json according to a schema.

[guntars]

With function calls the model may or may not output something that matches the schema, with structured output the schema is enforced at the logit level.

[jasonjmcghee]

At least in the case of openai, you can set "strict" to "true" and function calling / tool use must / is enforced to follow the schema too.