Fingers crossed OpenAI / Anthropic / etc do this! Would make working with these APIs for prediction projects that much easier.
Technically speaking it's pretty to force the model into an valid JSON-schema if you have access to the inference autoregressive loop and the logit activations. You can either force known areas of the template to a prefixed template, or fill in the basic JSON wrapper and force the model to choose arbitrary keys and values.
Imagine it might come down to how much of their usage is on generating text vs generating structured prediction payloads.
A lot of their value is turning unstructured text or user intent into the structured data. Even if the primary input is JSON, you want to explain what to do with it in words, and then get JSON back.
Why do you think they will be obsolete in 6 weeks?
Technically speaking it's pretty to force the model into an valid JSON-schema if you have access to the inference autoregressive loop and the logit activations. You can either force known areas of the template to a prefixed template, or fill in the basic JSON wrapper and force the model to choose arbitrary keys and values.
Imagine it might come down to how much of their usage is on generating text vs generating structured prediction payloads.