[JoshMandel]

FWIW, I've seen stronger performance from gpt-4-1106-preview when I use `response_format: { type: "json_object" },` (providing a target typescript interface in context), vs the "tools" API.

More flexible, and (evaluating non-scientifically!) qualitatively better answers & instruction following -- particularly for deeply nested or complex schemas, which typescript expresses very clearly and succinctly.

Example from a hack week project earlier this month (using a TS-ish schema description that's copy/pasted from healthcare's FHIR standard): https://github.com/microsoft-healthcare-madison/hackweek-202...

Or a more complex example with one model call to invent a TS schema on-the-fly and another call to abstract clinical data into it: https://github.com/microsoft-healthcare-madison/hackweek-202...

[minimaxir]

For posterity, this is the "JSON mode" mentioned at the bottom of the post.

The docs say it's on by default if you use function calling normally: https://platform.openai.com/docs/guides/text-generation/json...

> Note that JSON mode is always enabled when the model is generating arguments as part of function calling.

[JoshMandel]

Yes -- the distinction with "function calling" is that you have to play a game of telephone where you describe your target schema in JSON Schema (only, apparently, for OpenAI to turn into a typescript interface internally) vs describing it more directly and succinctly (and with opportunities to include inline comments, order fields ordered however you want, and use advanced TS features... or even use an adhoc schema "language").

[msp26]

Yeah I was using hacky workarounds like this to get Object[] types in my schema. The API seemed to throw a fit if I did this in any other way.

``` {"type": "array", "items": {"type": "object", "properties": {"object": {"type": "object"}}}} ```

Somehow the thought to just write the typescript myself never occurred haha.

[o_____________o]

fwiw JSON Schema allows for $comment fields, which seem effective in OAPI function calls.

https://json-schema.org/understanding-json-schema/reference/...

[lukasb]

What do you use $comment for that doesn't work in "description"?

[bugglebeetle]

Something’s been broken with their JSON mode/function calling since the Dev Day launch. I have a bunch of scripts that stopped consistently returning JSON when I swapped in the gpt-4-1106 preview and gpt-3.5-turbo has gotten similarly erratic. They really need to pause all the “move fast and break things” stuff because their API and services like ChatGPT are becoming increasingly unreliable.

[joshuanapoli]

I still have a lot of trouble using OpenAI function-calling API (and json mode) for returning source code. I have trouble when the source code has quotes, which is pretty common. The result may be missing quotes, have incorrectly escaped quotes, or use of the wrong type of quotes at the JSON object level.

[visarga]

Same for YAML, you got to be careful with strings

[XenophileJKO]

So something I have also noticed, mostly on 3.5-Turbo, is textual responses in json take a quality hit, full stop. This has caused me to use mixed output usually. Thoughts and process in json, then "exit" to text for a conversational response.

It is likely also a behavior in gpt-4, but I haven't studied it as closely.

[msp26]

Thanks for the example! This does seem a lot more flexible.