[quaintdev]

So I can use this with any supported models? The reason I'm asking is because I can only run 1b-3b models reliably on my hardware.

[parthsareen]

Hey! Author of the blog post here. Yes you should be able to use any model. Your mileage may vary with the smaller models but asking them to “return x in json” tends to help with accuracy (anecdotally).

[pamelafox]

Do you happen to know if got-4o would be negatively affected by the addition of “return x in json”? I’m debating whether I could use the same prompt across all models, hosted and ollama.

[dcreater]

Why do smaller models fail to follow? Isn't the objective of constraints that it always provides the right output type?

[parthsareen]

The constraints will always be met. It’s the data inside that might be inaccurate. YMMV with smaller models in that sense.