[the_mitsuhiko]

I get decent JSON from it quite well with the "assistant: {" trick. I'm not sure how well trained it is to do JSON. The template on ollama has tools calls so I assume they made sure JSON works: https://ollama.com/library/mistral-small:24b/blobs/6db27cd4e...

[a_wild_dandan]

And for anyone looking to dig deeper, check out "grammar-based sampling."

[azinman2]

What’s the “assistant: {" trick? You just end your prompt with that?

[simonw]

Mistral supports prefixes: https://docs.mistral.ai/guides/prefix/

[azinman2]

That’s cool. However it only shows a few odd. I’d imagine the model needs to explicitly support this (be trained with it). None are about json… do you use that trick with json?

[simonw]

I use it to get JSON pretty often. See also: https://twitter.com/simonw/status/1885091289554968975