[inaciom]

Great question!

The return value is not guaranteed to be correct. REASON uses function calling to force the model to return a valid JSON object; however, sometimes models hallucinate.

For OpenAI models that have JSON mode, REASON uses that instead.

However, it has an internal validation step where it checks the return value, and if it's not correct, it'll throw an error (this behavior can be disabled though).

Here's more info about it: https://docs.tryreason.dev/docs/essentials/reason#will-the-l...

Your suggestion to use logits is super interesting, though! And it sparked this line of thinking: since REASON uses function calling, it has been somewhat difficult to integrate with OSS models — however, if under-the-hood we use Jsonformer, that might be the missing key for incredible OSS model support (?). What do you think?

[quickthrower2]

I took a very quick glance at Jsonformer - it looks like the right tool here. If you are looking for a PR, I can help, but might take me a bit of time to get oriented.

I am considering using RΞASON for a side project so interested in it.

[inaciom]

Awesome! That'd actually be great — just a warning that the repo is a complete mess right now.

If you need any help, feel free to contact me either on Github or in my email ([e-mail]).