Hacker News new | ask | show | jobs
by tmzt 27 days ago
It's basically restricting what logits are allowed when sampling the model to conform with the JSON (or whatever) shape. It can also cause the model to get "confused" though and doesn't always result in the output you want.