|
|
|
|
|
|
by int_19h
852 days ago
|
|
|
It's pretty easy to force a locally running model to always output valid JSON: when it gives you probabilities for the next tokens, discard all tokens that would result in invalid JSON at that point (basically reverse parsing), and then apply the usual techniques to pick the completion only from the remaining tokens. You can even validate against a JSON schema that way, so long as it is simple enough. There are a bunch of libraries for this already, e.g.: https://github.com/outlines-dev/outlines |
|
|