[gmt2027]

One of the most powerful ways to integrate LLMs with existing systems is constrained generation. Libraries such as outlines[1] and instructor[2] allow structural specification of the expected outputs as regex patterns, simple types, jsonschema or pydantic models. Llama.cpp supports bnf grammars.

These outputs often consume significantly fewer tokens than chat or text completions.

[1] https://github.com/outlines-dev/outlines

[2] https://github.com/jxnl/instructor

[Der_Einzige]

I wrote a whole paper about this idea but for syntactic or phonetic constraints:

https://paperswithcode.com/paper/most-language-models-can-be...