[Der_Einzige]

There has been a huge explosion of awesome tooling which utilizes constrained text generation.

Awhile ago, I tried my own hand at constraining the output of LLMs. I'm actively working on this to make it better, especially with the lessons learned from repos like this and from guidance

https://github.com/hellisotherpeople/constrained-text-genera...

[rain1]

This looks incredible. Wow.

[killthebuddha]

I agree, it looks great. A couple similar projects you might find interesting:

- https://github.com/newhouseb/clownfish

- https://github.com/r2d4/rellm

The first one is JSON only and the second one uses regular expressions, but they both take the same "logit masking" approach as the project GP linked to.

[Der_Einzige]

I love the love from you two - I am trying right now to significantly improve CTGS. I'm not actually using the "Logitsprocessor" from Huggingface, and I really ought to as it will massively speed up inference performance. Unfortunately, fixing up my current code to work with that will take quite awhile. I've started working on it but I am extremely busy these days and would really love for other smart people to help me on this project.

If not here, I really want proper access to the constraints APIs (LogitsProcessor and the Constraints classes in Huggingface) in the big webUIs for LLMs like oogabooga. I'd love to make that an extension.

I'm also upset at the "undertooling" in the world of LLM prompting. I wrote a snarky blog post about this: https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...