[sunshadow]

What are the benefits over https://github.com/microsoft/guidance/ ?

[sandkoan]

We enable conforming to arbitrary context free grammars in addition to regex patterns, and have a bunch of speed optimizations, as well.

Though it may not seem too fast right now on account of the hundreds of simultaneous requests we're getting :)

[swyx]

100s of RPS? did you have a successful launch elsewhere or something? bc this repo currently only has 21 stars, which taking normal correlation into account does not imply that level of traffic

[sandkoan]

We have folks playing around with it mostly through the playground / raw HTTP endpoints as opposed to the Python API. And we've got some batch jobs running, which adds further traffic.

[behnamoh]

guidance is a dead project. it worked well as a hobby side project by MS researchers, but it clearly isn't a long-term solution as new LLMs are introduced.

[darkteflon]

Would you be able and willing to provide a bit more supporting evidence for this statement? We’ve been considering trialing guidance for a few weeks but won’t bother if it’s going nowhere.

[nestorD]

I am unclear on the status of the project but here is the conversation that seem to be tracking it: https://github.com/microsoft/guidance/discussions/201

[darkteflon]

Okay interesting, thanks. There does seem to be some recent activity on the `pythonic` branch but yeah, it does look like the open issues have been going ignored for a while now.

[IanCal]

If it helps I tried using it and the basic examples just straight up didn't work at all and regularly got broken in different ways. Even if it's going somewhere it was unusable for a serious project. I moved to custom code and left it on the "watch this project" list for the future.

[darkteflon]

It does help, thank you. We’re going to put it on pause for now, too.

[amkkma]

What about LMQL?

[darkteflon]

I hadn't heard of LMQL before - have you tried building with it and, if so, would you be willing to share your experience?