[redox99]

Is there a way to make it work with ooba+exllama? (much faster than llamacpp)

[thelastparadise]

You should be able to turn on the API in booba:

https://github.com/oobabooga/text-generation-webui#api

[redox99]

But that API isn't OpenAI compatible AFAIK