[akreal]

Cool!

HF Transformers is great for prototyping and research, but should not an interactive tool like this be based on something more speed-focused, like llama.cpp?

Any plans for languages beyond English?

[juberti]

We're running it on vLLM and are working with others in the community to bring it to other optimized inference frameworks.