[Der_Einzige]

Everytime I try to mention how shit Ollama is, I get mass downvoted here by folks that don't want to hear the truth: There are 4 good inference engines (okay, 5, but we don't count huggingface because it's slow):

1. vLLM

2. sglang

3. (nvidia only) TRT-LLM

4. llama.cpp (mac only, the above are better for non-mac)

If you're not using one of the above, you're doing it wrong