[verdverm]

There is a bug in llama-cpp for qwen/gemma models, use vLLM instead

[pdyc]

what bug and it affects what?

[verdverm]

it's a prompt cache invalidation bug that causes all input to be reprocessed instead of getting preloaded

There are other reasons to prefer vllm to llama-cpp as well