[jacooper]

But can you run llms that easily with vllm? do you have to fiddle with formats to get it to run?

[buyucu]

I'm still in early stages of exploration, but vllm seems to be compatible with most models on huggingface.