[whimsicalism]

It depends if you want ease or speed and if you are batching.

Ease? Probably ollama

Speed and you are batching on gpu? vLLM