[BoredomIsFun]

I'd stay away from ollana, just use llama.cpp; it is more up date, better performing and more flexible.

[mika6996]

But you can't just switch between installed models like in ollama, can you?

[BoredomIsFun]

llama-swap? https://www.nijho.lt/post/llama-nixos/