[silverliver]

In production, I'd be more concerned about the possibly of it going off on it's own and autoupdating and causing regressions. FLOSS LLMs are interesting to me because I can precisely control the entire stack.

If Ollama doesn't have a cli flag that disables auto updating and networking altogether, I'm not letting it anywhere near my production environments. Period.

[eclectic29]

If you’re serious about production deployments vLLM is the best open source product out there. (I’m not affiliated with it)