[lackoftactics]

Two questions.

How do you handle the competition like https://github.com/ENTERPILOT/GOModel and Bifrost? They already moved to more performant languages like Go.

What is the moat of litellm currently and why such a radical move?

[detente18]

1. I think rust is the better long term solution over go. Especially for a gateway. You can see other proxy's (e.g. cloudflare's pingora - https://blog.cloudflare.com/how-we-built-pingora-the-proxy-t...) doing the same thing.

Rust/Python hybrids are also quite well established, so it allows us to work off of a stable base - while ensuring we can deliver - low memory utilization - low request latency overhead

which is the primary goal here (be fast, lightweight and cheap to deploy).

2. LiteLLM has the most mature, and broadest range of unified api's x providers. This means you do not need to give developers raw LLM API keys, ever.

We see devs using/building agents that consume a lot of different API's - responses api, realtime, chat completions, messages - and no matter what they use we want them to be able to switch across providers without if/else statements in their code.

I can't comment on others, but that's our goal and what we work on doing everyday. So I would trust that we do it well.

Beyond that, we're also growing to become the single point of access for all AI resources. This makes it a lot easier when building agents, because you can give an agent 1 key, and it will have access to LLM's + MCP's (and in the future other resources like skills, api credentials, sandbox api's, etc.).