[rhipitr]

Switching out an LLM? What do you mean by this? Sure some models can run locally but in a company with lots do people they might not be willing to spend to self host a larger model that requires beefier hardware to host, plus all the complexity to scale that out to a bit internal user-base

[KetoManx64]

Most of the AI companies have OpenAi compatible API's, so you just get a subscription from another provider and change the URL that your LLM Agent Harness uses to talk to the AI.

I use OpenRoutet which lets you switch between providers (Anthropic, ChatGPT, Z-AI) whenever you want. Sometimes I'll have two different models from different providers evaluate each other's answers.