[arttaboi]

I guess this will happen soon. There are two catalysts needed for this to happen:

1. Evals that can quickly tell you how much downside there is to switching 2. Something like OpenRouter that can help you run those evals quickly

Now #2 is starting to become popular, and I think we'll soon see more people adopting a model-agnostic approach. Of course, there will still be high-intelligence use cases where nothing comes close to Claude or GPT.

[alexhans]

Exactly. I'm very happy the discourse has moved on from "but X model is the best" to "you can use open models".

Whether you're using SDK or harness based agents, having evals means you're able to modify any part of your agent and still know what satisfies your "good enough".

It's great for designing products that are easy to change as well.