[pants2]

If you're developing on top of LLM APIs directly, this is definitely not true. There are differences in how context caching works, in what's available through native harnesses, the types of tools you're fine-tuned on (GPT uses apply_patch while Claude uses edit, with different formats), the API surface (Agents SDK, Responses API, Managed Agents), cost structures, and best-practice guidance all around.

Not to mention the meta of account limits, billing, ZDR contracts, etc.

[peab]

It really depends on what you're doing, but most LLM usage and agentic runs are pretty interchangeable in my experience, and it's usually trivial to switch.

If anything, you're better off supporting multiple LLMs as backup because most model providers have been so inconsistent with working all the time

[saberience]

Dude it’s not trivial to switch because the behaviors are different!

You’re clearly not building a product based on an LLM.

I’m still using various old Anthropic and OpenAI models for products I’ve built and released because I can’t risk the behavior changing in unpredictable ways and the users being pissed.

It’s much easier to switch out some deterministic software than an LLM which you’ve spent a ton of time on testing and benchmarking and understanding its nuances. Changing it is like replacing an employee who’s critical to the business.

[staticman2]

Anthropic has discontinued models in as little as 13 months from launch so if you do business with them switching can't be that big a deal?

[vrganj]

Can you give specific examples on what differences make it hard to switch?

Because this claim is counter to my experience as well.

[mjhay]

The behavior of a single model and version can and does change. There’s not only built-in stochasticity, but closed hosted models like Claude are tweaked and changed all the time.

[moxza]

For the public facing consumer functionality I have Gemini Flash running on guardrails directed by a state machine that calls it statelessly everytime. For that, it's strictly locked to a version. I can't afford to suddenly get responses that the SM is not tuned for.

As for which model does the building... I'm not at all attached. Enough logic, and CI gates/tests live outside the whims of the LLM to be able to hotswap them any time.

[asdfaoeu]

I don't think they are saying it's trivial but compare say for example switching an organisation from Office or Windows the example that started this. They are not even in the same ballpark.

[peab]

I've been building multiple products with LLMs, and they are in fact interchangeable for the most part.

In fact, most benchmarks show this! Most benchmarks have similar performance for the same classes of models.

On top of this, there are tools like open router, or even the openai SDK which trivially allows you to swap endpoints for the LLM!

If you're using the agents SDK from openai or something, then yeah it's not interchangeable but that's you doing it wrong

[devsda]

Makes sense but honestly if you've spent more time testing and working around the nuances to build consistent experience doesn't it mean you actually need more standardization to easily switch models if/when your trusted model is not viable for you provider?

[byzantinegene]

just use your agents to do the migration, that's what it's good at.

[chairmansteve]

Exactly, the LLM itself can do the migration, surely?

[tharkun__]

Exactly, as in, really, will they? Where and at what price, especially across an actual enterprise that needs to deploy them to lots of devs? There's much more than just the actual model.

Of course my numbers are a sample of one and I am not spending a lot of money or time on it. Just lazily trying things on my "happen to have this" hardware. But basically trying out the Claude Code I'm used to from work but locally with a bunch of open weight models.

I can run super tiny models on my 8GB NVIDIA card. They all suck (I have to use <=~5GB models if I want "usable" ~250k context that doesn't need to use system RAM and CPU (which makes things super slow).

I've also tried a GLM 4.7-flash, which even though it's super slow (in comparison) with ~250k context and it just doesn't cut it vs. the Claude Sonnet or Opus I get to use at work. All the while these are all touted as "totally usable, Claude/ChatGPT killer!" replacements.

It's just not "there" with tool use or building software for that matter. Like, just a simple Claude "web search" fails with it. So I asked it to build itself its own "web search" functionality and it just couldn't. It made so many mistakes its just not funny any more. And it couldn't recover from them either. I retried a few times (as I didn't have python installed and it wanted to implement it using that - this happens to be new system - never mind other attempts). I spent as much time doing this (and failing) as I spent building an actual full feature at work last week w/ Sonnet.

If it can't build itself a simple web search to .md file tool/skill, how am I supposed to trust this with actual coding? I'm used to being able to point Claude at our large code base and essentially work with it like a junior doing my bidding. Maybe 5.2 is a killer game changer vs. what I was able to try out (if slowly) but you really have to show me to convince me at this point. And not with synthetic benchmarks. In those, all of the models I tried are supposedly super awesome.

[arcanemachiner]

4.7 Flash is a small model that's almost a year old, which is ancient. And yes, your dinky GPU will not run anything worthwhile.

Just spend $5 on OpenCode Go and give GLM 5.2 a shot if you have the time. It's not quite as good as Opus, but it's more than good enough for many tasks.

[mikae1]

> Just spend $5 on OpenCode Go

$5 the first month, then price is doubled.

[arcanemachiner]

The $5 is so they can see if open weights models are worth using, not so they can use it for a month. (Which you can't; The quota runs out way sooner than a month for any serious usage. Still worth the price of entry.)

[miroljub]

If you use DeepSeek v4 Flash as a daily driver, with an occasional usage of DeepSeek V4 Pro and Glam 5.2 when necessary, the monthly quota practically never runs out.

[wongarsu]

Getting the pay-as-you-go plan from DeepSeek is also a good alternative. When motivation strikes you never get slowed down by quota, and it's cheap enough that even with mostly DeepSeek V4 Pro it's price-competitive with a $5/month subscription. Depending on how bursty your usage pattern is it might even be cheaper

[Sevii]

With OpenRouter you can pay flat rate and try nearly any model on the market.

[wonnage]

Open weights != local models.

[Der_Einzige]

This is a meme and massively over-complicating what is ultimately quite simple.