[fg137]

This.

I have seen way too many people who are overly optimistic about local LLMs.

Having spent a decent amount of time playing with them on consumer nvidia GPUs, I understand well that they not going to be widely usable any time soon. Unfortunately not many people share that.

[close04]

Not this. Let's reframe the problem. How many years behind do you think they are? By all accounts Gemma 4 is better than a frontier model from 3 years ago. Back then we were wowed by frontier models but when the local model reaches the same performance it's no good anymore, because you moved the target?

Relatively speaking local models might always be behind the curve compared to frontier ones. You can tell by the hardware needed to run each. But in absolute terms they're already past the performance threshold everyone praised in the past.

Right now in a lab somewhere there's a model that's probably better than anything else. There's a ChatGPT 5.6, an Opus 4.8. Knowing that do you suddenly feel a wave of disappointment at the current frontier models?

[2ndorderthought]

So the cofounder of hugging face made a post about qwen 3.6 being atclaude level of performance for the lols?

When were you trying local models? The model releases from April 2026 are a serious change in performance.

[solenoid0937]

It's just not there yet. I have tried all the models from April, including the Gemma 4 variants.

These are so far from Opus it's not even funny. They are not close to being in the same league. Gemma might be like a frontier model from a couple years ago, but with much worse performance in long context chats.

[2ndorderthought]

Correct they aren't opus. They are sonnet with a little hand holding. They also run on a single GPU at 40 tps.

No one is saying a local model will give you anthropics business in a 5min download. People are saying, "hmm, maybe I should do this one locally". People are also saying "this is surprisingly good enough for me given the trade offs"

[fg137]

> "hmm, maybe I should do this one locally"

If your time is worth nothing to even triage that question.

Unless you have fanatic needs for data privacy or really don't have Internet, running local models almost certainly results in negative ROI overall.

Not to mention that you need to have decent hardware (that is getting expensive by the day) to even have this conversation in the first place.

People in this post talk as if everyone has a Mac with 24GB or 32GB RAM. When the reality is that most people use a Windows laptop with crappy integrated GPU.

[anon373839]

Hm. I think there is a bit of a shifting goalpost dynamic at play here. Those April releases, even the fast MoE versions, are better than big cloud models from 18 months ago. I remember when everyone was gushing about Sonnet 3.7 and what a transformative experience development was using it. So was it useful or wasn’t it? A tool doesn’t lose its usability just because a better one comes along.

To me, these small local LLMs are highly useful (and this “usable”) even though they don’t match the output of today’s frontier models.

[2ndorderthought]

Completely agree. I would even shift the 18months up a bit. I have been impressed with qwen3.6

[fg137]

I'll believe that when Uber deploys local models for developers and ask them to prefer local models over proper Anthropic ones.