[have_faith]

What’s the time horizon do you think for free models matching today’s SOTA on average consumer hardware? I see people building 6k+ machines to run the best of them at the moment, which are behind SOTA by maybe 6 - 12 months or so right now.

[CuriouslyC]

Open models lag the frontier ~3-6 months, though they're likely smaller than frontier models as well so that lag might not be fully real. Qwen 3.6 27B is very usable for average coding, and Gemma4 31b is very usable for day to day tasks.

The problem there isn't the models, it's consumer hardware. Even 16GB cards aren't the norm, and even with massive improvements in per-parameter performance we probably still need 48GB memory to get models that feel smart enough to trust.

[everforward]

“Average” is also doing terrible things there. The “average GPU” is probably the integrated graphics on the CPU of a laptop.

If you scoped it to “average gaming desktop”, double digit VRAM is pretty normal at this point. If costs came down, I imagine the higher end GPUs would start including enough VRAM for 30B-ish models.

[JumpCrisscross]

> I see people building 6k+ machines to run the best of them at the moment, which are behind SOTA by maybe 6 - 12 months or so right now

SOTA in open source (frontier Kimi MOE) requires terabytes of RAM. At DDR5 prices, that's $40k alone. For HBM, higher. We're years away from consumer hardware matching the power and latency of e.g. Claude.

[abustamam]

I don't think free/open model necessarily means local. I use open code Go for $10/mo for pet projects and deepseek v4 pro is largely comparable to my workflow at work using Claude code. Obviously this wouldn't work for someone wanting to do more than just per projects (I hit my weekly quota 5 days in, on basic usage) but I'm just saying that local doesn't have to be part of the equation