[Codefrontier]

I agree with your speculation to be honest. And yet I’ve tried several local open weights model now and none gives the same quality of answers as Claude gives me on a regular Sonnet model. Mind you: I am “running” 48GB of RAM so I can’t try every model. Where does this difference come from? Can we actually get close locally?

[midnightbobarun]

Something I've noticed is that local models are giving better answers these days than they did a year or two ago, even if the size (in parameters and in the amount of RAM used) hasn't increased. I'm not familiar enough with the technical side of model training to explain how they're doing this, but I think in another couple of years, models that use up 48 GB will be able to squeeze out even more incredible performance.

Though on the level of something like Sonnet 5... well, maybe not.

[pocketarc]

You’re running 48GB now but imagine a future where everyone has 512GB RAM or 1TB RAM in their computers (it might sound like a lot but also 20 years ago we had 512MB PCs).

It’s not hard to imagine what 5-10 years of pressure to increase RAM will do to specs, on top of the normal tech improvements.

That’s worth bearing in mind when thinking about local models.

Plus, local models keep getting better and better; 2 years ago what you could get out of those 48GB of RAM was embarrassing compared to what’s doable today.

We’re getting there. Just takes time.

[xyzsparetimexyz]

I seriously doubt that we'll be running 1TB ram setups any time soon. I was hearing about people running 64GB back in 2014 or so

[timmytokyo]

Moore's law also applies to memory. With personal computers, we've gone from ~16KB in the late 1970s to ~640KB in the mid-80s to ~512 MB in the late-90s, to ~1GB in the early 2000s, to ~64GB today. If you look at the memory per dollar graph, it's even more obvious. This is exponential growth.

I see no reason to think the trend won't continue for the next couple of decades.

[xyzsparetimexyz]

Moores law is dead for memory and other things