[rubyn00bie]

We'll likely see a transformation in how frontier models are trained as a result of a push towards local inference. While it seems unlikely now, given current pricing for RAM, in 10-15 years it's not unthinkable to assume we could see individual machines with 10-12TB (and well beyond that) of RAM which are accessible to the GPU. Min/max system RAM increased a LOT from 2010-2025 and largely because it was cheap. Once the hyperscalers aren't generating revenue for the RAM manufacturers, I wouldn't be surprised to see a massive push towards consumers in order to maintain gross profit. Not to mention new players who enter the market because the margins are measurably absurd right now.

At some point there will be diminishing returns towards the "just throw more RAM at it" approach the current frontier models are taking. Commoditization is just as inevitable as it ever was... and in doing so will enable actual leaps of what AI/ML is capable of. That's not to say there won't be a place for 99.999999% accurate vs 99.99999% but those cases will be limited and likely prime to disruption based on real innovation vs access to capital.

[Culonavirus]

The 1080ti is out there for almost 10 years now. It has 11GB of VRAM. A 5090 has 32GB.

SOCs with unified memory have shifted this a bit forward, but they're also expensive as shit.

10TB ram in a consumer device is simply not happening in the next 10 years.

[rjzzleep]

Half a year ago you could get a AI max 395+ with 128GB ram in mobile form factor for ~$2200. The same thing costs $3700. Same SoC, same memory.

[rbanffy]

10TB is about 80 times that, 200K in today’s money. A lot of capacity is coming online in the next 5 years and it’s reasonable to think we can get there with better process and stacking (the latter does little for pricing, but enables shorter latencies).

[interludead]

I agree with the general direction but I'm a little skeptical of the "just add a few more TB of RAM and the frontier moves local" version of it