[saghm]

The original claim from the parent comment was running a Fable-level comment within a decade. Even if you're right about whether it's possible that another model could support that level physically, do you really think that we'll figure it out and ramp up the infrastructure to profitably sell on come consumer hardware anywhere close to that soon?

[marcus_holmes]

Well, we did do similar stuff back then. All it takes is money ;)

[saghm]

You're talking about going from a single gigabyte to 16 terabytes in a low-power consumer form factor, which is 1600x. A generous estimate of the factor of the RAM sizes for low-power consumer devices between 1990 and 2000 would still be a few orders of magnitude short, if I'm doing my math right.

Examples of what exactly you're claiming is precedent for this would be helpful.

[marcus_holmes]

This went down a rabbit-hole, which was fascinating, so thanks for the push :)

Not sure where the 1Gb number comes from? A standard laptop now is ~16Gb of RAM, so 1000x (and 1Gb -> 16Tb would be 16000x not 1600x). We went from Kb to Mb and then Mb to Gb of memory roughly every ten years from ~1990 -> ~2010. Each of those jumps is 1000x

Talking this over with claude, though, it pointed out that the need in dealing with LLMs is bandwidth and read-only storage, since the weights aren't dynamic. So we're not necessarily looking at 1Tb of RAM, we could be looking at 256Gb of faster RAM, and multi-TB of (much cheaper) flash storage, with extensive caching built in at OS level. This is all technically do-able with current tech, so it'll be interesting to see if it happens.