[saghm]

You're talking about going from a single gigabyte to 16 terabytes in a low-power consumer form factor, which is 1600x. A generous estimate of the factor of the RAM sizes for low-power consumer devices between 1990 and 2000 would still be a few orders of magnitude short, if I'm doing my math right.

Examples of what exactly you're claiming is precedent for this would be helpful.

[marcus_holmes]

This went down a rabbit-hole, which was fascinating, so thanks for the push :)

Not sure where the 1Gb number comes from? A standard laptop now is ~16Gb of RAM, so 1000x (and 1Gb -> 16Tb would be 16000x not 1600x). We went from Kb to Mb and then Mb to Gb of memory roughly every ten years from ~1990 -> ~2010. Each of those jumps is 1000x

Talking this over with claude, though, it pointed out that the need in dealing with LLMs is bandwidth and read-only storage, since the weights aren't dynamic. So we're not necessarily looking at 1Tb of RAM, we could be looking at 256Gb of faster RAM, and multi-TB of (much cheaper) flash storage, with extensive caching built in at OS level. This is all technically do-able with current tech, so it'll be interesting to see if it happens.