[whaleofatw2022]

Part of me wonders, would 3d xpoint (if still around) be a viable option?

Yeah it is slower than real RAM by a good amount for latency, but you can get similar bandwidth and the cost was history about half of the same size DDR.

[cmrdporcupine]

I was thought experimenting the other day... ~10 nVME drives striped and running parallel could approximate the memory bandwidth of DDR5 DRAM in a box like this. Like you say, latency wouldn't compete but on raw throughput would be comparable.

Not anymore cost effective, I guess, but gets you the ability to work over very large model sizes maybe. But the problem is that tensor matmul etc hardware wouldn't work effectively with it.

Useful for KVCache though.

[ciupicri]

I'll just leave this here: "Achieving 11M IOPS and 66 GB/S IO on a Single ThreadRipper Workstation" (2021), https://news.ycombinator.com/item?id=25956670 / https://tanelpoder.com/posts/11m-iops-with-10-ssds-on-amd-th...

[science4sail]

Possibly - I've heard anecdotal reports that old Intel Optane chips are in hot demand right now. Intel/Micron probably would have made a killing if they had kept that product line alive for a few more years. Never miss an attempt to snatch defeat from the jaws of victory!