Hacker News new | ask | show | jobs
768GB Intel Optane DIMMs to run 1T-parameter LLM with single GPU at 4tps (tomshardware.com)
30 points by walterbell 17 days ago
2 comments
lostmsu 17 days ago
The bottleneck in this setup is PCIe bus. You don't need optane to saturate it. 4 regular SSDs might do just fine.
link
musicale 17 days ago
Ah Optane, what might have been...

Even over PCIe, I imagine the advantage vs. NVMe is lower latency and more operations per second.

link