Hacker News new | ask | show | jobs
by jurgenburgen 22 days ago
Does more RAM increase performance? This approach sounds like it could eventually be fast enough for local use as hardware and models improve.
1 comments

Faster SSD access improves performance more than RAM does, at least until all of the model is being cached in RAM. So older and cheaper HEDT platforms with lots of PCIe lanes to attach storage to are best for this approach.