Hacker News new | ask | show | jobs
by Archit3ch 19 days ago
You can keep the KV cache in (possibly Unified) RAM to avoid SSD writes entirely. Not sure if it would fit on a 32GB laptop, though.