Hacker News new | ask | show | jobs
by aswegs8 34 days ago
How does the model qualify as local? ~192 GB RAM needed sounds a bit much for local.
2 comments
antirez 34 days ago
Runs on 96GB MacBooks. 128GB is better. Check the README of DwarfStar.
link
defen 34 days ago
If you were buying a computer today to use for DS4 (budget under 10k) what would you get? I care more about inference quality than speed. Or is it better to wait for the rumored M5 studio?
link
bigyabai 34 days ago
For large contexts, you're probably going to want CUDA. The DGX Spark is expensive but would get you decent speeds on DS4 Flash. DS4 Pro is probably not possible with $10k if you want sub-10 minute prefill.
link
embedding-shape 34 days ago
Can you download it and run it given you have the hardware? Then it's local, regardless if you happen to have the needed hardware or not.

Bit like asking if Zigbee can be considered local/LAN for people who don't have the required radio/antenna.

link