Hacker News new | ask | show | jobs
by nightski 475 days ago
$10k to run a 4 bit quantized model. Ouch.
2 comments
OriginalMrPink 475 days ago
That's today. What about tomorrow?
link
water9 475 days ago
The M4 MacBook Pro 128GB can run a 32B perimeter model with an 8 bit quantized model just fine
link