Hacker News new | ask | show | jobs
by hadlock 407 days ago
With 16gb you can comfortably run a 12b model that's been quantized. Quantizing is (bad example) effectively lossy compression.