Hacker News new | ask | show | jobs
by Wowfunhappy 83 days ago
Only the 'it' models seem to have quants. I was really hoping to try a base model.
1 comments

Basic quantization is easy if you have enough RAM (not VRAM) to load the weights.