Hacker News new | ask | show | jobs
by chrsig 903 days ago
How does one rawdog a 16-bit model?
1 comments

Usually, for efficiency, you use quantized models. Quantized models reduce the number of bits available for each parameter, saving space and reduce RAM usage.