Hacker News new | ask | show | jobs
by alex43578 128 days ago
NVIDIA is showing training at 4 bits (NVPF4), and 4 bit quants have been standard for running LLMs at home for quite a while because performance was good enough.