Hacker News new | ask | show | jobs
by yencabulator 46 days ago
4-bit quantized 30B-A3B MoE models can run at something like 21 tokens/sec on a several year old AMD CPU.