Hacker News new | ask | show | jobs
by SparkyMcUnicorn 406 days ago
You can use 0.6B for speculative decoding on the larger models. It'll speed up 32B, but slows down 30B-A3B dramatically.