Hacker News new | ask | show | jobs
by Balinares 63 days ago
Isn't that exactly how draft models speed up inference, though? Validating a batch of tokens is significantly faster than generating them.