Hacker News new | ask | show | jobs
by freakynit 33 days ago
80B + INT4 + speculative (FP8 mode) => 72,188 tokens/s effective

..da fck!!