Hacker News new | ask | show | jobs
by juancn 20 days ago
Prefill is around ~600 t/s.

I don't remember what the 27B was, I tried a 27B with different quantization at some point for that one, but I settled on the 31B.