Hacker News new | ask | show | jobs
by girvo 55 days ago
Not that you asked but I’m getting ~20 tokens/s on my DGX Spark (Asus actually) using an Int4 AutoRound quant, MTP 1 and some other tricks