Y
Hacker News
new
|
ask
|
show
|
jobs
by
syntaxing
5 days ago
Which quants? I get these speed (only 20-30 TPS) at Q4_K_M with MTP for 27B on my framework desktop. I draw sub 130W for the whole machine.