Y
Hacker News
new
|
ask
|
show
|
jobs
by
Saris
28 days ago
You can try tweaking MoE offload, I found the sweet spot after a few tries and even changing it by 1 can reduce speed by a few tok/s. I think around 45 is the average I get but sometimes it'll hit 50.