[Ilaurens]

These token numbers look off for a 5090. For comparison, an rtx 5000 Blackwell SFF with just 470gb/s bandwidth gets me 30 tok/s on gemma4-31B-QAT. Almost 60 tok/s with MTP enabled. A 5090 should get you much more than that!

[jszymborski]

Thanks for flagging, this is on my local rig and it's driving my display too. I'm curious now, will take a closer look. These are the tok/s as reported by LMStudio.

EDIT: Updating llama.ccp gets me 58 tok/s on Gemma 31b

[fhars]

The Qwen-35B-A3B numbers are even weirder, did you drop a digit? I get half of that speed on a AMD Radeon RX 5500 XT (RADV NAVI14) (8192 MiB) (unsloth/Qwen3.6-35B-A3B-MTP-GGUF:Q6_K with q8_0 context).

[jszymborski]

No, it's what I get repeatedly. Recall that it's QB_0 though. I'll get back to you when the Q4_K_M quant finishes downloading.

EDIT: I get 259 tok/s with the Q4_K_M quant

[pixelesque]

I assume you mean Q8_0, not QB_0? (was confused, as I've not obviously seen a QB_0 before and can't find one now).

[jszymborski]

bahaha, LMStudio's badge looks a lot like a B but the tooltip says it is indeed an 8 (I just assumed it was B for "byte", but that's a poor initialism considering "bit").

https://dl.jszym.com/share/boards/pictures/Screenshot_202607...

[mft_]

Agree - I got 9.7 tok/s on an M1 Max with unsloth's gemma-4-31B-it-qat-UD-Q4_K_XL.