Hacker News new | ask | show | jobs
by jszymborski 3 days ago
No, it's what I get repeatedly. Recall that it's QB_0 though. I'll get back to you when the Q4_K_M quant finishes downloading.

EDIT: I get 259 tok/s with the Q4_K_M quant
1 comments
pixelesque 3 days ago
I assume you mean Q8_0, not QB_0? (was confused, as I've not obviously seen a QB_0 before and can't find one now).
link
jszymborski 2 days ago
bahaha, LMStudio's badge looks a lot like a B but the tooltip says it is indeed an 8 (I just assumed it was B for "byte", but that's a poor initialism considering "bit").

https://dl.jszym.com/share/boards/pictures/Screenshot_202607...

link