|
|
|
|
|
|
by regularfry
9 days ago
|
|
|
I got usable token rates (10-20tps from memory, so marginal) with the Qwen A10B a while back, well before all the new speculative speedups landed in llama.cpp. There's an unmerged branch which allegedly supports this, but I don't know how well yet. Looks worth investigating but I might give it a few days to see what bugs get shaken loose. |
|
|