[ch_sm]

Nice, I tried it too with oMLX — agreed, it seems very capable for coding! Was slightly underwhelmed by performance though. I got about ~24 t/s on the ternary version on my M2 Max 64. That’s quite a bit slower than Qwen A3B 35B (4 bit unsloth). How was perf for you?

[anon373839]

I had roughly the same performance on M3 Max 64: ~30 tokens/sec. Which is not terrible, but with the latest Lightning MTP optimization I am getting ~100 tokens/sec with Qwen 3.6 35B-A3B.

[ch_sm]

WHAT. that‘s amazing. thanks so much for sharing, i‘ve been looking for ways to speed a3b up for days. It‘s 11pm here but i‘ll try this right now

[anon373839]

I’m using the new oQe quants in case that matters.

[ch_sm]

i see! would you mind sharing your model string?

[midulan]

i'm getting similar toks on my 36gb m3 max. new omlx release (0.5.2) has dedicated decode kernels for the bonsai models — see if this improves perf for you.