[Havoc]

Is there something different about that accuracy measure? i.e. relative to perplexity term usually used

Going from BF16 to 2.8 and losing only ~5% sounds odd to me.

[kouteiheika]

It's accuracy across GSM8K, MMLU, IFEVAL and LiveCodeBench.

They detail their methodology here: https://byteshape.com/blogs/Qwen3-4B-I-2507/