[throwa356262]

Better performance than TQ and better quality than FP16?

Am I reading this right??

[qeternity]

It's not better quality: 59.3% vs 59.4% fp16 on AIME 25

[sheepscreek]

0.1% is within margin of error. Depending on the performance boost, it might be worthwhile taking a minuscule quality hit.

[qeternity]

I think it very much is worth it!

But the point was that quality didn't magically increase.

[electroglyph]

any divergence (even if the benchmark is better) from full precision is error

[7e]

Just pretend that it is the next step update when training. You didn’t train your model to step=inf, I hope?

[thefox96]

Faster than Fp16, not better quality i guess