[serjester]

It's interesting that it seems to the non thinking variant has actually regressed on a quite few benchmarks compared to flash-2.0. They seem to be prioritizing coding above all else. Even the thinking variant only has marginal gains on non coding.

Our table parsing benchmarking has flash-2.0 at 0.84, flash-2.5-lite at 0.80 (non-thinking), flash-2.5-lite at 0.80 (thinking). Kind of unfortunate to see.

[1] https://github.com/Filimoa/rd-tablebench

[suddenexample]

This makes sense though, right? Flash-Lite is intended to be weaker than Flash - the comparisons should be flash-2.0 vs flash-2.5 and flash-lite-2.0 to flash-lite-2.5.