[lnenad]

Considering the DeepSWE result (imho if you're gonna give value to benchmarks this is one of the best) it's not good enough.

[svantana]

It's a high quality benchmark for sure, but it being public means it's at risk of leaking into the models (unintentionally or not), right? For that reason I prefer to look at the private ones, like: HLE, SimpleBench, Kagi, ARC-AGI.