[mewpmewp2]

I wonder how accurate those benchmarks are in terms of actual problem solving capability. I think there's a major line at which point LLM becomes actually useful and it actually feels like you are speaking to something intelligent and that can be useful for you in terms of productivity etc.

[icelancer]

They aren't at all. They are synthetic benchmarks that carry little resemblance to real world experience.