[cubefox]

They are already above average human level on many tasks, like math benchmarks.

[amusedcyclist]

They really aren't better than humans at math or logic, they are good at the benchmarks because they are hyper optimized for the benchmarks lol. But if you ask LLMs simple logical questions they still get them wrong all the time

[lawn]

Yes, there are certain tasks they're great at, just as AI has been superhuman in some tasks for decades.

[cubefox]

But now they are good or even great at way more tasks than before because they can understand and use natural languages like English.

[lawn]

Yeah, and they're still under delivering to their hype and the improvements have vastly slowed down.

[cudgy]

So are calculators …

[kranuck]

If you ignore the part where there proofs are meandering drivel, sure.

[cubefox]

Even if you don't ignore this part they (e.g. o1-preview) are still better at proofs than the average human. Substantially better even.