[ak_111]

I think because if you end up having an AI that is as capable as the graduate students Tao is used to dealing with (so basically potential field medalists) then you are basically betting that 85% chance something like AGI (at least in consequence) will be here in 3 years. It is possible, but 85% chance?

[mewpmewp2]

It would also require ability to easily handle large amount of complex information and dependencies such as massive codebases etc and then also be able to operate physically like humans do. By controlling a robot of some sort.

Being able to solve self contained exercise can be obviously very challenging, but there are other different types of skills that might or might not be related and have to be solved as well.

[sigmoid10]

>then you are basically betting that 85% chance something like AGI

Not really. It would just need to do more steps in a sequence that current models do. And that number has been going up consistently. So it would be just another narrow AI expert system. It is very likely that it will be solved, but it is very unlikely that it will be generally capable in the sense most researchers understand AGI today.

[benchmarkist]

I am willing to bet it won't be solved by 2028 and the betting market is overestimating AI capabilities and progress on abstract reasoning. No current AI on the market can consistently synthesize code according to a logical specification and that is almost certainly a requirement for solving this benchmark.

[sigmoid10]

What research are you basing this on? Because in particular fill in the middle and other non-standard approaches to code generation have shown incredible capability. I'm pretty sure by 2028 LLMs will be able to write code to specification better than most human programmers. Maybe not on the level of million line monolithic codebases that certain engineers worked on for decades, but smaller, modern projects for sure.

[benchmarkist]

It's based on my knowledge of mathematics and software engineering. I have a graduate degree in math and I have written code for more than a decade in different startups across different domains ranging from solar power plants to email marketing.

[sigmoid10]

I've been actively researching in this field for close to a decade now, so let me tell you: Today is nothing like when I started. Back then everyone rightly assumed this kind of AI was decades if not centuries away. Nowadays there are still some open questions regarding the path to general intelligence, but even they are more akin to technicalities that will probably be solved on a time frame of years or perhaps even months. And expert systems are basically at the point where they can start taking over.

[benchmarkist]

What are the technicalities?