[Chathamization]

> Our belief is that once we can no longer come up with quantifiable problems that are "feasible for humans and hard for AI" then we effectively have AGI.

I don’t think that follows. Just because people fail to create ARC-AGI problems that are difficult for an AI to solve, doesn’t mean that said AI can just be plugged into a humanoid robot and it will now reliably cook dinner, order a pizza and drive to pick it up, take a bus to downtown to busk on the street and take the money back home, etc.

ARC-AGI is an interesting benchmark, but it’s extremely presumptive to think that these types of tests are going to demonstrate AGI.

[Palmik]

In your example you already indicated two tasks that you think might be hard for AI but easy for humans.

Who said that cooking dinner couldn't be part of ARC-AGI-<N>?

[Chathamization]

That’s precisely what I meant in my comment by “these types of tests.” People are eventually going to have some sort of standard for what they consider AGI. But that doesn’t mean the current benchmarks are useful for this task at all, and saying that the benchmarks could be completely different in the future only underscores this.

[pillefitz]

They are useful to reach Arc-N+1

[Chathamization]

How are any of these a useful path to asking an AI to cook dinner?

We already know many tasks that most humans can do relatively easily, yet most people don’t expect AI to be able to do them for years to come (for instance, L5 self-driving). ARC-AGI appears to be going in the opposite direction - can these models pass tests that are difficult for the average person to pass.

These benchmarks are interesting in that they show increasing capabilities of the models. But they seem to be far less useful at determining AGI than the simple benchmarks we’ve had all along (can these models do everyday tasks that a human can do?).

[fastball]

The "everyday tasks" you specifically mention involve motor skills that are not useful for measuring intelligence.

[mchusma]

Genuine question, do you feel Waymo is not L5 self-driving? I Waymo has L5 but its not truly economic yet.

[jononor]

The task you mention require intelligence but also a robot body with a lot of physical dexterity suited to a designed-for-humanoids world. That seems like an additional requirement on top of intelligence? Maybe we do not want an AGI definition to include that?

There are humans who cannot perform these tasks, at least without assistive/adapted systems such as a wheelchair and accessible bus.

[Chathamization]

> at least without assistive/adapted systems such as a wheelchair and accessible bus.

Which is precisely what the robotic body I mentioned would be.

You're talking about humans who have the mental capacity to do these things, but who don't control a body capable of doing them. That's the exact opposite of an AI that controls a body capable of doing these things, but lacks the mental capacity to do them.

[aziaziazi]

I read that has “humans can perform these task, at least with…”

Put the computer in a wheelchair of his choice and let him try to catch the bus. How would you compare program and human reasoning abilities, but disregarding human ability to interact with the outside world?

Edit: Arc-AGI itself is only approachable by visually and manually valid humans, others needs assistive devices.

[CooCooCaCha]

The statement you quoted is a general statement, not specific to ARC-AGI.

The scenarios you listed are examples of what they’re talking about. Those are tasks that humans can easily do but robots have a hard time with.

[yorwba]

The point isn't demonstrating AGI, but rather demonstrating that AGI definitely hasn't been reached yet.