[jakeinspace]

Just curious, is it absolutely necessary that a single model can solve all these problems to satisfy you? As long as it’s a finite and relatively small list, why not allow for N models boxed together wrapped in a switch statement? Or if you’re picky, a top-level language model which tries to decide which of its subsystems to employ for the problem at hand. After all, the human brain is at least somewhat partitioned (although not at the granularity of chess vs hotdog identification).

[frgtpsswrdlame]

It wouldn't even have to be a language model, just some sort of meta program that handled the switchboarding, as it were, between subsystems would be enough. But my real point is just more narrow, you can't measure progress to artificial general intelligence with a list of tasks satisfied by artificial specific intelligence. You have to measure the generalness directly. And thats also why I don't think you'd have to fulfill the list. Plenty of people suck at poker and golf and talking philosophy. People mistake the generalness of AGI for something like 'smarter than humans' but it's totally reasonable for there to be an artificial general intelligence which sucks at chess, just like a lot of people.

[jakeinspace]

It’s very difficult to come up with an objective metric or benchmark for general AI that can’t be gamed, or which won’t turn out to be disappointingly easy. It makes sense that most research would be in the direction of tasks which are easily quantifiable. More hazy benchmarks like the Turing test are possibly better but that one in particular isn’t so good unless it’s enhanced (I’d say a 1 hour conversation with an AI posing as someone with graduate-student level understanding of a field of science or art I know something about would be adequate proof, but maybe I’m being naive).

[frgtpsswrdlame]

1. If it's too hard to come up with an objective benchmark for AGI then we should rethink what it is we're doing talking about progress toward that's goal.

2. You could also rework this list and say that an AI which can fulfill three tasks even badly is one step forwards. An AI which can fulfill three tasks from three separate categories is another step, etc. But I think it's a categorical mistake to count progress in specific AI as progress in general AI, there's not a very good reason to believe they are in a continuum with one another.