[mlyle]

> beat humans at a 5k race also isn't very interesting (at this point), because fine, you created a running robot, but that robot's not going to do it's own taxes at the end of the year, so in the sense of AI, who cares.

It depends exactly on what we're saying. So far we have machines that can navigate language well, but aren't good at navigating real world environments. If we're talking about a road 5k race, with myriad ways that courses are marked and many, many distracting, this is at least an interesting problem (perhaps as difficult as self driving vehicles).

[kbenson]

Being able to navigate the course is an interesting problem. I'm not sure beating a human at it is, unless we change the problem to be accomplishing something useful (lugging a certain amount of weight up a cliff face for rescue operations, for example), or make the contestant AI sufficiently human like.

A 5k race is about competition, not about doing useful work. For AI or a robot to compete and win and have it mean anything means it needs to be similar enough to the other contestants to make that competition matter, otherwise I could slap a picture of my face on an RC car with enough batteries and "win". We all realize intuitively that a "win" such as that tells us nothing useful, so would ignore it even if it somehow was allowed. AI is a bit less intuitive to many at this point, so it's not always as obvious.

[mlyle]

Well, sure. IBM's Jeopardy PR stunt with Watson is an instance of that: it was definitely "buzzer doping."

At the same time, included in that "win" was a big technical feat. So it would be with winning a 5k race.

(Not to mention that right now nothing we have capable of bipedal or even quadripedal locomotion is capable of the feat, so if we rule out the RC car it's quite the mechanical accomplishment, too--- even if we're throwing a ridiculous amount of power density and big actuators at the problem).

[kbenson]

Beyonh Watson buzzer doping, it's interesting but not as much as some people make it out to be. Since it isn't an autonomous package equivalent to a person. At the time, it was the size of a master bedroom. If we compared it to a room jull of smart people, does Watson still seem impressive in it's feat?

Similarly, getting a mechanical package that can traverse a course like a human using locomotion like a human, which would be much more impressive if it's not running of flat asphalt. But like you note, we can't do that even entirely abstracting away the AI portion, so the question of what will happen when an AI wins a 5k is mostly moot, there's many steps to get there that we haven't gotten close to, and even when we've solved all the aspects separately (a general AI, locomotion, power density) it will likely be a while after that (if ever) before they are solved together in a package that compete.

Im not saying humans are the epitome of these systems come together, but I doubt evolution has left us with a completely horrible design, especially if were talking about thinking and running, two things humans are known for being quite good at (to our own knowledge).

[mlyle]

Eh, picking arbitrary metrics like that -- the size of Watson -- isn't too useful. For most of these things, like Watson-- we're concerned about whether it could do it more cheaply (less resource intensively) than a human. Watson also does the job on much less sleep and doesn't get bored.

(And a room of smart people is not likely to be much better at this kind of quick decision task than a single smart person).

And now, you can fit 16TB of ram in a couple rack units with boring hardware, though you'd need perhaps a quarter rack to get equivalent memory bandwidth.

> but I doubt evolution has left us with a completely horrible design, especially if were talking about thinking and running, two things humans are known for being quite good at (to our own knowledge).

LLM's are making me less sure that we're so good at thinking. We may be good at making quick decisions and navigating social hierarchies in a relatively low power budget, but that's a different thing...

[kbenson]

> we're concerned about whether it could do it more cheaply (less resource intensively) than a human.

That's what I was getting at before. For a specific work based competition about who can do a specific job better, I agree. For a general "run five kilometers" case without any other explicit constraints, those constraints are basically "be human", because it's about competition.

Sometimes we even break it down more than that when "being human" isn't enough to provide useful information across the population of contestants. There's a reason some competitions are broken into separate categories for the sexes. Putting biological males and females together in power lifting competitions is less useful from a standpoint of determining how like organisms compete than not (which is one of the reasons, along with using substances to move someone from one category closer to another, people get upset about gender in sports competitions now).

> (And a room of smart people is not likely to be much better at this kind of quick decision task than a single smart person).

To a specific degree it certainly does. As many people as you can fit around the buzzer, or giving each person their own buzzer that connects to the central "real" buzzer seems like a simple enough solution to yield good results - as long as false buzzers are low enough.

> And now, you can fit 16TB of ram in a couple rack units with boring hardware, though you'd need perhaps a quarter rack to get equivalent memory bandwidth.

Don't forget power. If it's not solar powered, you're forgetting a very large aspect of what it means to be a human. The equivalent "human" would likely just be a brain with some electrodes for input and output, so would still be much smaller, if it was possible. Until Watson is able to move itself to a power source, sip from that power to refill some battering, and them be autonomous for a few hours, even it it's presented as the same "size" as a human I'll not be convinced.

Hell, they can start by ignoring the movement and just giving it a battery pack that can actually sustain it for a few hours of operation in a similar size package. We're closer to that, but I'm not sure where there yet (it probably depends on how much we care about weight or mass compared to volume).

> LLM's are making me less sure that we're so good at thinking.

That' funny, LLM's are doing pretty good and convincing me most the time we're thinking in a similar manner, with the same problems. ;)