[hwillis]

The time between gpt 2 and 3 was 15 months. The time between gemini robotics 1 and 2 was also 15 months: https://blog.google/products-and-platforms/products/gemini/h...

The difference between gpt 2 and 3 was insane. 2 could generate limericks when it wasn't repeating a word 300x. 3 could actually do some things. By comparison gemini robotics has hardly changed at all.

I will also point out that slow, non-fluid robotics is on a totally different level of difficulty from fast fluid motion. Asimov could walk pretty smoothly, but it didn't fall over because it used a very careful sequence that was never unbalanced; you could pause at any point without falling over. Move faster, like boston dynamics, and you need to account for the change in balance from your arms swinging... or rather, you need to be able to account for the rotational inertia etc from moving multiple masses along complex paths with multiple points of articulation at hundreds or thousands of times per second.

An algorithm to fold tshirts 90% of the time is easy. The cloth hangs down by gravity and you can just look for right angles (corners), find their coordinates with binocular matching, and move them to meet each other. Getting 99%, or folding them quickly, so that the fabric is actually moving instead of just hanging still- incredibly, incredibly more complex.

[kibwen]

Even being able to simply do the task isn't enough here. It also has to do the task while also never, ever, ever accidentally folding your toddler as though it were a towel. Having industrial-strength robots around inherently unpredictable humans in a fail-safe way is an extremely difficult problem that will not be addressed any time soon.

[gretch]

This is not the bar for societal adoption, and we can already see a demonstration that it is false.

A waymo is basically an industrial strength robot. It operates among inherently unpredictable humans already. There is no guarantee that it will never, ever hurt a human.

And yet in many cities around the world, you can call a waymo and ride it and society accepts it.

[YeGoblynQueenne]

I think that's not a bad point but it ignores the different safety standards that most people accept when it comes to a) driving and b) their toddlers. Like a household appliance that kills 20-40k toddlers every year would be a major scandal, but it's OK for automobiles.

[boelboel]

Waymo will have existed for ~25 years by the time it'll be available in most US cities, that's a long while.

[fc417fc802]

> folding them quickly, so that the fabric is actually moving instead of just hanging still- incredibly, incredibly more complex.

TBF fabric is much more difficult to simulate than a bipedal body is.

As I recall openai had mujoco playing soccer nearly 10 years ago. Obviously real world bodies are much more difficult but I'd be curious to learn why that is.

[YeGoblynQueenne]

I don't think that there's any inherent difficulty to do with physical bodies, it's just that the learning techniques used by the big labs, that is to say, various forms of Reinforcement Learning, don't transfer to the real world. Because in the real world if you try stuff at random until you figure out the right policy, you end up destroying too many robot bodies. Imitation learning that went wild a couple years ago with ALOHA looks like a great way to get around this on paper, but then you realise that you have to train your robot for hours on every single task you want it to perform and it still won't generalise even between similar tasks. The sim-2-real gap also remains a big obstacle and you can't just train in simulation and transfer to the real world seamlessly; and then the seams end up being more like chasms.

Then there's the problem of generalisation. RL is insane in figuring out the dynamics of any arbitrary environment you drop it in at any time. Unfortunately once you have a policy trained in one environment (or one task) you have to train again for the next environment (or task) you want to deal with. The number of different environments, situations or tasks in the real world ends up being overwhelmingly large.

So for example DeepMind has been trying for ever to train various robots to do stuff like grasp arbitrary objects by training in the real world or in simulation and still grasping is an unsolved problem, in the general case; and in fact in most cases.

With LLMs it was different in that there was a www of text to train on and the stakes are lower because there's no sim-2-real gap and there's no need to deal with the real world, in training or deployment, it's just text on a screen. That's very hard to replicate in robotics.

There are major breakthroughs still to be made and they aren't currently even on the horizon. And no, I don't have the answer with my super secrete robotic AGI homebrew project either, just saying :P