| HN Mirror

I don't think that there's any inherent difficulty to do with physical bodies, it's just that the learning techniques used by the big labs, that is to say, various forms of Reinforcement Learning, don't transfer to the real world. Because in the real world if you try stuff at random until you figure out the right policy, you end up destroying too many robot bodies. Imitation learning that went wild a couple years ago with ALOHA looks like a great way to get around this on paper, but then you realise that you have to train your robot for hours on every single task you want it to perform and it still won't generalise even between similar tasks. The sim-2-real gap also remains a big obstacle and you can't just train in simulation and transfer to the real world seamlessly; and then the seams end up being more like chasms.

Then there's the problem of generalisation. RL is insane in figuring out the dynamics of any arbitrary environment you drop it in at any time. Unfortunately once you have a policy trained in one environment (or one task) you have to train again for the next environment (or task) you want to deal with. The number of different environments, situations or tasks in the real world ends up being overwhelmingly large.

So for example DeepMind has been trying for ever to train various robots to do stuff like grasp arbitrary objects by training in the real world or in simulation and still grasping is an unsolved problem, in the general case; and in fact in most cases.

With LLMs it was different in that there was a www of text to train on and the stakes are lower because there's no sim-2-real gap and there's no need to deal with the real world, in training or deployment, it's just text on a screen. That's very hard to replicate in robotics.

There are major breakthroughs still to be made and they aren't currently even on the horizon. And no, I don't have the answer with my super secrete robotic AGI homebrew project either, just saying :P