Are physics engines not yet accurate enough to enable "virtual" pre-training / full training of the networks, lighting conditions, etc? If they are, exclusively using physical robots seems somewhat inefficient.
Their system evolves a virtual body which is evaluated by comparing its predicted behaviour (e.g. if motor A is rotated by X degrees, sensor B should get response Y) to real physical movements (moving motor A and reading sensor B). Once an accurate virtual body has been made, it's used to evaluate a bunch of (again, evolved) movement styles in simulation. Once an efficient style has been found, it's used to control the physical motors on the robot.
Hmmm... does anyone know if Grand Theft Auto has an API ? I would like to pre-train my autonomous vehicle controller before connecting it to an actual car.
Ideally, yes, we want to pre-train in a virtual environment using as close to the real model robot as possible. I worked on such a problem as part of my PhD research on mobile robots using the Webots simulator (https://www.cyberbotics.com/overview) as my virtual environment.
In my case, I was working on biologically-inspired models for picking up distant objects. It's impractical to tune hyperparameters in hardware, so you need to be able to create a virtual version that gets you close enough. Once you can demonstrate success there, you then have to move to the physical robot, which introduces several additional challenges: 1) imperfections in your actual hardware behavior vs idealized simulated ones, 2) real-world sensor noise and constraints, 3) dealing with real-world timing and inputs instead of a clean, lock-step simulated environment, 4) having different API to poll sensors/actuate servos between virtual and hardware robots, and 5) ensuring that your trained model can be transferred effectively between your virtual and hardware robot control system.
I was able to solve these issues for my particular constrained research use case, and was pretty happy with the results. You can see a demo reel of the robot here: https://www.youtube.com/watch?v=EoIXFKVGaXw
That's a very interesting question. My guess is that the physics of grabbing things, especially non-rigid things, is very messy and difficult to simulate. It would be great if someone here were able to give a detailed answer to this question though.
1. The best / most recent attempt at this was for the DARPA robotics challenge and the Gazebo simulator.
This was still very buggy and prone to hilarious / depressing physics.
2. Almost all game physics engines start from rigid body and slap on particles, deformables, etc.
An exciting counter example to this is nVidia Flex which starts with unified particle simulation (much closer to molecular dynamics simulation used for, you know, real work).
3. From the perspective of AI, accurate simulation might not be required.
Intelligence requires complexity and a certain degree of predictability. So as long as you can build a rich and consistent / learnable world then whatever simulation you have could be super useful.
From the perspective of transferring that knowledge into a robot though you need accurate physics.
4. Natural touch sensors are hard to do in rigid body simulators but are super important to naturalistic learning.
There's a ton of information that your sense of touch and body position provide about how the world works, and getting the tens of thousands of soft-contact touch points simulated you need for this kind of sensing is pretty challenging today.
Lots of physics engines do all sorts of things to minimize contact points, or ignore them if there's no motion. You have to work against optimization a lot if you want mechanoreceptors and proprioception.
I agree with all the points you made but in addition I would add another - with external cameras for positioning and movement feedback, you don't need to have accurate geartrains or encoders nor have a rigid robot. Since the localization is all in software (and software is scalable/free from Google's standpoint) there are potentials for lots of weight and cost savings on the hardware side.
Kind of like my robot:
https://github.com/jonnycowboy/YARRM
I was hoping to use a robotic arm for a project I'm working on, and wondering if you guys could answer a question about motors. In my very limited research it looked like one of the factors that make the industrial (kuka, etc) robots so expensive was that they use backlash-free motors. What does that even mean?
I also saw a couple startups aimed at sub-$5k robots (like carbon.ai). Are they solving this problem in some novel way?
Backlash free motors are motors where the output shaft begins moving as soon as the motor starts moving. In particular, when the motor reverses direction there is no "slack" to pick up before the output shaft starts to move. The slack is called backlash when talking about gears and motors and what have you.
It's important for robots to not have backlash, because as movements are repeated, each bit of backlash adds up into a potentially big cumulative error. It could end up with the robot operating outside of the intended design envelope, which might be a safety problem.
I think you mean backlash-free gearboxes (ie: cable driven gearboxes, harmonic drives or spring-loaded gearboxes that always apply a minimal but constant tension).
It is difficult, but doable assuming Coulomb friction.
The two main issues are that it is computationally expensive, but also that your mechanical modeling has to closely match that of the actual robot (especially the contact model), otherwise I suspect the training data will be useless in the end.
So if you can afford an actual robot, it makes sense to do the training using it.
How computationally expensive ? Are we talking supercomputer time to simulate the few seconds it takes to grab an object ? Advanced robots are expensive too and usually much harder to get access to then computational ressources (ie. AWS).
It depends: rigid/non-rigid objects, stiffness for non-rigid objects, approximate/exact Coulomb model, spatial/ temporal resolutions, and solution precision.
On a typical desktop computer, that would probably range from real time for the fast/imprecise simulation, to maybe one day for a full-blown simulation.
But again, most roboticists will tell you there is a world between the simulation (even an accurate one) and the actual robot.
Gazebo with Mike Sherman's physics engine might be good enough. DARPA paid to get a decent physics engine into Gazebo; the ones from games were never quite right.
There are things you can't simulate (yet). In my experience it's beneficial to run real live testing to gather data about individual parts themselves. For example, I had a robot's navigation fail when it encountered a certain type of water container (one gallon type in a given color found in US supermarkets). Like kissing, you can't replace the real thing.
Their system evolves a virtual body which is evaluated by comparing its predicted behaviour (e.g. if motor A is rotated by X degrees, sensor B should get response Y) to real physical movements (moving motor A and reading sensor B). Once an accurate virtual body has been made, it's used to evaluate a bunch of (again, evolved) movement styles in simulation. Once an efficient style has been found, it's used to control the physical motors on the robot.
Also related, their lab has a "universal gripper" made out of a balloon filled with coffee granules: http://creativemachines.cornell.edu/positive_pressure_grippe...