[jacquesm]

It may well be because it simply can't complete that track yet. As soon as it can complete it at all it will probably be near the top performers. But if the situation changes considerably from the training conditions I would still expect it to fail, as well as with relatively minor (but unexpected) changes to the track.

[bri3d]

I think that the idea of using the residual model (trained on high-resolution outside-in pose estimation) to correct the inside-out model is particularly interesting because it might actually be possible to generalize; if the kinematics residuals (basically the delta between simulation training and the real world aerodynamic + inertial behavior of the aircraft) are able to be made general enough to reflect the flight dynamics of the drone, rather than the specific actions on a specific course, this is a promising approach for general purpose flight.

It's a really fascinating approach compared to most "AI-guided" drones which use models for vision and pathfinding but a traditional IMU+PID loop for kinematic control.

[jacquesm]

It reminds me of the way the Mario Brothers game changed overnight: for the longest time it was considered impossible and then suddenly it was unbeatable. But both benefit from the static course, if it generalizes it will be a game changer, otherwise, I'm not that impressed. But there may well be some gold to be found here and I applaud them for making it work this far. Maybe they could purposefully improve their performance in real world situations by making this one harder, for instance by changing the lighting from one run to another, introducing or removing obstacles or moving goals around. That might force the model to come out more general. We've seen similar strategies used with good results in image classification problems. In fact I used them myself when building the lego sorter, as long as everything was always lined up perfect it worked a lot worse then when introducing various complications. During the real world runs those would show up all by themselves anyway and where before they were classified wrong or ended up in the recycling bin for another shot they were suddenly classified right.

Of course a setup where you can gather your training data with thousands of images per hour has some advantages over one where if you get it wrong you have to rebuild your drone...

[supergeek]

A changing course is one of the biggest impacts of flying outside. You have the wind directly acting on your drone, and you have the wind acting on the gates and flags. Flags will spin around in the wind, double gates will lean, sometimes quite a lot in the wind.

There's a whole skill to feeling the wind on your body and anticipating how the drone will behave. When I feel a big gust of wind I'm going to slow down out on the course to get my bearings.

[wiw2]

Is the Mario Brothers reference to some AI in "solving" auto-playing Mario successfully?