[simias]

I'm not saying that you're wrong or that one couldn't come up with a better algorithm but at the same time I wonder if it's a good idea to have ultra-complicated heuristics in something like an airplane. For one thing a complicated heuristic with many inputs and weights is a lot harder to validate and can lead to strange and unexpected failure modes.

Besides that the problem of having a very clever autopilot is that it makes it a lot harder for humans to react appropriately when it messes up, because they both don't expect it and are more likely to panic. The Tesla autopilot accidents are good examples of that, those have seen would've been easily avoided by a human but because the autopilot was usually fine on its own the driver didn't react quickly enough.

An other much more tragic example is the Air France Flight 447 crash in the Atlantic where the autopilot detected a faulty sensor and disengaged and the two pilots managed to get the otherwise perfectly functioning plane to crash into the ocean because they basically freaked out and failed to understand what was going on.

That's a strange paradox in a way, as we move from fully manual to fully automated we have a strange "uncanny valley" for safety where the computer is clever enough to handle most situations which lulls the human operator into a false sense of safety. The operator pays less attention, eventually starts losing their skills and reflexes and then at some point, maybe years later, the computer messes up and you have a handful of seconds (if even that) to remember what you're supposed to do.

On the other hand a simple but predictable autopilot might not be quite as autonomous but at least it's easy to understand and anticipate, and it forces humans to remain attentive.