[lucasjung]

I have a master's in aeronautical engineering, and one of my major areas of study was stability and control, which is what your Segway and F-15 examples fall under.

Automated flight control systems definitely do not exercise "judgement:" they have inputs, a transfer function (typically MIMO, these days), and outputs. It used to be that the transfer function was fixed, but more sophisticated systems (e.g. fighter jets) often have many different transfer functions and switch between them based upon various inputs. They don't "decide" or "discover" anything: for any given set of inputs, they will predictably produce a pre-determined set of outputs.

Aircraft stability and control systems are not programmed to care about, or even know about, the existance of the wings. The closest they get to this is that they will know the current states of the control surfaces on the wings. So it doesn't really make sense to say that the F-15 CAS was "not programmed for" the state of missing a wing (although it almost certainly was programmed to respond properly in the situation where it gets no feedback from some of the flight controls). As you say, it's designed to reach a specific goal (keep the plane level) and then maintain that state. If the system detects an uncommanded roll-rate, it will move the flight controls to stop that roll-rate. It doesn't know or care that the uncommanded roll-rate is the result of asymmetric lift due to an (almost completely) missing wing: it's just going to keep moving the flight control surfaces until that roll rate goes away. If the aicraft had been damaged in a slightly different way, it's possible that the CAS would have issued commands to the flight controls that would have departed the plane, but fortunately the handling characteristics of the aircraft remained close enough to normal that the control laws still produced good results.

Unfortunately, this behavior can result in mishaps when a flight control system gets erroneous inputs: when it believes that it is rolling when it is wings-level (or believies it is level when it is rolling). This was a major contributor to the Air France fligh 447 crash. In such situations, it takes judgement to realize that something is wrong and to figure out what to do about it.

[onemoreact]

Ok, it was my understanding that the F-15 adjusted the transfer function based on a feedback cycle rather than simply picking from a list of them. However, thinking back the conversation was ambiguous and I don't have the clearance required to find out the correct answer.

However, while flight control systems have been responsible for plenty of crashes pilots have often mistaken level for non level flight and focused on faulty instruments rather than switching to working backups etc. An automated system can handle redundancy much more efficiently than people in such situations so while it's little value for a person to have ex:7 gyroscopes if they need to pick between them autopilots can gain from access to such information.

[lucasjung]

I never claimed that human pilots are superior to automated flight control systems in every aspect. In fact, I have stated explicitly that computers perform some tasks better and more safely than humans.

I was simply making the point that when you encounter a situation that isn't covered in the instructions, you need juman judgement to figure out the best way to proceed. I also explicitly stated that human judgement is far from flawless, and that if you can develop a sufficiently comprehensive automation program, you can get safer results than you would on average with human judgement.

[onemoreact]

If you feed completely new information into a system it's going to do something. Some times it's even the correct choice, but really people also do the same thing in novel situations. I have no problem calling judgement simply deciding what to do based on the current situation and as soon as you add any form of adaptation then computers can do that. But, I am also willing to concede your probably using a different definition.

PS: IMO, what separates people from machine learning systems is treating everything as training data, a much larger training set, a lot more processing power, and a tendency to explore novel situations. The trade off is efficiency and reaction times. Still, when you get into thrust vectoring, super sonic flight, high g turns, rapidly changing weight/drag/thrust at the same time trading consistency for improved handling of novel situations is probably worth it so I expect the air force uses systems that are fare more adaptable than the civilian world.