[nradov]

There's no reason think that removing pilots altogether would enhance safety. Military experience has shown that mishap rates for unmanned aircraft are significantly higher than similar crewed aircraft. Routine flight operations can be automated pretty well now but where human pilots really earn their pay is with handling emergencies and other unexpected situations. The variety of potential emergencies is effectively infinite so it's simply impossible to write and test code to handle them in advance. Instead pilots have to improvise in real time based on intuition and first principles logic. Current AI technology still isn't capable of that.

[coderenegade]

When I last looked into it (admittedly a number of years ago now) this was true for remotely piloted aircraft, but there wasn't enough data on systems that pilot themselves with humans doing the navigating.

I'm not convinced that humans are particularly effective in emergency situations, especially when they're expected to go from monitoring systems that work 99.99% of the time, to suddenly aviating with alarms going off everywhere. I guess there's not really much of an alternative as it currently stands, but seeing how effective agentic AI is at chewing through data and analyzing systems makes me think that eventually, such systems will be used if they ever get to the necessary level of reliability.

[nradov]

LLM based agentic systems can never meet current FAA certification requirements for civil aircraft. The problem is that the internal logic is essentially a black box and there's no way to verify that it won't go off and do something totally bizarre in an unpredictable way. Now you could argue that human pilots are subject to the same limitation, and there have been a few cases where humans have intentionally crashed airplanes, but we have an intuitive model of how other human minds work and in an emergency you can depend on human pilots to at least try to save themselves.

[coderenegade]

It would depend on what the data says. By the time putting such systems into civil airframes is even a conversation, there will be an enormous amount of data generated by unmanned systems and military vehicles. If the data shows that a given system is acceptable, I don't see any reason it wouldn't be allowed.

That's not the same as saying that current architectures are acceptable, because clearly they aren't. But both neural net and stochastic controllers have been tested in aircraft since at least the 1990s. Probably earlier, but that's as far back as my knowledge goes, anyway. I don't know of any neural nets doing active control type tasks, but I don't see any intrinsic reason they couldn't be used as long as you can show that they're good enough.

[nradov]

No, you're missing the point. You can't demonstrate safety regulation compliance with data. The entire avionics software development process has to be structured from the start to allow for validation.

Military experience, especially with unmanned systems, is largely irrelevant. They operate on a different set of rules and are generally willing to accept a higher level of risk in order to accomplish the mission.

[coderenegade]

Fair enough, I was under the impression that enough data was sufficient.