[nradov]

LLM based agentic systems can never meet current FAA certification requirements for civil aircraft. The problem is that the internal logic is essentially a black box and there's no way to verify that it won't go off and do something totally bizarre in an unpredictable way. Now you could argue that human pilots are subject to the same limitation, and there have been a few cases where humans have intentionally crashed airplanes, but we have an intuitive model of how other human minds work and in an emergency you can depend on human pilots to at least try to save themselves.

[coderenegade]

It would depend on what the data says. By the time putting such systems into civil airframes is even a conversation, there will be an enormous amount of data generated by unmanned systems and military vehicles. If the data shows that a given system is acceptable, I don't see any reason it wouldn't be allowed.

That's not the same as saying that current architectures are acceptable, because clearly they aren't. But both neural net and stochastic controllers have been tested in aircraft since at least the 1990s. Probably earlier, but that's as far back as my knowledge goes, anyway. I don't know of any neural nets doing active control type tasks, but I don't see any intrinsic reason they couldn't be used as long as you can show that they're good enough.

[nradov]

No, you're missing the point. You can't demonstrate safety regulation compliance with data. The entire avionics software development process has to be structured from the start to allow for validation.

Military experience, especially with unmanned systems, is largely irrelevant. They operate on a different set of rules and are generally willing to accept a higher level of risk in order to accomplish the mission.

[coderenegade]

Fair enough, I was under the impression that enough data was sufficient.