[gpcz]

Most safety standards that address software's contribution to hazards use some notion of "controllability" to determine risk. For example, ISO 26262 explicitly calls it controllability [1], and it directly contributes to how rigorously the software functionality has to be tested.

MIL-STD 882E uses "Software control categories" that determine the risk introduced by software [2, page 15-17]. Since those definitions are hard to track, they also provide a table near the end that tries to be a little more specific [2, page 96].

In other words, the safety community already takes into account the need for human or mechanical intervention when it comes to risks introduced by software.

[1] https://neweagle.net/how-iso-26262-2018-update-affects-you/ [2] https://www.dau.edu/cop/armyesoh/DAU%20Sponsored%20Documents...