[hackermailman]

For minimizing disaster, the standard grad text for this type of thing is Requirements Engineering: From System Goals to UML Models to Software Specifications by Axel van Lamsweerde. There you learn about creating models and risk analysis, fault tolerance modeling[1], privacy requirements etc., all established methods based on engineering foundations but applied to software modeling and development.

If you mean software development for mission critical things that control movement like aircraft, drones, factory robots etc., these engineers I would assume use verified compilers/toolchains like CompCert project to implement the models they have already formally analyzed http://symbolaris.com/course/fcps17.html but I've never done mission critical work, just dabbled in it to apply methods there to non-critical software.

[1] https://arxiv.org/pdf/1611.02273.pdf - Application-layer Fault-Tolerance Protocols