[minsight]

It's pretty easy to think that software is basically designing search engines and providing a place for someone to play Farmville. But keep in mind that there are many places where software and automation plays an integral role in a process that can become dangerous very quickly. My resume includes stints designing thermal imaging software to control glass plants, implementing shutdown and notification controls for power generation plants and writing automation control logic for drywall plants. There are a bunch of opportunities for people to get hurt or killed in places like this. The people who build and manage these places require that the architecture of such systems be performed by engineers because of the legal jeopardy that would exist if they didn't have such requirements. This legal jeopardy extends into plant control and automation and employers in these areas insist that workers be certified professional engineers, too.

[WkndTriathlete]

If those places are relying on software to keep people from getting killed you should assume they are already dead.

The Therac-25 case study (linked elsewhere in these threads) is an excellent study on why fail-safes should be mechanical in nature and based on well-known physical processes that have very few, well-studied states. Software has an effectively uncountable number of states; there is no way to guarantee that a complex software system won't fail at a critical moment due to it being in an unexpected, untested state.

There are ways to model software to prove the absence of certain bad behaviors - TLA+, proof languages, and whatever software was used to validate seL4 come to mind - but they are impractical to use on large systems.