[blub]

You might find Normal Accidents by Perrow interesting. He describes accidents caused by systems (e.g. nuclear power plants) becoming so complex that they are incomprehensible to humans.

"I used to believe that solid components strung carefully together could add up to something understandable. But no, we are beyond that."

I think that's true, but it turns out that we don't have any solid components. None.

[jeffdavis]

If you put two nuclear power plants on the same grid, it is pretty hard to imagine how a meltdown of one plant would trigger meltdowns elsewhere on the grid (because the grid carries electrical energy and is incapable of carrying high-speed neutrons). But with software, you don't have to imagine such failures, they happen all the time (because the internet can carry any data, including more software). So I still maintain that software systems are more complex. And if they aren't more complex today, they will be soon, because the complexity is growing without any obvious bound.

But for the sake of argument, let's say they are of comparable complexity. If you show a layperson a nuclear power plant, and say "who do you think should run this: you, or a team of nuclear engineers?" they would probably answer "a team of nuclear engineers, please". Show the same person a television, and they will feel like they should be able to operate it. But it's actually an internet-enabled TV running sophisticated software that is on the same home network as your internet-enabled security camera system, and it's a very unsettling situation. In other words, now software makes everything -- toasters, TVs, phones, cars -- into incomprehensible systems.