[aeturnum]

This reminds of of a 2019 paper on "moral crumple zones"[1] which talks about how the human component of automated systems are increasingly there to act as the focus for moral failures. Did your giant automated system do something bad? Blame the one human who was assigned to somehow stop that from happening, no matter how impossible that might be.

[1] https://estsjournal.org/index.php/ests/article/view/260

[aftbit]

Also see Normal Accidents[1] which discusses "human error" as a PR cover for systems that are simply too complicated for unaided humans to monitor and understand.

1: https://www.goodreads.com/book/show/192408.Normal_Accidents

[aeturnum]

Normal Accidents is a real classic of the genre of disaster studies and points out some very useful realities for tightly coupled systems. Engineers building highly complex systems would do well to read the book and take its lessons to heart.

[tt433]

The film "Brazil" was mentioned on another comment on another story a few days ago that touched on this theme, very good movie.

[SilasX]

Interesting, that’s a great concept. But I’m not sure of the applicability to this case. I’d expect most people to feel icky contacting a lead on this basis, and so that feels like it’s well within the kind of thing a low level employee should throw a red flag at.

[aeturnum]

The aspect that reminded me of the paper (and the concept) was how the low level employee can really only screw up. If they do well, then it's a credit to their boss, but if they do something wrong it's on them and they'll be fired.