[VeninVidiaVicii]

For the past two years I have worked in the local emergency room as a technician working on getting some clinical experience. Let me tell you, in the constant face of life and death, you really just develop a kind of "danger fatigue" and even the most critical moments become somewhat prosaic.

You begin to develop a false sense of security after nobody really does from a gunshot wound. Then, someone septic comes in, and "seems" fine, and they're dead in a few hours.

I'm not sure what this kind of logical fallacy is, but I suspect it's similar in a government environment, where you're constantly at RED ALERT. The risk of danger just seems overstated, even when it isn't.

[utborin]

A very similar phenomenon happens in aviation—we call it complacency. Thousands of successful takeoffs in a row make it hard sometimes to remember that each one is a completely independent event.

The way I fight it is by explicitly reminding myself that just because something worked yesterday, that doesn't mean I can skip a step today or let my guard down at any point.

It really does take deliberate thought though. Funny how the brain works.

(The upside is that every successful takeoff becomes a delightful surprise!)

[brazzy]

Same thing for rock climbing, and complacency born from routine is exactly the problem: you tie knots that your life will depend on every day, thousands of times, and then stuff like this happens to some of the most skilled and experienced people:

- get distracted while tying the rope to your harness and leave the knot unfinished, fall 20 meters from the top of the climb (Lynn Hill, by sheer luck only broke her foot and elbow)

- use a slightly unusual rope setup and when preparing to be lowered, tie in on the wrong side of the anchor, fall 14 meters onto rock (Rannveig Aamodt, broke her spine, pelvis and ankles)

- have your partner point out damage on your harness, shrug it off because there's plenty of safety margin, continue climbing for 3 days in a manner that puts repetitive abrasion on exactly that part of the harness, have it snap while rapelling and fall 150 meters to your death (Tood Skinner)

[bloak]

Also on the roads. So many people drive round a blind corner at high speed because they do the journey every day and the road is always empty. Until one day it isn't.

[Schwolop]

Sometimes there are comparatively simple lifehacks you can establish to prevent complacency from leading to problems.

A little example from my life is forgetting to secure the buckle of my motorcycle helmet. Once I've done everything else and gotten my gloves on, it's a pain to take them off again to buckle it up if I forget, and a few times I just shrugged it off and rode anyway.

Then an instructor suggested he always follows exactly the same procedure each time, to the point that he always even puts the same glove on first. It made me wonder if I should do that, and in thinking it through I realised if I changed my order to always buckle up before putting my glasses back on, I'd never forget. There's no way I'd ride off without the glasses (because I can't see without them!), so if that step can only come _after_ buckling up, then there's now no way I'd ever not buckle it up either.

[andreareina]

Not quite what you're talking about, but very much related is normalization of deviance:

What these disasters typically reveal is that the factors accounting for them usually had “long incubation periods, typified by rule violations, discrepant events that accumulated unnoticed, and cultural beliefs about hazards that together prevented interventions that might have staved off harmful outcomes”. Furthermore, it is especially striking how multiple rule violations and lapses can coalesce so as to enable a disaster's occurrence.

[1] https://danluu.com/wat/

[yitchelle]

I hear what you are saying and agree with the danger fatigue observation, but it is different to the managers at NASA and Boeing that are reviewing the slides.

By default of your situation, you are facing these life and death decisions on a constant basis. The NASA and Boeing managers do not. I can't imagine they are part life and death scenarios very often, if at all.

Critical thinking failed them that day.

[jplayer01]

I don't think he's that far off the mark. Take the seals. This wasn't a one-off decision. This was a problem that was known for years and repeatedly ignored at lower and lower temperatures (thus lowering the allowed limits far below the original specifications) until the disaster. The reasoning being "well, you warned us last time, but everything went fine, so let's just keep going, it'll keep being fine".