[hinkley]

This whole thread is missing the fact that the NTSB had a theory that transparency leads to safer airplanes, they tried it, and it works. People hesitate to self-report when it comes with punishment (fines, demotions, or just loss of face among peers). You need a formal “safe space” where early reporting is rewarded and late reporting is discouraged.

Safety is a lot about trust, and there is more than one kind of trust. At a minimum: are you capable of doing this thing I need you to do? Will you do this thing I need you to do?

[zettabomb]

It's not just the NTSB, it's part of things like the Toyota Production System. There's ample evidence to show both that punishment discourages safety and that lack of punishment encourages safety, across multiple industries.

[nyrikki]

Yes this is cross industry best practices.

Goodhart's law also applies, as in the case of the edoor bolts, Spirit intentionally bypassed safety controls to meet performance metrics.

The Mars Climate Orbiter is another example. While unit conversion was the scapegoat, the real cause of the crash is that when people noticed that there was a problem they were dismissed.

The Andon cord from the Toyota Production System wasn't present due to culture problems.

Same thing with impact scores in software reducing quality and customer value.

If you intentionally or through metrics incentivize cutting corners it will be the cost of quality and safety.

I am glad they called out the culture problem here. This is not something that is fixable under more controls, it requires cultural changes.

[StableAlkyne]

> The Mars Climate Orbiter is another example. While unit conversion was the scapegoat, the real cause of the crash is that when people noticed that there was a problem they were dismissed.

Challenger too. Multiple engineers warned them about the O-rings. They weren't just ignored, but were openly mocked by the NASA leadership. (https://allthatsinteresting.com/space-shuttle-challenger-dis...)

A decade later a senior engineer at NASA warned about a piece of foam striking Space Shuttle Columbia and requested they use existing military satellites to check for damage. She was ignored by NASA leadership, and following (coincidentally) a report by Boeing concluding nothing was wrong, another 7 people were killed by a piss-poor safety culture. (https://abcnews.go.com/Technology/story?id=97600&page=1)

[ethanbond]

But but but what about my intuition and gotcha questions about how this could never work in practice?

[hinkley]

The dirty secret of why traffic circles reduce accidents? Stoplights feel safer than they actually are, while circles feel more dangerous than they actually are. That nervousness becomes vigilance, which reduces accidents. It’s also why people intuitively hate them. They’re actually right, but also wrong.

Feeling safe is an illusion that governments try to maintain for their people. It’s one of their biggest jobs. But the illusion has dimensions and it’s hard to keep several going at once.