Hacker News new | ask | show | jobs
by zb 1928 days ago
The film “Challenger: The Final Flight” on Netflix includes interviews with the most important participants in the launch decision, including McDonald. (It also has interviews with many of the astronauts’ families and does a great job at putting their experience front and centre.)

The guy whose job it was to send the fax because he happened to be the one who knew how to operate the fax machine is to this day utterly devastated; weeping.

The MTI VP who signed off (although the decision was made above his head) admits that he agreed with it at the time, but acknowledges it was a mistake. He’s being interviewed in a large room full of very expensive furniture.

The NASA manager who bullied them into agreeing to launch because they couldn’t prove that it was unsafe basically says that if he had his time over he would kill all of the astronauts again. Truly terrifying.

4 comments

It's interesting to contrast this with the Chernobyl HBO documentary. Both are engineering disasters, both have very complicated cultural and political underpinnings to why they were allowed to happen. It's not to say the Challenger disaster is comparable to the scale of the Chernobyl disaster, but more crucially: what if the same poor incentives and decisions in place that cause Challenger caused other engineering disasters in the US.
HBO’s Chernobyl is more of a dramatization than a documentary. It certainly doesn’t contain interviews of people who were actually involved. There are quite a few good bits highlighting the dangers of nuclear energy, particularly in the context of the Soviet bureaucracy, but there also a few liberties taken with the science and reality of the event. The fact that people think of it is a documentary despite that is also concerning.
I'd go even further and say that the way most people treat "documentary" to be equivalent to "unbiased recitation of factual events" is problematic itself.

I love docs but often research the subject after watching and it's INCREDIBLY rare to see a doc that doesn't play fast and loose with the facts for the sake of creating a dramatic arc or thrilling moments.

It's ESPECIALLY true in "true crime" docs. The director has an idea of painting the subject as either sympathetic guy who was wronged by a corrupt system (Making a Murderer) or evil mastermind (The Jinx) just to give two recent examples.

Turned out years later the giant reveal at the center of The Jinx which made it such a viral hit was 100% manufactured by the director cutting up audio to make Durst say things he didn't. He also lied to the police about the audio so it wouldn't spoil the ending of the doc.

Jarecki never had to apologize for the blatant dishonesty in the doc, never had to give back the Emmy. It's still universally acclaimed.

https://www.nytimes.com/2019/04/24/arts/television/robert-du...

Sorry to rant, but the "documentary" film industry is a fucking joke. I have so many more examples...

I agree with this take so much.

It's because with documentaries, the target audience is someone who is expecting to get some semblance of truth from the thing. But the issue is that given two documentaries, A and B, if A is more narratively/cinematically/etc titillating, it will get produced over B. Which means that the cost function of engagement will drive a documentary right up to the constraint of "not lying" as it can possibly go.

If you really care about accuracy, don't watch documentaries. And please if you are one of my friends stop recommending that I watch them to get informed about something.

Yeah, well stated.

The incentives are definitely misaligned. Viewers want and expect accuracy, but most of all what they want is to be entertained and most aren't going to look to hard to see how accurate a really entertaining doc is.

And as you said the incentive for filmmakers is to produce the most entertaining doc possible to get more butts in seats and more streams which means funding and continued work in the industry. Accuracy doesn't really play much into what gets produced and released.

Agreed that people don't understand that documentaries are still narrative. My cinema studies professor emphasized that it's impossible to make an unbiased documentary. You could take security cam footage and it'd still have some bias from where the camera was placed and which footage you decided to show.
Serhii Plokhy’s Chernobyl: History of Tragedy seems a great text on the subject, and while the tv series is undoubtedly dramatised and many characters are morphed into one, the tv series seems to broadly follow the history of events as described in the book. Serhii seems to have a favourable view of the tv series.

Is there something glaring I am missing?

https://en.hromadske.ua/posts/chernobyl-expert-serhii-plokhi...

> Is there something glaring I am missing?

It is worth listening to the podcast that accompanied the TV show. One of the things they mention quite a bit is where they deliberately deviated from the truth for practical/dramatic/pacing reasons[†], had to pick a narrative path from conflicting records, or had to make bits up to fill gaps in the (publicly available) records, and one or two cases where they toned down rather than ramped up an issue for tonal or "no one would believe it was quite that way" reasons.

It is both an enlightening insight into the process of making a show like that, and gives useful context to start on your journey if you want to delve deeper into the real reality of the events.

[†] merging many people into a single character, exaggerating immediate effects, reordering/repurposing actual events (a helicopter did crash but not at that point), pretty much that entire courtroom scene in the final episode, ...

There are quite a few scientific or technical aspects of the HBO show which are completely fantastical and typical Hollywood tropes. Like that the core might blow up like a megaton bomb, or render all of Ukraine uninhabitable.

Edit: oh and that’s without mentioning all the factual errors in the story itself. Like the three volunteers who went into the plant to open the drains didn’t die, but are alive today, cancer free and collecting their pensions. So are most of the people who watched the plant burn the first night on the bridge.

From what I know about it whether or not the core blew up (again, a small part of it blew up in the beginning of the incident and spewed radioactive graphite all over the site) was a dime on its side, the core was well underway towards landing in the water underneath it, the resulting steam explosion could have thrown all of the core all over the surrounding site. That it didn't happen is due to the heroics of a couple of people who never really made a big deal of it, they went underneath the reactor core to manually open the valves that drained the basin.
This is correct. However the writers for the show played it up to be much larger than that. In one of the episodes Gorbachov asked how big the explosion would be, and the reply was somewhere in the multi-megaton range IIRC, complete with a description of the predicted damage to the surrounding area equivalent to a major nuclear blast.

The biggest steam boiler explosions in history were still many orders of magnitude less than that, and those were purpose-built pressure vessels. The core wasn't going to drop into a pressure vessel, just whatever makeshift containment they had enacted at that time. Had the core come in contact with the water it would have converted a large chunk of it into steam, which would within moments blow open whatever cracks or leaks existed in the containment, blowing a lot of radioactive rubble into the surrounding environment.

That would have been a huge setback, but nothing near a multi-megaton nuclear explosion.

The epilogue of the show makes it very clear that the three survived and at the time of broadcast two were still alive.

Also, I believe the Soviet authorities at the time may have incorrectly believed that a large explosion was possible - in that respect the show may be correctly repeating a mistake that was made at the time.

If memory serves this is confirmed in the podcast - i.e. the soviet engineers at the time believed it, even if we now know that it was unlikely.
I have to read the official reports yet, both of them. But from what I understood, while it turned out the massive steam explosion was no real threat, the sincerely believed it would happen. And the three guys draining the reservoirs lived, one died in 200X (I can't remember), the other two are still alive.

There are other things I don't like about the mini series, but really just minor ones. The last episode was a wasted opportunity, so. Using the Vienna meeting would have been the perfect setting to cover the international reaction as well.

That being said, I saw a lot of similar decision processes in my career in purely capitalist jobs to the ones that lead to the screwed up test in Chernobyl.

Thanks for pointing the bridge thing out. For starters, nobody knows who was there that night. And since nobody counted deaths, because nobody wanted to know, the series final just put a lot of urban legends out there. Not that the fact the nobody wanted to count isn't troubling enough in itself.
> Not that the fact the nobody wanted to count isn't troubling enough in itself.

This crappy trait isn’t limited to that time and place. The incident that comes to mind is gulf war 2 Bush “We don’t do body counts”.

https://www.thenation.com/article/archive/we-count-they-dont...

> The fact that people think of it is a documentary despite that is also concerning.

That's why I don't like entertainment that stylizes itself as factual. For general audience, there are only two modes of understanding: either something is obviously fiction, or obviously reporting. There's no middle line.

From the shows that try to blend the two, you get things like people believing fictionalizations in HBO's Chernobyl and then becoming opinionated on nuclear energy; people learning history from docudramas; people thinking Top Gear is factual and not staged; people thinking all those performers on talent shows are actually doing these things for real...

I had assumed from the comment that there were two shows, one a dramatization and one a documentary. Surely no one thinks that fiction was a documentary?!
softwaredoug (GP poster) used the word “documentary”
And I was confused by that word choice, thinking I was/am unaware of HBO's Chernobyl documentary, which they must have produced to accompany the excellent film/miniseries.
I have friends who keep citing 'The Big Short' for what's wrong with our financial system.

A lot of people can't tell the difference between fact movies and fiction movies.

And that doesn't even get into clearly biased documentaries (but I assume that most documentaries are at least trying to be factual instead of entertainment...)

The book “The Big Short” is a documentary. The movie is based reasonably closely on the book. The movie is a fictional re-telling, but the people are real and their motivations and actions are accurate.
The Jenga scene to describe CBOs is quite cringe to me.

If a BBB tranche goes under, the investors in the BBB tranche get nothing to protect the AAA tranche (and above). In effect: BBB tranche can fail safely, that's the entire point of them.

That's why they only shorted the BBB tranche (with exception of Brownfield Capital, who did go all the way to the AA tranche). AA was safer and more reliable: so for a short its a riskier move to short.

------

The scene does in fact lay the ground basis of tranches and CDOs, which is better than most Hollywood movies. But its still filled with misconceptions, and the Jenga tower (though dramatic) isn't helping at all.

---------------

Synthetic CDOs was very poorly described. The "rating agency" scenes were pretty much purely fiction and just designed to enrage the audience and IMO unhelpful to the general discussion. Etc. etc.

----

CDOs of CDOs were accurately described IMO. Hammed up by explaining the "yesterday's fish in today's soup), but that at least is somewhat of an accurate analog.

------

I mean, it was a solid movie. But look, I know how reality works. I've actually taken the time to look at (some) of the Congressional Hearings and read some of the papers for how that whole thing worked back in 2008. And there are also some good Frontline Documentaries on the whole 2008 crisis in general.

>but I assume that most documentaries are at least trying to be factual instead of entertainment

Given the recent thread on adam curtis documentaries I'm inclined to believe the opposite. https://news.ycombinator.com/item?id=25880448

Same.

By the way, I thought Inside Job was a vastly better film about the crisis that was criminally under-rated. Definitely worth checking out.

Not perfect by any means, but far more informative than The Big Short, and not as dumbed down.

A docudrama might be a more appropriate description. Nevertheless, Chernobyl on HBO was haunting and forces you to stop and think how a similar tragedy could be avoided. More broadly speaking it forces you to think about systemic power structures and perverse inventives that lead to these situations. These two are outliers, you need to ask yourself how many millions of times does this happen with localized/limited consequences.
I would have said it highlights the dangers of Soviet bureaucracy and tyranny, particularly in the context of nuclear energy.
The only "soviet" factor was, IMHO, the reactor design and the fact that the behaviour during shut down was kept secret. All other decisions can happen exactly like that in any other environment. Upper management ignoring risks for promotion? Yep. Bad risk management and safety culture? Check. Ignoring procedure to meet deadlines? Check. That emphasis on the "soviet" angle is the one criticism I have against the HBO series. Small things like getting the evacuation of Pripyat wrong (they evacuated before the west knew what happened, not after), and especially during the final episode which completely ignored the international reaction, or non-reaction. Also the over dramatic death toll, nobody knows how many people died because nobody wanted to know, not the Soviets nor the West.
I am in the middle of INSAG-7, the revised Chernobyl accident report from the 90s. And the positive reactivity effect of the control rods was, as shown in the HBO series, known since 83. The RBMK chief engineer suggested changes, technical and in procedure, immediately after that. These have not been implemented, because it was considered to be an extreme edge case. Might have been nice to portray it that way.

After all, I love the HBO series, watched it three times by now. Still one of the best mini series ever produced. As shown by the fact that you have to dig that deep to find deviations from reality. In most other cases, you don't even have to scratch the surface.

INSAG-7 includes a soviet report as annex. That report is really fascinating. Already in 1976 the soviet authorities and relevant institutes were aware of the design related issues of RBMK reactors and had identified necessary changes and modifications. Obviously, none were taken, but all these recommendations were included in the initial soviet report on the Chernobyl disaster.

And in my lay man eyes, the RBMK reactor (which was also found to violate soviet requirements from the 70s) was a disaster waiting to happen. Inherently unstable, optimized for grid stability instead of safety, lacking control and monitoring, erratic behaviour under certain conditions and no clear operating procedures.

Edit: Also nice is that the first reaction was to blame the operators and not the system as a whole. Kind of what always happens with aviation accidents as well, it always the pilots fault first.

The BP Deepwater Horizon oil spill and Boeing 737 Max, both of which have killed more people, come to mind.
We all deal with the "normalization of deviance" all the time, in large and small ways. Like being in a group where it's "ok to take off your mask" or basically any form of teenage peer pressure. These flaws are basic to humans for obvious reasons (we want to be cool / successful / not problematic) and it takes massive courage to blow the whistle.
In fact, in the social sciences, this is a known phenomenon called "normal accidents" [1].

[1] https://en.wikipedia.org/wiki/Normal_Accidents

I think both of these events together undermined people's general faith in governments to accomplish large scale projects. The two largest superpowers each failed in a big way at a task which should have been within their capabilities.

It shook the assumptions and foundations of modernity and we lurched closer to overvaluing the virtual accomplishments of economic growth and financialization. Wealth is being used to create more wealth, it is not serving any productive purpose, directly or indirectly:

https://en.wikipedia.org/wiki/Financialization#Roots

>In the United States, probably more money has been made through the appreciation of real estate than in any other way. What are the long-term consequences if an increasing percentage of savings and wealth, as it now seems, is used to inflate the prices of already existing assets - real estate and stocks - instead of to create new production and innovation?

We kept trying after the Apollo 1 fire, we didn't really keep trying after Challenger or Chernobyl. Instead of trying to accomplish truly great and difficult things, we became satisfied with making numbers go up on a Bloomberg terminal.

> The NASA manager who bullied them into agreeing to launch because they couldn’t prove that it was unsafe basically says that if he had his time over he would kill all of the astronauts again. Truly terrifying.

I only know two people who spent their careers at NASA, but with my small sample size this isn't surprising at all. Both of them have never been wrong in their lives. Their hubris is off the charts.

Well I did work at NASA, although I left for the private sector to finish my career, and this does not describe 99% of the people I met. In the circles I ran in there was a great deal of humility and respect for process as something which saves lives. Maybe it depends on the center? Which part of NASA did your friends work at?
I'm at GSFC, and arrogance is definitely not a character trait of those on my team. Earth science may select for a different type of persona.
For the sake of completeness:

I don't work at NASA, and I know a few arrogant people who also don't work at NASA.

Now the question is, is he wrong? Or are you proving his point? If he is indeed correct your post could still be true seen with your eyes.
If my personal experience is helpful to those trying to answer those questions then great, but I don't know the answer to them.
While we’re dunking on nasa , I gave my resume to a recruiter for the nasa jet propulsion lab at a career fair and he said they’d call me and they never called me. Bunch of liars!
For perspective, 17,000 people work for NASA.
You don't manage to get that high up the ladder if you let a silly thing like your conscience bother you.

(This is not a good thing, to be clear, and whoever figures out a systematic solution to this problem will save countless lives in many generations to come.)

I doubt the people high up the ladder think themselves without conscience. Quite the opposite.
I think there are people in both the camps. There are some with consciences that believe in doing the right thing, but also those that are in the sociopath camp. The news is full of the bad leaders that take ridiculous risks (e.g. Boeing) but the good guys don't make the news often.
If there had been good people in power at NASA at the time, the most important thing they could have done to save lives (not to mention run an effective organization) would be to refuse to promote folks without a demonstrable conscience to management positions.

It's all well and good to celebrate Allan McDonald for courageously speaking up, but the hard question is this: had Larry Mulloy or George Hardy ever been in the position where they had to make a decision about whether to courageously speak up, at any point between when they joined NASA and when they got their management roles?

If your promotion criteria says that someone's qualified for a higher-level job because they've been doing good work in fair weather, you have absolutely no way to know whether they listen to their conscience - you simply have no data about what happens when they have to make tough calls. And in fact your process is slightly biased against people who do, because sometimes people who don't will take an unwise risk and get lucky. And so over time, as long as disasters remain less common as worries about disasters, the folks who don't listen to their consciences get a little bit more done during their career compared to their peers.

We know that Thiokol demoted McDonald for speaking up and sidelined the others who also did. This is a system that systematically avoids empowering the good guys.

(And, from all evidence, Mulloy and Hardy were highly capable engineers. I'm not saying NASA should have never hired them - they should have had senior IC roles to fit their strengths and NASA should have looked for folks more like McDonald to make the launch decisions.)

> If your promotion criteria says that someone's qualified for a higher-level job because they've been doing good work in fair weather, you have absolutely no way to know whether they listen to their conscience

How can we possibly know if someone else listens to their conscience? Their conscience is not accessible to anyone else.

I mean, we can go back to the original article for that. Why are we praising McDonald? Why did his obituary get posted here, and why did Mulloy, who passed in October, not get any press? Their consciences were unknowable, yes, but fortunately we're not actually looking for the ineffable conscience. McDonald did a praiseworthy thing, which we wish to encourage. And maybe Mulloy had a stronger conscience, but he just was more deferential to the pressure on him to launch. Maybe he did listen carefully to his conscience, but he had too much of a sense of optimism and so didn't internalize the worry. Who knows? In the end, whatever the reason, he pushed for Challenger to launch.

We're looking for whether someone is empirically willing to make a decision that's unpopular but right, whatever the reason. If they did something like McDonald did, where they were under pressure (including career pressure) to do something, they refuse to do it, and the data eventually shows they had good reason for it, then you've got some data. If they do like his colleague Bob Ebeling did and they write a memo to upper management because they don't feel their direct management is taking concerns seriously, you've got some data, too.

What I'm saying is that, if the person you're considering promoting has never faced a hard decision, and you're promoting them because they had the good fortune to face years of easy decisions through which they could do high-quality work, you know you don't have any data. I agree that it's hard to get the data (and there are obvious problems with that metric turning into a target), but if you don't even try, you're certainly not going to succeed.

All the engineering process in the world will not save you if your hiring and promotion processes incentivize the wrong things. Hence my question: did NASA have a process for deciding that Mulloy and Hardy were good at making life-or-death decisions, or did it simply have a process that determined that they were good engineers?

> I doubt the people high up the ladder think themselves without conscience. Quite the opposite.

Some might, and just not care. Psychopaths aren't that rare (although how frequent the incidence is within Nasa is I have no idea)

> Psychopaths aren't that rare

Any evidence for that statement? Wikipedia and its sources suggest otherwise.

https://en.wikipedia.org/wiki/Psychopathy#Frequency

> 1.2% of a US sample scored 13 or more out of 24, indicating "potential psychopathy"

Those 1.2% are only "potential psychopaths" so the real number might be even less. An illness that only affects <1% of a population can definitely be seen as rare.

Funny thing about power ladders and sociopaths, though... they do love to climb.
Alan McDonald, the subject of this thread, is clear proof that this is obviously false.

The fact that so many people believe this is true, however, is why avoidable accidents like this happen. Being willing to compromise your ethics isn’t bravery.

Not a single person with McDonald's sense of ethics had the authority to scrub the launch. And multiple ICs, not just McDonald, had serious reservations.

If you want to say that my "You don't" is technically false because, if you listen to your conscience, you might get proven right after seven people die and a major investigation happens that remains world news for decades afterwards, and even then you'll get demoted and sidelined until the US Congress intervenes, then ... yes, you can. I will rephrase to "You usually don't." (But even so, that just gets you promoted at Thiokol, and my statement stands for NASA.)

Why didn't anyone scrub Columbia's launch? Why did the investigation board say that NASA had most of the same cultural and leadership flaws that the Rogers Commission had raised concerns about?

The SREs have a saying, "Hope is not a strategy." You can hope that the person you promote will have a sense of ethics, and maybe they will, but that does nothing to ensure that you'll have ethical decision-making.

Is it though? The company demoted him and his career was only saved by direct intervention from the congress.
It it were true, the management would face consequences and McDonald would have been put in charge, instead he was sidelined.

How many people do C-levels have to kill for heads to roll?

Don't have 300 rung ladders.
I saw the interview you're talking about and I admit I respect it.

He had two choices: cowardly pretend it's not his fault or admit that given all he knew, he took a risk to break it or make it and broke it.

You don't know how liberating hearing a guy like that say that. In my company, no way someone says that, they'd rather do absolutely nothing than risk anything.

> You don't know how liberating hearing a guy like that say that. In my company, no way someone says that, they'd rather do absolutely nothing than risk anything.

It's also worth understanding the difference in risks and incentives.

Do you work in an industry where someone failing to speak up about issues with the work will directly risk other coworkers lives?

Where I work, the worst that happens from bad decisions is reduced profits. Besides the personal glory of "being right", there's no upside to sticking ones neck out. Especially if your manager is vindictive, and takes your "being right" over him as a reason to punish you. Better to let bad things happen, and then help fix the inevitable clusterfuck.

At the end of the day, reduced profits aren't great, but they're not an existential threat for the biz where I work.

I had the same takeaway. The guys attitude was “if you want to travel in space you need to take risks and based on what I knew this was a risk worth taking”.

Of course it comes across as quite callous considering it’s not his life that’s at risk, but he does have a point (not necessarily a valid one for the o-ring issue, but more generally speaking).

That's what Feynman's issue was as he reported it in the Roger's Commission[0]. Is they _didn't_ understand the risk when they thought.

Management thought the risk of lost was 1 in 100,000 which is launching everyday for 274 years. Engineers polled was 1:50 to 1:200. Obviously a massive disconnect.

The thing that gets me is they broke their own protocol operating below 53'F. This wasn't a calculated risk where it's 1-2 degrees out of spec, it's wildly out of spec, below freezing into a completely unknown, untested and un-spec'ed space.

This is what frustrates me.

[0] https://en.wikipedia.org/wiki/Rogers_Commission_Report#Role_...

Since we are already comparing Challenger to Chernobyl, that fact is a common thread isn't it? Ignoring temp specs, and at Chernobyl they ignored power and operation specs. In case of Challenger to get a launch, and in case of Chernobyl to conduct a test.
He had no idea whether it was a risk worth taking or not, because they rejected any evidence short of certainty that the O-rings would fail on a given flight. The results of the Rogers Commission make this abundantly clear to anybody who cares to pay attention, yet he professes to have learned nothing.
So easy to gamble with other people's lives and money. Let's admit it was really about his position and career, not 'wanting to travel in space'. He wasn't travelling; he was approving dangerous vehicles. He was trusted to delay launches when necessary. It was a complete fail on the VP's part, and all to improve his own 'numbers'. Not some noble goal.

I'm sure he rewrote it in his mind later, so he could live with himself. Because, of course, he was the kind of guy that rewrote things to suit his agenda.

> “if you want to travel in space you need to take risks and based on what I knew this was a risk worth taking”

The first thing we sent into space wasn't alive. The first living thing we sent into space wasn't a human. Taking risks is for idiots, not researchers or engineers.