[giantg2]

Another NASA screw-up that they're trying to pin on the vendor engineers, just like Challenger.

The title is not reassuring. Conservatism in engineering is essentially about creating safety margins through conservative estimation. The title is saying we need to be careful because a tile was likely penetrated. Hell, if I remember correctly they were reporting that there was known tile damage on the news before reentry, but that they didn't know the extent.

"NASA felt the engineers didn’t know what would happen but that all data pointed to there not being enough damage to put the lives of the crew in danger."

If you thought they didn't know, then ask them what they do know! It's right on the slide that flight conditions are outside of test parameters and that the mass of the projectile was much higher. How the F do you work at NASA and not understand the basic principles of mass, velocity, and energy well enough for that to stand out enough to ask questions or run your own calculations...

The reason the slide is laid out the way it is, is because it's describing the thought process and creates a deductive argument for how they got to their concern. This is a presentation for a briefing for other engineers, not a conference or sales pitch. It's supposed to be formal and contain the synopsis of technical points. Using projectors for technical briefings predates the use of PowerPoint. I see nothing wrong with the layout in that context.

Edit: why downvote without a reply? NASA has a history of blaming vendors when they screw up. This looks like another example to me. The presentation format does not have any issues given the setting and target audience.

[lostcolony]

Tufte is not an employee of NASA. He is an employee of Yale, and a thought leader in information design. HE was the one saying the slide design is poor, and doing so not in the interest of assigning blame, but in the interest of highlighting ways to communicate better.

To say that "the slide doesn't have any issues" is laughable on the face of it. But it's immaterial; your claim is that "NASA just ignored the engineers from Boeing" rather than "NASA didn't understand the engineers from Boeing". Communication is a two party process, and believe it or not, NASA isn't actually incentivized to take risks that lead to loss of life and damages public perception of them; it's far more likely they didn't understand the stakes, and looking at the slide from that perspective, it's very easy to see why they would not have understood the stakes even if the Boeing engineers did.

[giantg2]

I understand that he's not a NASA employee. Do you think it's fair to claim that the slide killed 7 people? I don't. Could it be worded better or have a better layout - sure. But there's no problem with that slide that would support the claim that 7 people died because of it. The information was there.

As you said, it is a 2 way street. Slides are accessories. Do you have the conversation that unfolded during this slide and presentation? Did the audience ask questions about things they didn't understand?

Is there even any evidence that NASA didn't know about the damage or had a rescue plan?

You claim they wouldn't have taken the risk, yet if I remember correctly they had no rescue plan and gave a relatively low (70%ish maybe) survival rate. Low level employees did raise concerns about severity of the damage. This seems to support the idea rather the communication between the vendor and NASA was sufficient since some NASA employees shared the same view.

https://www.dailymail.co.uk/news/article-2271525/It-better-d...

[bumby]

There was serious consideration to sending Atlantis as a rescue mission as Columbia was not in a position capable of rendezvousing with the ISS to use the later as a lifeboat. To your point, subsequent missions were required to have a formal rescue mission outlined.

[andi999]

Agree. Also where is the executive summary for that slide on that slide?

[logifail]

> Also where is the executive summary for that slide on that slide?

Q: Shouldn't every PowerPoint slide be an executive summary? PowerPoint can be a terrible way to [attempt to] present detail.

[ziml77]

I believe they were using that question to point out the absurdity of making a slide that long by suggesting that it is made even longer with a summary of itself.

[krisoft]

> The title is saying we need to be careful because a tile was likely penetrated.

It is a godawfull title. I believe that is how it reads to you but it totally reads the opposite to me. I read the title and it translates in my head as “we reviewed the test results and they suggest that the tiles are built sturdy enough to not get penetrated”. Exactly because what you say the word “conservatism” means to me that a system is designed to meet the loads plus reasonable safety margin. So if the review of test data indicates conservatism that means to me that the test found the test object roboust even with a reasonable safety margin. Otherwise i wouldn’t say that it “indicates conservatism” but that it “indicates lack of safety margin”.

> The reason the slide is laid out the way it is, is because it's describing the thought process

I agree, but that is not a good thing. People think in all kind of haphazard ways, before you communicate to others it is on you to look at your ramblings and make it orderly. The penultimate sentence is the most important one that should go first “flight conditions is significantly outside of test database”. That doesn’t mean that the tile is broken, nor does it mean that it is not broken. It means that we can’t tell from our tests.

[SiempreViernes]

I read the title as saying "be careful when predicting tile penetration" based on one of the first point they make being that the method used to predict penetration "overpredicted penetration of tile coating significantly".

Though in fairness they then have sub points that kind of contradict this conclusion rather than support it.

But honestly this slide would make more sense if several of the sub points where top level independent statements instead.

[josefx]

> based on one of the first point they make being that the method used to predict penetration "overpredicted penetration of tile coating significantly".

That point read to me as if they completely overestimated the damage.

[pdonis]

> Another NASA screw-up that they're trying to pin on the vendor engineers, just like Challenger.

And, for context, Edward Tufte, whose review of the slides is being referenced in this article, is the same one who misunderstood and misrepresented what the actual issue was during the briefing the night before the Challenger launch, in his paper reviewing the presentation the engineers made then.

Edit: previous HN discussions of Tufte's Challenger review here:

https://news.ycombinator.com/item?id=10989358

https://news.ycombinator.com/item?id=19034783

[dr_orpheus]

Yeah, its very hard to say that Challenger was because of a miscommuncation from the vendor engineers. They wrote out directly [0] "Recommendations: O-ring temperature must be > 53F at launch", and the temperature of the air at launch was 36F and measurements on the solid rocket boosters (where the o-rings are located) got down to 25F and 8F [1].

But their recommendation was challenged by the NASA SRB managers. And after an offline discussion the SRB vendor came back and had changed their opinion that it was safe. And the NASA SRB manager never brought up the o-ring temperature concern to the rest of the management team.

[0] https://history.nasa.gov/rogersrep/v1p90.jpg

[1] https://en.wikipedia.org/wiki/Space_Shuttle_Challenger_disas...

[pdonis]

> their recommendation was challenged by the NASA SRB managers. And after an offline discussion the SRB vendor came back and had changed their opinion that it was safe.

Yes, because the Thiokol managers overrode their engineers and said it was OK to launch.

> And the NASA SRB manager never brought up the o-ring temperature concern to the rest of the management team.

That was probably because NASA had already classified the O-ring issue as a Criticality 1 flight risk, which is supposed to mean that a failure could result in loss of vehicle and loss of life and that the Shuttle can't fly until the issue is resolved--and then waived it. So it wasn't as if the O-ring issue was a new one or that NASA wasn't already aware of its seriousness.

[giantg2]

Wow, I didn't know that. What a poor track record.

[zelos]

Maybe, but I don't think anyone can deny that it is an appalling slide. I count 3 spelling and grammatical errors alone.

It looks like the bad "before" example in a presentation skills workshop. This was created by engineers working on life and death issues involving billions of dollars of hardware.

[cxr]

> Maybe, but I don't think anyone can deny that it is an appalling slide. I count 3 spelling and grammatical errors alone.

The slide from the post that you're referring to (which is 16:9) is a fabricated, probably created for this post, and is not a faithful copy of the actual deck that was presented in 2003. The actual slide—complete with a Boeing watermark, sans some of the errors and presentation issues we can see in the blog post, and in 4:3, of course—can be seen in Tufte's book.

Who knows why the slide in this post was fabricated (and why the author failed to indicate this fact anywhere).

[bumby]

I agree with you, but to be charitable, this was a real-time and evolving situation where I'm willing to bet the slide was expected to be finished "yesterday"

[phicoh]

This seems to be the kind of slide made by someone without a lot of experience presenting work.

The other issue is, some people really, really, don't want to speculate.

In this case it seems that the person who made the slide probably assumed that the tile could be broken with a high enough probility. But because it was outside all available data, the slide says that we don't really know.

Of course, anybody in a position to make such a go-no-go decision should have enough experience talking to engineers, and seeing this effect in action to recognize the slide for what it is. It is really weird to conclude that based on absence of data, it is probably safe.

[bumby]

>It is really weird to conclude that based on absence of data, it is probably safe.

Considering that's exactly what happened nearly 20 years earlier with Challenger, it seems to be more common and likely the result of a number of cognitive biases. We read these with some hindsight and are disconnected from all the other pressures (schedule, budget, peer, etc.) they are dealing with at the time.

[phicoh]

That points to a far more fundamental problem. Related to information processing higher up in the organisation. Just making better slides is unlikely to solve that problem.

[bumby]

Probably correct, and I have doubts that those types of problems are easily fixed because they're rooted in human psychology. It's interesting to me that the "big" incidents seem to occur every 15-20 years, almost as if there is a new professional cohort who has to learn the hard way. I do think clear communication is a necessary, but insufficient, element of fixing that problem.

[mirker]

One factor for why is that bringing bad news may poorly reflect on the organization, and therefore the person’s career.

[bumby]

It takes some intestinal fortitude to be in a role that is tasked with communicating information people don't want to hear. It's part of the reason NASA created it's "Safety and Mission Assurance" organization after this incident and gave them a completely different chain of command. In theory, that mitigates some of the career threat, but in practice it may be different.

[Iolaum]

I tried to read the powerpoint and it was not an easy task. The main point of the powerpoint is not supposed to be an answer to a riddle of fonts and words.

A quick and dirty re-writing of the title (and slide):

_______________

Review of Test data indicates incident is well outside of safety margins.

- Volume of ramp is 1920 cu in vs 3 cu in for test

- Once tile is penetrated SOFI can cause significant damage.

- Flight condition significantly outside of test database

_______________

Now that should get a reader's attention.

[giantg2]

True, there are better ways to word it. That does leave out some detail that the original one had, like test velocity and not showing the thought process as well (like a formula on a math slide or deductive argument in philosophy).

My main point is that the title claims thus slide is what killed 7 people and basically blames the creator, but leaves out all the other failures. Slide formatting and wording (which ignores the actual discussion that should have gone with it) is really inconsequential compared to the rest of the process in a briefing.

[Iolaum]

The slide being such a mess, makes me think that the speaker's arguments may not point to the danger so clearly either. This makes me look favorably on the title even though it is hyperbole.

Granted this is pure speculation on my part and should be treated accordingly.

[lostcolony]

Hell, it's even probably fair to make an editorial conclusion at the end -

"Given this, we strongly recommend against launch"

[giantg2]

The damage happened during launch. Unless you're talking about how the insulation was old and NASA knew that insulation can, and had in the past, struck shuttles.

[bumby]

>The title is saying we need to be careful because a tile was likely penetrated.

To the point of the article, I think this is the wrong takeaway, meaning the slides were not communicating effectively.

The second point illustrates this. It says the models were overpredicting the penetration. Meaning the models were conservative and the the actual penetration was likely less than what models show. They were setting the table for an optimistic outlook.

The real issue, IMO, is highlighted later in the article where there isn't sufficient fidelity in the tests to back up those claims. Tests after the incident showed the foam acted very differently at the delta-v that actually occurred.

And regarding your point about blaming contractors, the vast majority of work done by NASA is done by contractors. NASA is, to some extent, a pass-through organization that funds other organizations like Boeing, Lockheed, Honeywell, Jacobs, etc.

>If you thought they didn't know, then ask them what they do know!

This gets to the same cognitive biases that led to Challenger, EVA 23 and a host of smaller incidents nobody hears about. Data is not objectively weighed in these situations because of schedule pressure, optimism bias, etc. In this case, most launches were showing foam shedding with no issue, so it lead to a false belief that it wasn't dangerous even though it was out-of-spec. Add to that a slide that says the models are too conservative and you can see where cognitive biases may influence the decision. Lastly, most people like to think they're self-aware enough to identify these biases in real-time, but they aren't. It's also why the incident lead to a separate organization within NASA focused on safety, quality, and risk that has a segregated chain of command.

[ubercore]

I came in knowing the outcome, and roughly the point being made, but still found the conclusion of the slide hard to suss out. There were other failures in the chain as well for sure, but I don't think this is just a hit job on NASA vendors.

[giantg2]

The title of this post is claiming this slide killed 7 people. That's a pretty bold and accusatory claim that seems to leave out the other failures, right?

[TobTobXX]

From the article (emphasis mine):

> This, however, is the story of a PowerPoint slide that actually _helped_ kill seven people.

[giantg2]

Any evidence to support that claim? NASA employees raised concerns about the severity of the damage, which shows the contents of the slide were effectively communicated to NASA engineers, but that leaders ignored them. Thus the slide was not a contributor.

https://www.dailymail.co.uk/news/article-2271525/It-better-d...

[amenod]

Pretty bad title for a blog post that talks about misleading powerpoint presentations. Some readers might make conclusions from the title alone, instead of reading the whole text below.

[lostcolony]

Yep; this slide was worse than useless in that to the given audience it could instead read as an endorsement that launching is fine.

[mypalmike]

Please read the article so you don't come across as ill informed. This was not a launch/no launch decision.

[giantg2]

This wasn't about a launch decision. An audience of engineers would not view this slide as an endorsement.

[raverbashing]

No, even for an engineering slide this is apalling

I'm not so sure how many people are familiar with the term "conservatism" as used here. I'm not. Some might be aware, those who are not aware will just skip over

I read this slide a couple of times. There's no thought order, no connection between the topics (even if we assume people are familiar with the subject) and several typos.

It is not a Powerpoint fault it is a fault of whoever wrote this.

This is an issue with information hierarchy. If this is a risk (and I can't imagine what might have been a bigger risk at that mission) it needs to be brought into attention. Not added to line 4 of slide 7 and be done with it.

[silvestrov]

When engineers are not allowed to use "alarmist language" the organization often changes to use such unreadable texts.

I don't expect "foam strike more than 600 times bigger than test data" would go over well politically. You'd be telling the audience that these people will die while everybody watches. No manager want to be the messenger for that kind of messages.

[techsupporter]

> When engineers are not allowed to use "alarmist language"

This has been a recurring theme throughout my career, the struggle between being seen as "alarmist" and accurately conveying urgency.

We get this in the medical field all of the time. "Outcomes delivered via mechanism seen as potentially contraindicated" or some other spaghetti. I've been in these meetings many times, where we (the individual contributors) have to tell the bosses or peers or partner group about something that might be bad. As in Y2K-style of bad, where it will be bad if we don't address it but if we do address it with the urgency needed, no one will be able to recognize the success for what it is.

As you said, no one's manager wants to be seen as crying wolf all of the time, but post-hoc there's the expectation that a couple of engineers way out at the end of the limb of the tree didn't just wait for the limb to be sawed off behind them, they took out the saw and did it themselves. That they stood up in the presentation and yelled "you're all idiots! This is going to kill the entire crew! Everyone will die, don't you see?! And I'm not standing for it!" just before they rip the badge off of their lanyard or belt hook and then righteously storm out.

[ryandrake]

Yep, a lot of organizations have a cultural taboo against telling the bosses bad news. The way one manager taught me: If it's good news, use plain language and big text. If it's bad news, soften it, shrink it, slather it with jargon, obfuscate it. Don't hide it but don't raise an alarm. Couch any unpleasant language with possible-this and unknown-that. Terrible advice but that's the way a lot of exec readouts work.

[brimble]

Maybe it's survivorship bias and I'm only seeing in-fact-rare good examples, but organizational communication seemed so much better back in the 1940s-~1960s.

And other communication, for that matter. Instructional videos from that era make "pro" YouTube look like amateur hour, let alone modern material produced by industry and government, which is even worse.

[mdekkers]

> How the F do you work at NASA and not understand the basic principles of mass, velocity, and energy well enough for that to stand out enough

Have you ever worked for a business with “management”?

[dekhn]

You're being downvoted because the entire process that lead to this decision has been massively analyzed and the root causes were determined.

[giantg2]

And was it determined that this slide killed 7 people, as claimed in the title? That seems overly dramatic and ignores the other root causes.

[dekhn]

No, it wasn't this slide.

[NotAWorkNick]

In this case the vendor engineers were from Boeing - Ya know, the same company that brought us the 737MAX MCAS fiasco. I mean sure, hindsight is a wonderful thing but given the history of Boeing engineering culture since their merger with McDonnell Douglas I can see the possibility of someone gliding over something inconvenient

(not the one that downvoted your comment btw)