Hacker News new | ask | show | jobs
by avar 3026 days ago
Even better, the timeline:

    - February 11th: Vendor informed of the issue
    - February 25th: 28 people die because of the issue
    - February 26th: The vendor ships a fix
I'd have loved to be a fly on the wall for that phonecall on the 25th (or early on the 26th).
4 comments

You missed this date--

Feb 21--notice goes out to users to avoid "very long run times". Users do not know what that means, and ignore warning.

https://www.gao.gov/assets/220/215614.pdf (page 9)

"On February 21, 1991, the Patriot Project Office sent a message to Patriot users stating that very long run times could cause a shift in the range gate, resulting in the target being offset. The message also said a software change was being sent that would improve the system’s targeting. However, the message did not specify what constitutes very long run times. According to Army officials, they presumed that the users would not continuously run the batteries for such extended periods of time that the Patriot would fail to track targets. Therefore, they did not think that more detailed guidance was required."

That's terrible. Competent technical writing is criminally undervalued.
But there's also "presumed" and "did not think" in there. When there's a problem with your killing device you probably shouldn't use it until you've clarified what the problem is and don't just assume your end users will use it correctly.

That's like saying "It's fine, the critical vulnerability patch will be applied on reboot", while in reality all your users just suspend to disk and move that annoying reboot nag window behind the task bar where it's out of sight.

“You probably shouldn’t use it until you’ve clarified...” doesn’t work so well for a defensive system.
How long was the off/on cycle? If it was short it would be reasonable to do it periodically. I don't think one can pin the blame on the vendor alone.
Clearly the vendor thought it would be reasonable. They failed at communicating it, though. Putting a number on it would have made things clear: “The system must be rebooted after at most 12 hours [or whatever the appropriate value would be] of operation.”
> When there's a problem with your killing device

Patriot, in the role deployed in that Gulf War, is a “not being killed” device, but not really a killing device.

Per the GAO report[0]

> According to Army officials, the delay in distributing the software from the United States to all Patriot locations was due to the time it took to arrange for air and ground transportation in a wartime environment.

I'm not knowledgeable at all on how software for missile batteries was distributed in 1991 from the US to the Persian Gulf but 11 days doesn't seem unreasonable to me.

[0] https://www.gao.gov/assets/220/215614.pdf

Oh, gosh. Really?

It makes me kind of sick imagining that call.

You're not even curious to see how it was dealt with and how the issue was expressed to the vendor? I'd never be in a meeting regarding deaths of users of my software, because I just make internal webapps, so I just cannot help but be curious as to how one of those meetings would go.
> I'd never be in a meeting regarding deaths of users of my software

I know what that's like.

About 20 years ago I was at a consulting firm supporting an electric and gas utility company. Among other things they had to do something called "markouts" which means they paint the ground at a location in a way that indicates exactly what infrastructure they have in the ground and precisely where it is. Markouts are a government organized thing. Before digging somewhere you can call a number and anybody that might possibly have infrastructure in the ground anywhere near your dig site is required to paint their markouts within a short time period. There are stiff fines if you "miss a markout."

Anyhow there was a data problem with a markout. The field worker was sent to paint a markout at the corner of two streets that actually ran parallel to each other and didn't meet. Instead of calling it in and questioning the task he did nothing. Shortly after a construction worker put a backhoe through an electrical conduit with 15K volts. There was an explosion that was heard for many miles. The worker died the next day. He died painfully.

> so I just cannot help but be curious as to how one of those meetings would go.

Finger pointing, of course. Data was being fed back and forth between systems and eventually somebody else took the blame. The field worker who ignored the markout also was blamed. We did add something to our system so that that kind of data error would raise an exception.

I learned a lot about care and diligence about data from this experience. Data errors are no joke.

Sure but

> I'd have loved to be a fly ...

loving anything about that sad scenario seems impossible.

It's a figure of speech, not a show of approval.
Even loving learning enough to avoid the next one? 'Cause that's what responder wants, to learn. "What the hell were they thinking?" is often the most pertinent knowledge of all.
I'm not, because it would bore me. I see shit like that for breakfast when studying transportation. But if you don't do it, I say it's because of your own primal instincts, so you stuff them down...
Who was the vendor?

Aren't defense contractors required to be on their toes all the time?

EDIT → Found it. The PATRIOT Project Office.