[jonawesomegreen]

I went to an engineering school in Canada for Electrical/Computer Engineering and this was an example of engineering failure we studied in a course about software quality assurance and engineering ethics. Not sure if its widely used as a case study outside of Canada, but it really stuck with me. Especially when designing critical systems.

There are two types of reactions to the Therac-25 story.

1) We need to put a lot of process in place to review and test and ensure we have proper interlocks in place.

2) They just didn't have enough talent on the team. It couldn't happen here.

You want some people with reaction 1 working on a safety critical system.

[noir_lord]

I'd say there are 3 with the third been

3) I'd never work on life critical medical (or anything else) software, I know I fuck up and even the best can (and I'm not).

I simply wouldn't want it on my conscience that an easily preventable error (in hindsight) hurt someone.

[URSpider94]

This is why Quality is such an awesome field. It's all about how a group of smart, skilled, but fundamentally fallible people can build a product with an arbitrarily high level of safety. If it's done properly, no one individual should be able to create a bug that can cause a failure like this.

[mcguire]

For a second there, I thought you were about to quote the best QA job opening I ever saw: "Want to work with fun, smart people? And make them cry?"

[mcguire]

4) I don't work on anything safety critical, I just work on (millions of dollars per second | stores of personal information) systems. I don't need to worry.

[londons_explore]

For many systems, the lack of you working on it might hurt someone.

For example, imagine you work in a bank on their mobile apps. You decide that implementing automatic bill payments for a certain type of account is too much technical effort and not worth it.

Hundreds of thousands of people with that account now have a subpar experience. They have to manually pay bills, and many will forget, leading to arrears, bankruptcy, payday loans, and prison.

Your decision not to implement that feature to make the lives of your customers better indirectly led to ruining the lives of tens or hundreds of people, who otherwise would have led a good life.

From an ethical point of view, unknowingly ruining a life due to an action or due to inaction, is pretty similar.

[carlmr]

If you think like #3 you're one of the engineers that should work on safety critical software. Recognizing that you may produce errors I think is akin to the Dunning-Kruger effect. It shows me you know enough that you don't know enough, which is far preferable to people who don't know they don't know and thus think they're good.

[LifeLiverTransp]

The nightmare employee in this situation is the "rockstar".

[WrtCdEvrydy]

Therac-25 is the standard Bachelor's Level Ethics and Software Engineering case studies in Florida at least.

[noir_lord]

It was standard on my foundation degree in software engineering years ago.

Ethics and responsible software development was a good unit.

[fuklief]

In my field, we use this kind of example to tell people that testing sucks or is not enough and that we need to formally/mathematically prove that software is free of bugs for critical systems.

[nugator]

What field is that and how do you achive that?

[throwaway5752]

What you described is testing.

[philjohn]

It's widely used around the world for teaching about engineering failure.

[ubertakter]

3) If you have a machine that blasts people with ionizing radiation, it needs some kind of hardware safety interlock.

[ClassyJacket]

They had one on previous models, but they removed it to save money.

[jschwartzi]

That's actually 1.

[bumby]

I think their point was slightly more detailed in that they were advocating non-software interlovks