[Avernar]

> When the disk is plugged back in, it should be able to restart without any problems. But I think that's a different kind of resiliency than what you're referring to.

Yes and no. I was referring to restarting internally when the error condition went away but restarting the app and waiting for telemetry to return can be a valid solution.

Think of your torrent software. If you crank your firewall to block it while it's running it will not crash. If your disk fills up it won't crash. When the network comes back or more drive space if freed it will restart it's internal mechanisms. You wouldn't want it to restart in these conditions. If it runs out of memory however choosing to exit might be the best recovery mechanism.

I think a life critical medical application can at least strive for internal restart and do an external restart if all else failed. The article stated they had to reboot the machine to get it back. Now that's way worse.

> The situation in the article seems closer to hardware failure than a design oversight.

Hardware failure is almost always a permanent condition. This was a "my I/O stopped briefly and would have came back if my code could handle it".

[sillysaurus3]

During a surgery, the program doesn't have the luxury of showing a screen that says "No telemetry available." Such a program would be considered equally unreliable. Worse, it would lead to confusion: "Why is the telemetry unavailable? What does 'Error Code 2931' mean?"

A spectacular crash immediately led to pinpointing the problem: The antivirus.

If the program's sole purpose is to transform a massive amount of data in real time, it must have disk access by definition. It can't not have disk access. What would you suggest it do?

[Avernar]

Yes it does! Showing "no telemetry available" is exactly what it should do. Crashing = unreliable. Reporting an error condition = reliable.

Immediately? Took them 5 minutes to reboot the computer. The scan of the folder would take seconds let alone minutes. Pinpointing the problem is secondary. Not killing the patient is primary.

> If the program's sole purpose is to transform a massive amount of data in real time, it must have disk access by definition. It can't not have disk access.

And that is the mind set the programmers of the software had. You have to take care of error conditions. The processing can't have no disk access but no disk access can occur temporarily or permanently. What can you do? Pause the processing part of your program. Or make the processing part treat "no data" as valid input and display something else.

Imagine taking that viewpoint with an ECG machine: This machine displays a heart rate waveform. So it must have a heart rate input. If there is no heart rate we'll just crash requiring a 5 minute reboot.

Hell no! Draw a straight line and set off a buzzer!

[Bluestrike2]

I agree with you, but the flat line might not be the best example because that has a very specific meaning (asystole) that doctors will take certain actions based on without necessarily trying to verify it manually when time is already critical. You should never be able to confuse an error message for anything else.

[Avernar]

> You should never be able to confuse an error message for anything else.

Exactly. Which is why "Can't read file sensors.dat" is way better than just crashing. Crashing is one of the worst error messages you can get because you don't know what happened.

[nickpsecurity]

" Avernar 18 hours ago

Yes it does! Showing "no telemetry available" is exactly what it should do. Crashing = unreliable. Reporting an error condition = reliable.

Immediately? Took them 5 minutes to reboot the computer. The scan of the folder would take seconds let alone minutes. Pinpointing the problem is secondary. Not killing the patient is primary."

Well-put. Far back as Burroughs B5000, the best way to handle erroneous software or I/O was to freeze it, notify the administrator/user of the problem, and give them sensible options for how to proceed. They might restart the I/O, restart the app, modify erroneous data to proceed (rare here), and so on. Crash and reboot is a Windows 95/NT strategy where incompetence dominated. Today's Windows OS and tooling can do much better with little effort by developers.

[Bluestrike2]

But the spectacular crash didn't pinpoint the problem. That came afterwards, when the manufacturer was able to look into the crash.

In a situation like this, confusion is all but inevitable. As a developer, the goal should be to minimize that confusion to the greatest extent possible. A blank screen and crash introduces another step to the process as people wonder "what's going on?" instead of "shit, it threw an error." It's probably not a big deal, but with medical devices during surgery, that extra step could be hugely problematic.

[chris_wot]

I wonder what surgeons think about software engineers? Except for open heart surgery, they don't normally do their fixes by stopping and starting the thing they are repairing...