Hacker News new | ask | show | jobs
by mbakke 957 days ago
Great article. Real life horror stories of life-critical software gore, with some good news at the end.

It should be illegal to sell software that someones life depends upon without giving the user the right to inspect and modify the code.

3 comments

I have an ICD (implanted cardioverter-defibrillator) to save my life if my heart stops.

I was also given a proprietary box that sits at home, reads data from it and sends it to my cardiologist over a cellular network, on demand. As part of periodic remote checkups I'm supposed to sit next to it, press the button, which causes it to read data and send any abnormal heart rhythms it detected (via cellular network), whether it treated it (via a shock, in which case I would have known anyway) or whether the abnormal rhythm resolved itself with no treatment (in which case it's worth it that they check out what it picked up). I have to do this about 2-4 times a year.

Every time I hit the button I'm charged $200. Even if there are ZERO events. 90%+ of the time there are zero events.

There is NO interface provided to me where I can read the data directly. There is no way for me to read the device on my own, see zero events, and inform my cardiologist that there are no events and that there is nothing new to diagnose.

I hate this medical system. The device is great for saving my life but I want access to read its data without being charged.

That's appalling and should be illegal.

I wish more programmers would refuse to contribute to this kind of exploitation.

I work in medical devices and it's extremely hard as a dev to figure out what's because of some regulation and what's just for profit.
If it was illegal he might be dead. If he refused, he could be dead. Is that a better world?
No, if it was illegal he'd have access to his data. I'm not saying medical equipment should be illegal.

And to be clear, I wasn't saying he should have refused treatment. I was saying I wish more programmers would refuse to help develop exploitative software like this.

It might not have even been the programmers of the device that chose to do this. It was very likely some manager somewhere who saw the dollar signs when they realized they could collect rent.
Programmers implemented it though. And they knew exactly what they were doing, too.
I don't think he had a choice.

If you had a good doctor that liked da Vinci robotic surgery, versus another one that did raven II would that factor more than the reputation of the doctor? Programmers who make life saving software are good in my opinion, even if the company they work for wants to make money.

I think we should strive for the best features, and also be grateful for "fascist trailblazers". Shockley was known to be an awful boss but our transistors started there and we are better off for it. Body warming methods were created by Nazi scientists experimenting unethically. These are the 2nd step, at least the profiteers show it's doable and the drive for profit made it in the first place.

I would argue that the discoveries would have happened anyway sooner or later even without unethical assholes. And for every example of a step of progress accelerated by them there is an example of a step of progress held back by them.

We do not need the monsters to make progress. Don't try to justify their inexcusable actions in some myopic utilitarian way.

Did you seriously just use the holocaust as an example of successful R&D?
This is nuts. Who charges you? Is it the company that makes these devices? What if you want a different “provider”?
Stanford Healthcare charges me for "general classification" just for a nurse to open up their computer and see that there are zero events.

Boston Scientific, the device maker, does not have an interface for patients, they only send data to hospitals directly.

I'm not currently willing to switch to a different ICD because Boston Scientific's ICD has successfully saved my life 3/3 times in out-of-hospital situations and 2/2 times during in-hospital testing where they induced ventricular vibrillation in controlled testing and I'd rather not risk trying something different. Insurance wouldn't pay for an extra surgery deemed unnecessary, anyway.

I could switch healthcare providers, but I'm not sure if the others in my area are better at cardiology.

I see you have your hands full, but perhaps a class action lawsuit should be in order.
> Stanford Healthcare charges me for "general classification" just for a nurse to open up their computer and see that there are zero events.

Okay so having access to the data wouldn't change a thing, surely you'd be charged even more if you wanted to talk directly to the cardiologist to do a report yourself, as you said?

> inform my cardiologist that there are no events and that there is nothing new to diagnose

This is giving me feelings similar to that movie repo men where you had to rent life saving organs and they could come repossess them at any time.
That is genuinely insane
Quality of life critical software should be ensured by FDA certification. Homebrew modifications of that software, even in the name of “freedom”, risks the patient’s life and health and should be illegal if uncertified.
In EU (and probably elsewhere), there are strict rules for the stability of power wheelchair. One such rule is "On a incline of x% (x chosen by the manufacturer), pushing for max speed from stop should not lift the front wheels"

To achieve that, the max acceleration must be quite low (software controlled), and the whole experience is sluggish, like trying to steer a car by pulling on rubber bands attached to the wheel.

From the moment I found a way to overcome this, I never went back. I know that I can hurt myself if I do something stupid, but I prefer this hypothetical risk instead of cursing 100 times a day because I cannot move how I want. It has been 10 years and I never got hurt.

I understand that such "high" risk device cannot be sold, but forbidding someone to change this is like inflicting a second handicap on him.

I suppose we all have, or should have, the right to try stupid things. Sometimes experience and competence are more important than 100% safety. Your comment made me realize how limiting it would be to be physically incapable of taking even the smallest risk.
That is a very poor regulation. Why enforce wheel lift? What matters is that the chair doesn't tip over - that the center of gravity remains in the center of the four wheels.

  > Homebrew modifications of that software, even in the name of “freedom”, risks the patient’s life and health and should be illegal if uncertified.
The official modifications of that software — in the name of "profit" — are currently risking the patient’s life and health, and therefore should also be illegal by your logic.

Surely you must also support effective (ie harsh/deterrent) prosecution and punishment for these crimes as well, correct?

>>>should be illegal if uncertified.

I think this is the key part of the comment - yes, uncertified changes by anyone could feasibly be illegal. The FDA or similar should probably do code reviews.

Looking at corner cases for this:

What if you fix a bug in your own pacemaker? Would it be ok to:

a) Fine you?

b) Jail you?

c) Force you to revert the change? (plausibly leading to death in an extreme case)

[edit: I do agree that there's a chance that making a 'fix' to your own pacemaker might also make it worse. In which case, who do we trust more? The person on the ground with a stake in the matter (however misinformed), or $government_official with no stake in the matter (however well informed).

I think it's tricky! ]

I don't think that scenario is particularly tricky. If you modify someone else's pacemaker, it's a tricky question, even with their consent. If you modify your own, absolutely nothing should stand in your way beyond a nice big notice saying "danger of death,on your head be it". That is, you should have the same freedom to screw with your own personal medical devices that you have to climb out of your own fourth floor window.

People have a right, albeit not enshrined in law, to do stupid things that might kill them - at least as long as they don't then ask someone else to save them.

This is a huge straw man/whataboutism that contributes nothing to the discussion.

Yes, bad software modifications are bad and should be punished wherever they arise.

Homebrew modifications make it way easier for bad stuff to happen, and make it harder to punish.

> bad software modifications are bad and should be punished wherever they arise.

That almost never happens. Software sux.

  >  This is a huge straw man/whataboutism that contributes nothing to the discussion.
It's a countervailing concern, not a strawman.

  > bad software modifications... should be punished wherever they arise
Corporations are currently unpunished (per TFA) when they alter software in a way that risks patient safety, and they have already caused documented harm to patients. This is a shocking failure of federal oversight, but the captured FDA will (by design) never fix it. Oops.

In light of the real harm caused by this neverending policy failure, the Library of Congress is morally and ethically obligated to permit fair use exemption. Individuals and homebrew communities must be unshackled to protect patients from the real (not hypothetical!), documented, and widespread harm caused by corporate-sponsored attacks on US medical infrastructure.

No, that's not an exaggeration.

Given the current anti-patient landscape, the protections of open source far outweigh any risk.

I think this might be a cultural thing.

In some (western) countries, your body is your personal private property, and you have the freedom and ultimate authority over how to use and abuse it, or anything on or in it. (you are still advised to treat your most precious property wisely, obviously)

In other (western) countries/subcommunities people feel that obligations to your community are stronger.

People from these different cultures can get into some pretty hefty discussions when it comes to things like abortion, drugs, euthansia, or -here- implants.

So like suicide, drugs and other and other cases where we are denied dominion over ourselves for our own good? IE. Your life and body are not yours, they belong to society and you only get limited access.
Society doesn't have to give you the rope to hang yourself.
I disagree or rather yes, it does have the responsibility to provide you a rope. It is up to you whether you hang yourself or not.
I disagree, I think if you walk into a pharmacy and ask for something dangerous without a prescription they shouldn't be obligated to give it to you. It's the same with medical equipment that keeps you alive.

If you want to risk your life you can do it but no one should be compelled to help you.

No one should be compelled. I mean it more in a negative manner, that it has an obligation to not stop people from helping. If someone wanted to offer a nitrogen tank, valve, tube and an easily head fitting bag for sale to people who want to commit suicide in a painless and ensured manner they be able to sell that (and people would). But in fact you cannot, and that is wrong.
You are taking the position that an individual "owns" themselves

That is not obviously true.

I feel I belong to my family and my community.

Your position is not universal, and in fact strongly opposed by many. I believe that I have the absolute right to edit or terminate my own existence, either on purpose or accidentally. To the extent that anyone can own a person, people own themselves exclusively.
Surely the patient should have the right to risk their own life?
To distribute? Sure. To make changes to your out of support cyber-eyeball? Nah.
Serious question, what does the FDA know about software quality?
Surely not less than the average consumer.

And surely they could hire experts to do the job.

1. Compared to the average person in the FDA's population of people who are in charge of evaluating the medical devices, the average person in the population of people who would make fixes and helpful modifications might have more expertise in determining the quality of the device's normal software.

2. It's not as if the people who depend on the medical devices have to take the word of the community of people who will mod the devices over the word of the FDA.

Safetism is a great curse on the world. I cannot disagree with you more.
So you would prefer it not be developed?
The software is clearly not the primary product. While there might need to be a carve out or a specific licensing scheme developed to protect them from liability in the case of modified software, I doubt these companies would experience serious financial setbacks if they made their software free and open.

And don't tell me that SaaS is an integral part of the business model for medical device companies. There's no world in which they can't figure out how to turn a profit without charging a monthly fee to use your tens of thousands of dollars eyeball.

> The software is clearly not the primary product.

Sure, in this case. But that means that the rule we're considering actually needs a big asterisk next to it, something like "when the software in question isn't the primary product." That sounds like a thorny regulatory question, and any answer to that question other than "I know it when I see it" probably has big loopholes. This might be unnecessary nitpicking on my part if we're just shooting the breeze about companies we don't like, but if we're actually interested in writing laws, this is a common failure mode. Maybe _the_ common failure mode.

On the other hand, "so you would prefer it not be developed" is a less-than-entirely-charitable way of making this point. Of course @mbakke would _not_ prefer that, and it might avoid an unnecessary round of back-and-forth to make a reasonable guess about what they would prefer and work from there :)

This is being downvoted yet there’s a reason why this types of treatments always starts being developed to serve the US market initially