[lostlogin]

>>As a result of the Therac-25 accidents, the FDA now requires documentation on software for new medical and other products: a paper trail, in other words, that can be examined by an independent body and retraced for flaws.<<

Anyone have any idea if this can be looked at by the end user? I'm not a radiation technologist of the flavour mentioned in the article, I'm on the diagnostic side. I use an MR scanner with numerous software bugs that I have reported but which remain. Similarly, the scanner can be made to display data which it says it is going to use in the next scan, but which it isn't. I suspected a bug and found the way to reproduce it. My last email listed 24 similar bugs (I've found more since) but other than a "thanks, we will forward this on" there has been no reply or comment. It is hard to imagine when this could be a safety issue, but it is a waste of valuable time, it is a waste of money and it's frustrating when I have gone to the trouble of working out the exact way of creating the issues. If anyone is interested, the interface is so god awful that instead of having an on off button or switch interface, the scanner gets the user to type 1 or 0 for on and off into a text field. Some fields take other values like 1, 2 and 3. Some take decimal values like 0 to 1 in 0.1 increments. There is no pattern to what the user is expected to type. Yuck. This data is not properly sanitized either, and you can make the scanner say its "doing" something it's not. Type in 1.999, and error message appears, the field corrects to 2.0 but the scanner does the thing that a setting of 1 would produce. These sorts of bugs occur all over the place.

Edit: The "thanks" email is the most positive I've ever got, my previous reports were me with statements like "we have some very experienced users who haven't had this issue" when there were clear safety problems with earlier scanner implementations (The scanner was producing axial slices at a location different to where I asked for them to be, on a spine patient due in theatre - good luck operating on the correct vertebral level). Its FDA approved and its on the latest software release. I have undergone manufacturer training and have had additional training half a dozen times at my request and at the manufacturers request after my bug reports were met with "you're doing it wrong". I'm not, the software is buggy and I have some excellent and amazing screen shots and camera phone video of the bugs in action.

[kohanz]

I don't know if this makes you feel any better, but if the device manufacturer are indeed playing by FDA rules, the e-mails that you have sent should have triggered serious investigations into these bugs. That doesn't mean that they would be fixed, but they would be triaged to assess how and when they happen, and what risk they pose to patients.

This is known as the Corrective Action, Preventative Action (CAPA) process [0]. Note that the investigation into your complaint is an absolute requirement. Not just e-mails, but even phone conversations, or comments made in passing verbally - if any of them constitute a comment (positive or negative) on the device, this needs to be logged and, if the comment warrants it, an investigation must take place.

So either your comments have been or are in the process of being investigated, or the device manufacturer is not following the FDA rules.

[0] http://en.wikipedia.org/wiki/Corrective_and_preventive_actio...

[tokenadult]

My son the hacker used to work in the medical device industry as a summer employee while he was a student. The code he wrote for a medical device user interface was to be submitted for a line-by-line code review by the FDA. He estimated that the product would actually come to market more than three years after the summer he worked on it. And maybe that is what you are encountering--the person at the company who built in the bugs you have discovered has moved on, and doesn't work at the company anymore, and the other employees there are trying to figure out how to debug that old code and fix the problem. (Similarly, my son groused about the code in the device he was working on, which was acquired by his company from another company that had originally developed the device.) Always comment your code. You never know how long after you wrote it someone else will have to fix it, especially if the code is embedded in a medical device.

[lostlogin]

Thanks - this has been in the back of my mind and is a reason I'm trying to be patient. A 2 line message saying what was happening would remove my frustration. Usually I get a corporate speak reply with a suggestion it is my fault though. What does the FDA code review do? If it isn't catching bugs that take the scanner offline for hours at a time, what is the point?

[kohanz]

I've worked on several FDA-regulated products and have never had the FDA review my code. I would guess this only happens in extenuating circumstances. The FDA does not have the resources to do this for most products out there.

We are required, however, to review our own code and maintain records of those reviews.

[vardump]

Exactly. FDA doesn't review code!

If there are complaints, FDA does sometimes review is the mountain of device related documentation. Design, assembly, maintenance, end user manuals, etc. Checking the paper trail. Is the paperwork done correctly, signed by a competent employee and reviewed by appropriate persons. There also needs to be watertight trail of employee training. Failure to have that does not end well!

Traceability (both physical and code) is another thing you better get right as a medical company. You need to know where, when, etc. each major component of the device came to be.

Medical companies literally generate so much paperwork, that separate storage facilities are needed for it. While you'd obviously have it in digital format for yourself, all of it is also printed out and signed.

[laurenstill]

Compliance officer for a med device company, can confirm. Even vendor audits don't look at code, just SOPs and spreadsheets documenting that you have the processes in place to log the shit out of everything.

[snom380]

There's a difference between bugs that cause downtime and bugs that endanger the life of the patient, and I think the FDA is primarily concerned with the latter. I would think a bug that caused the wrong image to be captured and could cause doctors to make the wrong decisions would be taken very seriously.

[cnvogel]

For future bugs:

Don't send random emails to people in the company, most people can't be held accountable for mishandling bug-reports (or ignoring them).

Look up the contact to send non-conformance reports in the user-manual of the device, there has to be a contact address, maybe even a (paper) form to fill out. Send it by paper-mail to the address (most likely the QA department). Request a classification of the issue (urgent, user-error, critical, ...) and an ID under which this issue is tracked. Set a deadline for replies to your inquiries.

If you really want to be serious, the FDA takes reports on defective medical products, here's a webpage on this process:

http://www.fda.gov/Safety/MedWatch/HowToReport/DownloadForms...

[dpe82]

GE scanner, by chance? I've heard their software is pretty terrible compared to their competitors.

[lostlogin]

I actually laughed. Yes. Latest release, flagship scanner 3T 750W. We have a few of the extra bells and whistles but not all. I'm not entirely sure which we have, as trying to run something you haven't paid for used to crash the scanner, so I'm hesitant to try some things. This bug I reported.

[gwillen]

Perhaps it's time to forward your materials to the FDA? (Or whatever the appropriate regulator is for your device.)

[ZoFreX]

Time to get in touch with the press. 25 bugs in something health-critical? They'll have a field day.

[lostlogin]

My hope (which is steadily fading) was to make contact with someone in the development team to suggest a few improvements and show a few of the more obscure bugs that I haven't reported. I'd love to be able to positively influence the development. There has been limited "end user" input as far as I can tell and some small changes would make the platform so much more powerful. The upside to the lack of communication is that I've begun leaning to code and use my code on my phone to bypass the scanner's crap - without the poor scanner interface I wouldn't have done this. I should note that my basic code is truly awful to look at, although it works reliably and does a better job of its small task than the multimillion dollar console I use.

[mgkimsal]

They're waiting for patents on all the stuff you suggested before contacting you back to say that you've violated their intellectual property. Don't worry. You'll hear from them soon enough...

[vardump]

There are so many layers between you and the developer... Without going to specifics, probably about 5. There's pretty much no chance you get to interact with the developers directly.

[vardump]

Like others have told, any complaint you make must be processed by the company. If it endangers human safety, failure to do so is against FDA's rules and can have dire and expensive consequences to the company in question.

Fixing bugs in a medical device is not slow because it'd take a long time to fix the bug itself. It's slow because there's a lot of paperwork (Device History Files, specifications, tons of reviews, etc.) and system testing that follows. That's why you'd only correct issues that do not endanger patient safety once there are enough or when they can be combined with new features in firmware.

There's also always a risk that those fixed issues cause patient endangering bugs.

It can take almost one year from implementing the fix until it is running on any significant number of medical (imaging) devices in question.

FDA's rules are there for a reason. Although... someone should tell FDA git exists. It's silly to revision things manually in Word documents.