Hacker News new | ask | show | jobs
by sathackr 3700 days ago
I'm with you...I can see exactly how this can happen.

Unfortunately the only thing that can solve the apathetic board and executive management problem(who only see dollar signs) is the actuality, or realistic possibility, of significant financial loss, or loss of their personal freedom(prison) due to the negligence of the system. And a $10 Mil fine for a fault in something that you make $100 Mil off of is not significant. That's $90 Mil profit in their eyes. And they probably get to write it off.

Even more unfortunate, is that, in the situation that this happens, the "engineers responsible" will be fired, and the executives will resign with a nice golden parachute, and go on to do the same thing somewhere else.

But then you have the company that does do it right, spend the time, and the money to make a truly redundant, fault-tolerant system. But, they come in at a price point 20% higher than their competitor, who doesn't. Which company survives and which doesn't?

Sad, but, unfortunately the way it is. I don't know a practical solution either.

3 comments

I've thought about this a lot. I've had private conversations with the CEO which lead me to believe that their apathy is a, if not the, primary driver in this situation, at least within the company. Ultimately, they are the single individual who can force these changes in the departments. As things stand today, as far as I can tell, the CEO and the rest of the executive team got theirs and that's that. Anything extra is just that, extra.

We've been close to undergoing "major" scrutiny (as it was sold to me, it was A Big Deal) from the FDA before. I, personally, just a lowly and underpaid engineer, have saved executive staff from having to sign their names on that noose. I had a manager once who seemed to want to push it that far, to stand idle-by while the walls fell down around us. I, unknowingly at the time, prevented it from happening because I was trying to help our customers. I don't regret that decision, actual patients shouldn't have to suffer because of a management teams ineptitude. I do think about it often, though. I understand this is nebulous, and I'm sorry for that. This is a reality, though.

I guess that's the thing that really gets me, the FDA. We sell FDA approved devices. Where the fuck is the FDA? We send them paperwork and they are happy. I can only form the opinion that they, the FDA, are ill prepared to handle this situation; The actual situation, the "the medical devices industry is a fucking train wreck waiting to happen" situation, and especially so they are ill prepared to handle it at scale. Audits are cursory and almost as a rule non-technical. I suppose it'll take a Toyota-level incident to bring change about.

Along the same lines as your 'where the fuck is the FDA' comment -- I've worked in Financial and Healthcare systems on and off for about the last 10 years.

I have seen SSAE16 audited companies that haven't patched anything in years. FDIC examined institutions with ATM machines still running OS/2 Warp(actually probably more secure than the ones running XP, with no updates installed. Ever.)

I once found the management interface of a SAN with a public IP address directly on the device, no firewall rules of any sort, and the device still had the default username/password. It hadn't been patched or rebooted in over 2 years.

More shocking is that a review of the logs didn't show any successful unauthorized logins. Of course, they could have cleaned up after themselves, but further investigation was outside the scope of my engagement(They didn't want to know. They were happy to present that, despite the oversight, there was no indication that PHI had been accessed by unauthorized people. Their conclusion, not mine.)

I can't help responding again. If you have tangible evidence of neglect or regulatory non-compliance, or even risks that are known about but not being dealt with by management - have you considered compiling this material and and reporting it to the FDA?

But as I've said before - I really hope you have written down your concerns to someone in management. If it gets to the point where negligence takes out the company, there's going to be an attempt to make someone a scapegoat. Depending on your role in the company you don't want to be held personally liable for the incompetence and ruthlessness of management...

>Where the fuck is the FDA? We send them paperwork and they are happy.

When regulation becomes more about permission than proficiency, you'll get corruption instead of competence.

> Unfortunately the only thing that can solve the apathetic board and executive management problem(who only see dollar signs) is the actuality, or realistic possibility, of significant financial loss, or loss of their personal freedom(prison) due to the negligence of the system.

Or developers refuse to build software without safety built in.

If they can't hire anyone to build their unsafe systems, they'll have to start building safe software.

Let the market work for you.

That sounds nice...but then you will be replaced by a developer that will toe the company line. You're making 'unreasonable' demands and holding up progress. 'We can fix that with version 2.0'

If every developer on the planet suddenly had a pang of consciousness, then something like this would work.

Fortunately I have never found myself in such a position, but I have seen it many many times.

That's why we should probably require engineering certifications for working on safety-critical software. Working on such software should require demonstrating a certain level of knowledge and upholding a code of ethics.

I generally oppose certification for engineers, but solving collective action dilemmas like this and saving lives in the process is exactly where it would help.

How do you ensure someone upholds a code of ethics? Licensing is not the answer. I'm sure there are many PEs that find themselves in similar situations.

I know examples of people in licensed fields who have sworn to uphold a code of ethics, but have been caught up in very similar situations.

I can't find it now but I just saw a video recently of a rail bridge with a crumbling foundation that had just been signed off on by a PE and declared safe by the railroad.

> get to write it off

A fine being tax deductible does not mean zero cost to the company, it means the profit is reduced before taxes are computed, i.e. the actual cost is reduced by the marginal tax rate. A tax credit means zero cost.