[KZeillmann]

Regarding the "we don't know how this stuff works" point, doesn't the FDA approve a ton of drugs where we don't know the exact mechanism of how it works? Do we need to know exactly and precisely how something works to know _that_ it works?

[a1369209993]

The overwhelming majority of medical treatments don't have inteligent humans actively trying to maliciously sabatoge them. Many drugs can be made horrendously lethal or otherwise dangerous with little effort (often just by significantly increasing the dosage) but we don't need to care very much because it's not possible to silently untracably apply that effort from arbitrarily far away, and there usually isn't anything to gain from doing so even if it were possible.

[ballenf]

At the end of Mickens' talk he suggests putting black box devices behind smart firewalls and routers. I'd say that's the equivalent role of doctors.

That is, the trifecta of bad decisions is black box functionality connected to an internet of hate (or unfiltered/tested input data) and given levers of power in society. Take away any one of those 3 and you're probably ok.

[emodendroket]

Is there any kind of sabotage other than malicious sabotage?

[dredmorbius]

Ignorant, unintentional, self, incidental.

Stupidity is probably the biggest threat. Dietrich Bonhoeffer:

https://religiousgrounds.wordpress.com/2016/05/11/bonhoeffer...

[mehrdadn]

I think the parent meant that "sabotage" is "deliberate destruction" by definition.

[sitharus]

No, but drugs spend literal years in testing through various models and animal subjects before even being considered for years more of human trials. Then the benefits are weighed up against the known side-effects and if they stack up the drug is approved. Even after that sometimes the analysis is wrong and the drug turns out to be ineffective or even harmful.

Technology companies churn out things with barely a few weeks of testing at times and no oversight.

[emodendroket]

"Barely a few weeks"? Ha. Many places it's days, hours, or a few minutes.

[nraynaud]

Yes they do, but they circle the risks with statistics, when the life expectancy and quality of life goes up vs the current best treatment, the unknowns side effects can’t be that bad.

[TallGuyShort]

In some cases, no. But let's say there was a drug for 99% of people, but totally messed up 1% of people. Well then it's probably really worth understanding how to predict or avoid that. Because the same thing happens with image classifiers. Google had an embarrassing incident where their classifier was very impressive, until it got it wrong and it was really embarrassing. Do you just accept that sometimes image classifiers act racist? Or would it be nice to have a tool that can highlight what parts of the picture contributed most to the classification, so you can identify pictures that would have prevented that error in the training set?