[notahacker]

Yep. Medical professionals have been aware of this dilemma for millennia: many people die from an ailment if no treatment is attempted, but bad approaches to treatment can kill people that would have survived otherwise. And setting 'better average accident rates' as the threshold for self driving vehicle software developers to be immune from the consequence of their errors is like setting 'better than witch doctors' as the threshold for making doctors immune from claims of malpractice.

Move fast, break different things, is not the answer.

[JoeAltmaier]

What if its very much better average accident rates? This isn't black-and-white.

[notahacker]

No, it certainly isn't black and white. Indeed 'much better' is hard to even define when human drivers cover an enormous amount of miles per accident, miles driven are heterogenous in terms of risk, there isn't even necessarily a universally accepted classification of accident severity or whether drivers should be excluded from the sample as being 'at fault' to an unacceptable degree. Plus the AV software isn't staying the same forever: every release introduces new potential edge case bugs, and any new edge case bug which produces a fatality every hundred million miles makes that software release more lethal than human drivers, even if it's better at not denting cars whilst parking and always observes speed limits in between. I don't think every new release is getting a enough billion miles of driving with safety drivers to reassure there's no statistically significant risk of new edge case bugs though.

And in context, we still punish surgeons for causing fatalities through gross negligence even though overall they are many orders of magnitude better at performing surgery than the average human.

[JoeAltmaier]

Sophistry. 'Much better' can be very clear, in terms of death or injury, or property damage, or insurance claims, or half a dozen reasonable measures.

Sure it takes miles to determine what's better. Once automated driving is happening in millions (instead of hundreds) of cars on the road, it will take only days to measure.

[notahacker]

I mean, the 'half a dozen reasonable measures' is a problem, not a solution, when they're not all saying the same thing. And sure, it only takes days before we know the latest version of the software actually isn't safer than the average human. And a lot of unnecessary deaths, and the likelihood the fix will cause other unnecessary deaths instead [maybe more, maybe less]. It's frankly sociopathic to dismiss the possibility this might be a problem as sophistry.

[JoeAltmaier]

Straw man? There are many phases to testing a new piece of software, short of deploying everything to the field indiscriminately.

Some of us believe (perhaps wrong but there it is) that the human error rate will be trivially easy to improve upon. That's not sociopathic. It would be unhelpful to dismiss this innovation (self-driving cars) because of FUD.

[notahacker]

Some of us believe, based on the evidence that the human fatal error rate is as low as 3 per billion miles driven in many countries, and some people actually are better than average drivers. Might be trivially easy to improve upon human ability to not to dent cars whilst parking or observe speed limits, but you're going to struggle to argue that improving on the fatal error rate is trivially easy for AI, or that the insurance cost of the dents matters more than the lives anyway.

People who actually want initiatives to succeed are going to have to do better than sneering dismissal in response to anybody people pointing out obvious facts that complex software seldom runs for a billion hours without bugs and successfully overfitting to simulation data in a testing process doesn't mean a new iteration of software will handle novelty it hasn't been designed to solve less fatally than humans over the billions of real world miles we need to be sure.