[bumby]

There’s been some cases that highlight the danger of this as well as bad examples of how mitigations were poorly implemented.

I believe a case of a pedestrian being killed by a self driving car was related to the system reclassifying enough to initiate a deliberate delay timer. In a system traveling 60mph this delay in decision was enough to cause a mishap

[Animats]

Cite? The Uber killing was simply that the self-driving system was not allowed to do an emergency stop. All it could do was alert the driver. Presumably because they had a high false alarm rate.

The serious accidents involving self-driving cars have all been huge failures at the basic "don't run into things" task. Google/Waymo, which seems to be past that, has subtle problems, like "projected that bus couldn't fit through space to left of car in wide lane and so turned slightly into path of bus and was scraped." The standard Waymo accident is "advanced into intersection, detected cross traffic, stopped, was rear-ended by human driver at low speed".[1]

[1] https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...

[bumby]

“The self-driving car was fully autonomous at the time at the accident” [1]

It appears the mitigation for a high false positive (I.e., nuisance alarm) was to program in a delay (I.e., “action suppression”) which inadvertently introduced a worse hazard, the timetable from it is in the link below

[1]https://www.theregister.co.uk/2019/11/06/uber_self_driving_c...

https://dms.ntsb.gov/public/62500-62999/62978/629713.pdf