[lx3459683]

It's a very reasonable assumption. When self driving cars give up, they give you a second or less to assess the situation around you and avert disaster. This is now well understood to be less safe than just 100% manual driving since your attention is then on the road at all times instead of inevitably wandering when the car is doing just fine 99% of the time. Don't forget that Wayomo killed off their SAE-2 test program after finding drivers were falling asleep at the wheel of these 99.0% autonomous cars.

And to build a self driving system that can give you a reasonable time to actually assess the situation and respond to it? That's the same system that's 99.9999% reliable.

This is the crux of why self driving cars will simply not work in the foreseeable future unless sequestered to their own tightly controlled road networks.

[alkonaut]

> When self driving cars give up, they give you a second or less to assess the situation around you and avert disaster.

I don't think the problem will be any situation with short reaction times at all. Nothing at freeway speeds, or situations like the Uber accident. That I think is where autonomous cars will shine, because sensors never get tired and reaction times are great.

The weird things that will happen which I count to the "not going to solve any time soon" category will be when the car comes to a completely snowed over roadworks, in the middle of the night, with the diversion signs completely hidden in snow. Construction workers barely visible in the snowstorm. Are those guys roadworkers or pedestrians? Are they working? Can I pass here? Will I get oncoming traffic because they narrowed it to one lane?

When things this weird happens at highway speeds or anywhere else where reaction is important - humans probably fail too. And at that point it's not really a question of technology but one of trust. Can we allow autonomous car to kill tons of people every year, with the sole excuse that humans would have killed all those people too, and then some? I'm not convinced of that either - I'm only arguing that from a technological standpoint, it should be possible to reach the 99% cars within a rather short timeframe. Those cars may be left on the scrapheap of history because of legal or ethical reasons, however.

[tigershark]

The scenario that you described seems to me less intractable than the “woman in an electric wheelchair chasing a duck with a broom” that Waymo handled correctly.

[emodendroket]

One problem with this theory is that the Uber car sensors misidentified the woman who was killed and they were so erratic that they felt like turning them off made sense.

[TangoTrotFox]

You're not considering why the AI turns it over the human. It's rarely about some crazy imminent emergency where if the human doesn't respond instantly and with super-human reflexes, it's all over.

In reality it's mostly just the AI expected one thing, and observed another - so something's not working right and it seeks a disengagement. California requires companies to quantify disengagements and most go a step further and specify the reason for the disengagement. I think the reason for this is precisely because of your intuition -- thinking that disengagement means imminent danger. Even for companies with relatively large numbers of disengagements, there were generally 0 that involved any danger whatsoever.