[dragonwriter]

That's a faulty assumption; if the 1% isn't “give up because otherwise we are certain to kill people” but instead “give up because this is a circumstance we aren't confident that the system will handle better than a human”, for instance.

[lx3459683]

It's a very reasonable assumption. When self driving cars give up, they give you a second or less to assess the situation around you and avert disaster. This is now well understood to be less safe than just 100% manual driving since your attention is then on the road at all times instead of inevitably wandering when the car is doing just fine 99% of the time. Don't forget that Wayomo killed off their SAE-2 test program after finding drivers were falling asleep at the wheel of these 99.0% autonomous cars.

And to build a self driving system that can give you a reasonable time to actually assess the situation and respond to it? That's the same system that's 99.9999% reliable.

This is the crux of why self driving cars will simply not work in the foreseeable future unless sequestered to their own tightly controlled road networks.

[alkonaut]

> When self driving cars give up, they give you a second or less to assess the situation around you and avert disaster.

I don't think the problem will be any situation with short reaction times at all. Nothing at freeway speeds, or situations like the Uber accident. That I think is where autonomous cars will shine, because sensors never get tired and reaction times are great.

The weird things that will happen which I count to the "not going to solve any time soon" category will be when the car comes to a completely snowed over roadworks, in the middle of the night, with the diversion signs completely hidden in snow. Construction workers barely visible in the snowstorm. Are those guys roadworkers or pedestrians? Are they working? Can I pass here? Will I get oncoming traffic because they narrowed it to one lane?

When things this weird happens at highway speeds or anywhere else where reaction is important - humans probably fail too. And at that point it's not really a question of technology but one of trust. Can we allow autonomous car to kill tons of people every year, with the sole excuse that humans would have killed all those people too, and then some? I'm not convinced of that either - I'm only arguing that from a technological standpoint, it should be possible to reach the 99% cars within a rather short timeframe. Those cars may be left on the scrapheap of history because of legal or ethical reasons, however.

[tigershark]

The scenario that you described seems to me less intractable than the “woman in an electric wheelchair chasing a duck with a broom” that Waymo handled correctly.

[emodendroket]

One problem with this theory is that the Uber car sensors misidentified the woman who was killed and they were so erratic that they felt like turning them off made sense.

[TangoTrotFox]

You're not considering why the AI turns it over the human. It's rarely about some crazy imminent emergency where if the human doesn't respond instantly and with super-human reflexes, it's all over.

In reality it's mostly just the AI expected one thing, and observed another - so something's not working right and it seeks a disengagement. California requires companies to quantify disengagements and most go a step further and specify the reason for the disengagement. I think the reason for this is precisely because of your intuition -- thinking that disengagement means imminent danger. Even for companies with relatively large numbers of disengagements, there were generally 0 that involved any danger whatsoever.

[grogenaut]

The problem is, as air flight has found, you're removing training hours from the humans so theyll be more and more dangerous also. And how do you safely hand off to a driver who is 99% of the time not doing anything. That's the thing humans are worst at, stayomg attent to long boring periods

[majewsky]

The safety driver will be sitting in a call-center-like office and taking remote control when the car requests it.

[emodendroket]

That sounds like a terrible job with a lot of potential for things to go wrong.

[icebraining]

Why? Sounds similar to the maritime pilots, whose job is to maneuver all kinds of ships just for small stretches (usually near a specific port).

These would be (remote) car pilots, probably specialized in specific areas.

[emodendroket]

High-pressure and probably a fair amount of death

[icebraining]

Well, I'm assuming we're in L4, so they don't have to pick up the controls of a car going high-speed or such. I can't imagine any company would sign up to have its employees regularly have the clients' lives in danger, it would be unsustainable legally and PR-wise.

[emodendroket]

It is a bit of an exaggeration, but the fact remains that handing off to a bored driver who likely isn't paying any attention and expecting them to take over a car at speed is not reasonable or safe.