[alkonaut]

I think the ambition will be adjusted and it will happen. The realization that will come is that solving the last percent (or five, or ten) of driving is so expensive that it's just not pursued. So we'll have these 99% autonomous cars that will be confused and handed off to a human driver, inside the car or remote. This will give most of the benefit at a tiny fraction of the cost. The race won't be to reach fully autonomous driving, but to reach 99.9% when the competition is 99.8%, because that is half the workforce needed to drive all those confused cars.

If we take "self driving cars" to mean "cars handling every siytuation", I completely agree that reaching 100% without either restricting the area or modifying the driving environment won't be possible within several decades. But my guess is we never get there because no one will make that investment for such little gain. Completely autonomous driving (handling the things that happen only once in a drivers lifetime) will require so much of human intelligence that if you have that kind of AI there are probably better things to do with it than drive cars around.

[emodendroket]

Cars that drive themselves 99% of the time but 1% of the time give up and hand off to human drivers are going to kill the drivers or somebody else about 1% of the time.

[dragonwriter]

That's a faulty assumption; if the 1% isn't “give up because otherwise we are certain to kill people” but instead “give up because this is a circumstance we aren't confident that the system will handle better than a human”, for instance.

[lx3459683]

It's a very reasonable assumption. When self driving cars give up, they give you a second or less to assess the situation around you and avert disaster. This is now well understood to be less safe than just 100% manual driving since your attention is then on the road at all times instead of inevitably wandering when the car is doing just fine 99% of the time. Don't forget that Wayomo killed off their SAE-2 test program after finding drivers were falling asleep at the wheel of these 99.0% autonomous cars.

And to build a self driving system that can give you a reasonable time to actually assess the situation and respond to it? That's the same system that's 99.9999% reliable.

This is the crux of why self driving cars will simply not work in the foreseeable future unless sequestered to their own tightly controlled road networks.

[alkonaut]

> When self driving cars give up, they give you a second or less to assess the situation around you and avert disaster.

I don't think the problem will be any situation with short reaction times at all. Nothing at freeway speeds, or situations like the Uber accident. That I think is where autonomous cars will shine, because sensors never get tired and reaction times are great.

The weird things that will happen which I count to the "not going to solve any time soon" category will be when the car comes to a completely snowed over roadworks, in the middle of the night, with the diversion signs completely hidden in snow. Construction workers barely visible in the snowstorm. Are those guys roadworkers or pedestrians? Are they working? Can I pass here? Will I get oncoming traffic because they narrowed it to one lane?

When things this weird happens at highway speeds or anywhere else where reaction is important - humans probably fail too. And at that point it's not really a question of technology but one of trust. Can we allow autonomous car to kill tons of people every year, with the sole excuse that humans would have killed all those people too, and then some? I'm not convinced of that either - I'm only arguing that from a technological standpoint, it should be possible to reach the 99% cars within a rather short timeframe. Those cars may be left on the scrapheap of history because of legal or ethical reasons, however.

[tigershark]

The scenario that you described seems to me less intractable than the “woman in an electric wheelchair chasing a duck with a broom” that Waymo handled correctly.

[emodendroket]

One problem with this theory is that the Uber car sensors misidentified the woman who was killed and they were so erratic that they felt like turning them off made sense.

[TangoTrotFox]

You're not considering why the AI turns it over the human. It's rarely about some crazy imminent emergency where if the human doesn't respond instantly and with super-human reflexes, it's all over.

In reality it's mostly just the AI expected one thing, and observed another - so something's not working right and it seeks a disengagement. California requires companies to quantify disengagements and most go a step further and specify the reason for the disengagement. I think the reason for this is precisely because of your intuition -- thinking that disengagement means imminent danger. Even for companies with relatively large numbers of disengagements, there were generally 0 that involved any danger whatsoever.

[grogenaut]

The problem is, as air flight has found, you're removing training hours from the humans so theyll be more and more dangerous also. And how do you safely hand off to a driver who is 99% of the time not doing anything. That's the thing humans are worst at, stayomg attent to long boring periods

[majewsky]

The safety driver will be sitting in a call-center-like office and taking remote control when the car requests it.

[emodendroket]

That sounds like a terrible job with a lot of potential for things to go wrong.

[icebraining]

Why? Sounds similar to the maritime pilots, whose job is to maneuver all kinds of ships just for small stretches (usually near a specific port).

These would be (remote) car pilots, probably specialized in specific areas.

[emodendroket]

It is a bit of an exaggeration, but the fact remains that handing off to a bored driver who likely isn't paying any attention and expecting them to take over a car at speed is not reasonable or safe.

[tialaramex]

You're imagining something akin to the autoland disconnect scenario for a jet liner, and you're correct that, in the absence of expert operators who are prepared to take over immediately this is often fatal. Which is why the jet liner has not one but two people specifically standing by to take over and try to land safely without the autoland. [ This is still scary because usually the autoland is trying to put the plane down in very poor visibility, and it is disconnecting because you're below decision height but something so bad went wrong that it's no longer able to land the plane - a human pilot may not be able to make whatever that bad thing was survivable either ]

The auto-land disconnect scenario isn't applicable to cars. It happens because jet liners are _flying_ and suddenly ceasing to fly in a jet liner is both very bad and perhaps unavoidable in the absence of enough information to operate the plane within parameters.

In contrast when a car becomes uncertain about what to do it's not flying so _stopping_ is almost always a good choice. It's not ideal, it may block traffic and be a nuisance, it might even cause a small accident of some sort - but it's very likely to end with everybody walking away, not with a burning wreck and dozens of dead.

[notatoad]

talking about cars that drive themselves only 99% of the time is a straw man. Waymo's disengagement rate in the year leading up to nov 2017 was once per 5,596 miles [1]. i'm not sure how to translate that to a percentage of time, but let's say a disengagement is a tenth of a mile - that means waymo's cars are self-driving 99.99998% of the time. they're the closest to market, but that was almost two years ago now, and in the 65 reported disengagements they didn't kill anybody.

[emodendroket]

Waymo measures their disengagements differently than the competition.

https://blog.piekniewski.info/2017/05/11/a-car-safety-myths-...

> Now it is important to note that the definition of a "disengagement event" may vary between companies. Most companies report every case in which a human grabs the wheel for any reason. Waymo (*) only reports the events, in which if not for the human intervention the car would actually cause a dangerous situation [read more here]. The way they do it, is for every physical disengagement they gather all the sensor data and next simulate multiple scenarios. If these scenarios lead to a dangerous situation, such event is being reported. According to Waymo in 2016 nine events would have lead to the car hitting an obstacle or another road user, approximately 1/10 of all disengagements they've reported (124). Hence there is such a gap between Waymo and the rest of the pack.

I also think it's hard to generalize from Waymo employees being attentive and general road users achieving the same thing.

[TangoTrotFox]

I don't think the goal should be seen as perfect driving, and that is likely impossible in any case. You can take the most extreme scenario that AI or human can possibly successfully overcome, and then make it just slightly more challenging to induce failure. You'll always be able to 'break' a driver's ability to correctly respond whether they're human or AI. So the goal, to start, should simply be to overcome humans on average. If a human will end up in a fatal crash every x 'average' miles, then an AI is a success if gets into fatal crashes at an average rate less than x.