[thumbsup-_-]

It works most of the time but the issue is that "most of the time" is not good enough for these systems. Even if the failure rate is <1% that may end up being lots of accidents and deaths at scale.

People often make arguments that "oh it will still be less accidents than human drivers", which is true, but, the problem is that human accuracy is a very poor benchmark for autonomous systems. Autonomous systems need to be held to a higher bar, and it's better if that accountability and expectation is held from the beginning.

[andrewmutz]

> Autonomous systems need to be held to a higher bar

Why? Won't this lead to a lot of needless deaths at the hands of human drivers while we wait for driverless cars to improve? Why not roll them out once they are safer than human drivers?

[zug_zug]

Well think about it this way:

I'm a driver, but I'm a safer than average driver. So why would I want a system that's better than "average" where average includes drunk people, speeders, new/bad drivers, driving in ice, etc.

I don't think it should be illegal to use a system that's actually better than average (which is 1 accident every 18 years), but many drivers might not want to.

Also when some company's financial success hinges on them reporting their safety being above a certain accident threshhold, I think any statistics provided by that company on safety should be doubted unless an indepedent third party can verify them.

[maximus-decimus]

Because they don't make laws just for you, they make laws for everybody including the drunk 16 inexperience texters and the new parents who survive on 3 hours of sleeps.

And those people drive on the same roads as you, so you're still affected. Don't you want other drivers on the road to be less likely to t-bone you because they ran a red light?

[nixgeek]

I’ve experienced both learning to drive in the United Kingdom, and later, in the United States (you have to test again if you immigrate). In summarizing both experiences I’m trying hard to avoid being too biased.

In the UK, almost everyone learned (and learns) using a manual/stick shift vehicle, and if you learn and test in an automatic gearbox, your license is limited and you legally can’t drive manual. Hill starts and clutch control can be much fun! Lessons and the test involved difficult city situations ranging from extremely narrow streets, through 7-lane roundabouts, country roads both single-track and unrestricted (so 60mph speed limit, but not necessarily safe to drive that quickly - good judgement is required). You must pass a theory and hazard perception exercise, and the testing is government-administered.

In the US it seems almost everyone learns in an automatic vehicle, your license then lets you drive stick with no restriction. At least near cities, the roads you learn and test on are seemingly not 2” wider than your vehicle (measured at the mirrors), the situations are comparatively simple as well. There’s no hazard perception test. In my state, the test is administered by the instructor and not an impartial/neutral party.

I haven’t gone looking for large datasets to support this but it feels like the “I just passed my test” driver competence is going to be different.

I would be extremely happy to buy and trust a system like Ultra Cruise if it could navigate UK roads and city situations autonomously with less accidents/incidents than drivers at the 75th percentile in those environment, meaning with widespread adoption the system would raise the bar, and improve median safety properties of being a driver/participant on the roads. However, I would guess had they not cancelled it, being acceptably good for US driving conditions / better than the average US driver really just means you’ve built a system which can work in the US but absolutely won’t work in London, Paris, Berlin or anywhere else?

[danhor]

People are less fine with risk if it's less controllable by themselves. Every mode of transport where people don't directly control their fate is held to a much higher safety standard (and partially as a result) much safer. Aviation, Trains, Busses, ... .

[lorstic]

Because society can't handle that.

And I think that's a fine reason. Even if you don't think it's a fine reason, it's not going to change.

[dgfitz]

> People often make arguments that "oh it will still be less accidents than human drivers", which is true, but, the problem is that human accuracy is a very poor benchmark for autonomous systems.

Never let a good solution get in the way of a good problem.

“But this medicine, while it can cure cancer, has a 1% chance of death!” “Ah shit good point, fuck it then. As long as we can still get pissy with that musk fellow.”