[slg]

There is zero context on that video that tells us what is happening there. We have no idea if that is the Autopilot failing or the emergency braking failing. We also have no control group to tell us what percentage of humans would stop short of that dummy. It is inexact due to Twitter's video player not showing fractions of a second, but it looks like there were approximately 2 seconds between when the dummy started moving forward and when it was hit by the car. The average human time to braking is 2.2-2.3 seconds[1]. Is the car even failing that test in comparison to a human?

It also isn't clear from watching that video what the safest and therefore desired behavior should be in that situation. A self driving car is obviously not going to prevent all accidents, so it is a question of minimizing potential harm. We don't want a car to aggressively brake whenever someone at a street corner takes a step towards the road. We therefore need to balance the chance of a person stepping into the path of the car with the risk of braking when it is unnecessary and causing a rear end collision. The problem in the linked thread is overaggressive braking so forcing the car to pass a test that rewards overaggressive braking would only make that specific example worse.

That leads back to my point about needing a huge amount of data. You can't just run a car through an obstacle course to know whether it is safer or more dangerous than a human. You need to have it interacting with unpredictable humans and you need to do it repeatedly before you can confidentially predict whether it is safer or more dangerous than a human.

[1] - https://copradar.com/redlight/factors/IEA2000_ABS51.pdf

[dragontamer]

> We also have no control group to tell us what percentage of humans would stop short of that dummy.

IIRC, Subaru and other companies pass these simple emergency braking tests 100% of the time.

That test was a Chinese test IIRC, but the software doesn't change between countries. Similar tests have been done here in the USA by insurance groups to set insurance rates, but a government-mandated test for what "emergency braking" really means (before you "sell the feature to the public", lets actually have a government-mandated test similar to that video).

You shouldn't be allowed to call your stuff "autopilot" or "full self driving", or "emergency braking" or "pedestrian avoidance" (or some other set of words) unless you can... you know, avoid pedestrians and emergency brake in a well-controlled test.

Avoiding balloon people is enough. But its a well known fact that Tesla repeatedly fails at these simple tests, when other groups (ie: Mobileye group / Mobileye hardware) manages to emergency brake in time.

----------

IIHS test: https://www.iihs.org/news/detail/performance-of-pedestrian-c...

The issue is that 3rd party non-government groups (ie: IIHS) are the ones running these tests. There's no advocacy group for US consumers as far as I can tell. IIHS is primarily about serving their master (insurance companies).

Don't get me wrong: IIHS is doing good work here. But its not their job to protect the consumer.

EDIT: I got my sources mixed up. Tesla apparently passed the IIHS test.

It was the AAA test they failed: https://insideevs.com/news/377427/video-tesla-model-3-failed...

[slg]

>IIRC, Subaru and other companies pass these simple emergency braking tests 100% of the time.

They do not pass 100% of the time unless you have a very narrow definition of "these simple emergency braking tests". No emergency braking system is foolproof.

>But its a well known fact that Tesla repeatedly fails at these simple tests,

You say this while at the same time the source you include has Tesla in the middle tier of results.

Either way, my point is not that Teslas are safe or that they perform well on this test. The point is that this test does not tell you whether a car being driven by Autopilot is safer than a human.

[dragontamer]

> They do not pass 100% of the time unless you have a very narrow definition of "these simple emergency braking tests"

Lets get them working consistently under well defined, standard, simple, emergency braking tests before worrying about the real world.

Like not hitting a balloon dressed up as a pedestrian during clear skies in sunny weather. I don't care about rainy days until we get the bright / sunny weather figured out.

[slg]

>Lets get them working consistently under well defined, standard, simple, emergency braking tests before worrying about the real world.

Automatic emergency braking is the exact wrong feature to use for your example. Either the driver sees the pedestrian, stops in time, and the automatic emergency braking is of no use or the driver would have hit the pedestrian and any effort from the automatic system is a benefit. This is the type of feature that should be deployed as soon as possible assuming it is not tuned too aggressively to stop at false alarms.