[TaylorAlexander]

Andrej Karpathy was the AI lead for most of the project and he has talked about the general system design.

They have a set of regression tests they run on new code updates either by feeding in real world data and ensuring the code outputs the expected result, or running the code in simulation.

It does seem worrying that they would miss things like this.

Here’s a talk from Karpathy explaining the system in 2021:

https://youtu.be/aNVbp0WKYzY

Though I don’t recall if he explains the regression testing in this talk, there’s a few good ones on YouTube.

[ertian]

It's not even a bit surprising they'd miss things like this, IMHO. They do tests with a few (maybe even a lot of) intersections, but there are thousands upon thousands of intersections out there, including some where bushes are obscuring a stop sign, or the sign is at a funny angle, or sunlight is reflecting off the traffic lights, or heavy rain obscuring them, or plain old ambiguous signage...there's _bound_ to be mistakes. Human drivers make similar mistakes all the time.

I used to think that fact was going to delay self-driving cars by a decade or more, because of the potential bad press involved in AI-caused accidents, but then along comes Tesla and enables the damn thing as a beta. I mean...good for them, but I've always wondered if it was going to last.

I've been using it pretty consistently for a few months now (albeit with my foot near the brake at all times). I haven't experienced any of the above. Worst thing I've seen is the car slamming on the breaks on the freeway for...some reason? There was a pile-up in a tunnel caused by exactly that a month or so ago, so I've been careful not to use FSD when I'm being tailgated, or in dense traffic.

[saalweachter]

You know, there was an article on here last week about how there are only 4 billion floats, so just test them all.

There are only like 16 million intersections in the US. Why not test them all?

[TaylorAlexander]

The thing is you already know everything you need to know about all 4 billion floats. Collecting data on every intersection in the US is quite difficult.

Tesla does however collect data on edge cases and then train their system to respond correctly. They can for example trail a collection network to identify things that might be obscured stop signs, then have the fleet collect a whole bunch of examples, hand label those samples, and roll this new data in to the training system. This is explicitly how they handle edge cases.

They can also create a new feature or network and roll it out in “shadow mode” where it is running but has no influence on the car, and then they can observe how these systems are behaving in the real world.

The real issue I guess is when they release a new feature without trialing it in shadow mode, or if they have gaps in their testing and validation system.