[TaylorAlexander]

I would say the race isn’t over yet. One thing to consider is the training dataset. By using only vision, Tesla was able to start collecting data with their entire fleet starting many years ago. They probably have more edge case data than anyone else, in more diverse driving situations. They would not have been able to do this if they needed to use expensive lidar on company owned training vehicles.

What we will likely see is that lidar equipped vehicles gain a lot of ground at first, but have to be rolled out slowly both due to cost and how they are trained. Tesla’s vision based fleet could get an update tomorrow that theoretically was able to drive almost anywhere.

Of course, Tesla has clearly been finding out that while vision is in principle sufficient, the brains behind the cameras are not so simple. In my opinion it’s still too early to call who will win in the end.

[fh973]

This point about Tesla has been made frequently in this discussion. I still wonder: how does Tesla store any meaningful video data in customer vehicles and transfer it?

I know that other vendors have a hard time putting the necessary fast storage in the trunk and get the data off with 100G cables in the garage.

[TaylorAlexander]

Their AI system has something called “shadow mode” where it can make observations without having any effect on the car. When Tesla needs to collect a dataset of something, like short videos of cars that put on their blinker but did not change lanes, or partially obscured stop signs, they can train a net to run in shadow mode on all their cars and collect more of this edge case. I presume that they also collect a lot of data around disengagements. And then they send the data presumably over Wifi when the customer gets home, or otherwise over the car’s 3G connection. But they collect targeted data, not mass dumps of drives.

Andrej Karpathy has described this system in various talks including on Tesla AI day.

[317070]

As I understand it, it's mainly collected when the car goes in for maintenance.