[jowday]

Neither of them are “full AI/ML”, they’re both traditional robotics systems with ML used for detection/prediction/planning at certain steps. Elon will sometimes say something about moving to a “new ML stack”, but Tesla reverse engineers regularly look inside of what’s running in the cars and that’s not the case at all.

Contrary to what other people in this thread are saying, the remote support isn’t remote direct driving of the car - essentially what will happen is that if the car finds itself in a situation where it’s unsure of how to proceed and it’s safe to stop, it will pause for a few seconds and wait for a remote operator to clarify a situation for it.

A good example might be road construction - if the car detects new road construction work that doesn’t match its map of the area, and its onboard systems determine that it’s not sure how to proceed through the construction with confidence, it will send what it thinks the top five likeliest ways to proceed to a remote operator. The operator then selects the proper path (or says that none of them are proper). The car will then follow the path presented by the operator, but actual driving behavior /collision detection / pathfinding is still determined locally. Think of it like ordering a unit around in StarCraft.

[pnw]

You can actually see this behavior in the car when it runs into a difficult situation. It tells you it's asking for assistance or something similar, and pauses for a few seconds.

I also made the mistake of assuming the remote operator drives the car but if you watch Waymo's technical videos, it's clear that the AI is in control of the car all all times and the remote operator is just doing near real time labelling of what the car is seeing.

[spankalee]

> Tesla reverse engineers regularly look inside of what’s running in the cars and that’s not the case at all

How are they doing this on the software stack? Any references?

[jowday]

Same way reverse engineer get access to internals of other devices - a bunch of tricks. :^)

In one case, greentheonly realized some fraction of Tesla’s cars are shipped out of the factory still in dev mode, with debug mode enabled and increased privileges. He found someone with a car like this who was down to helped and swapped part of his cars hardware with their car, and from then on was able to get a much better view of what was running on his car.

Unfortunately twitter is awful to search and a lot of his info is buried deep in old threads, but a few (old) examples to illustrate that he regularly does this.

Visualizing the outputs of the models running in the car: https://x.com/greentheonly/status/1404164587927314435

Tesla’s dev debug menus circa 2020: https://x.com/greentheonly/status/1336467014727110656

It’s been a while since I’ve worked in the space so I haven’t followed green as closely.

[spankalee]

Wow, thanks!