[xendipity]

A YouTuber named DirtyTesla has done a series of drives over the years throughout his hometown where he's tracked the intervention and disengagement rates per mile. He often shows those numbers at the end of his videos.

[mg]

Is it the same route every time?

If not, they might take harder and harder routes as FSD gets better. Making it look like there is less progress than there really is.

[buro9]

You just know that Tesla would tune the software for this specific route, if the value of the comparison number became meaningful.

Just like you cannot trust benchmarks, Goodhart's law will kick in https://en.wikipedia.org/wiki/Goodhart%27s_law

> Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes

[mensetmanusman]

Their model architecture don't allow for individual route tuning.

[krisoft]

I think that is not true. They can just take data from the route, annotate it and put it into their training set. That is how you would do route tuning.

[mlrtime]

Sort of off-topic but I believe Openpilot allows this. You can record every route you take, copy it off to your PC and run it in a simulator. You can then tweak the model for this particular issue.

[tsimionescu]

They claim.

[numpad0]

There is no publicly known archive of Tesla software images, at all. Which, with the benefit of hindsight, do sound wrong.