[buro9]

You just know that Tesla would tune the software for this specific route, if the value of the comparison number became meaningful.

Just like you cannot trust benchmarks, Goodhart's law will kick in https://en.wikipedia.org/wiki/Goodhart%27s_law

> Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes

[mensetmanusman]

Their model architecture don't allow for individual route tuning.

[krisoft]

I think that is not true. They can just take data from the route, annotate it and put it into their training set. That is how you would do route tuning.

[mlrtime]

Sort of off-topic but I believe Openpilot allows this. You can record every route you take, copy it off to your PC and run it in a simulator. You can then tweak the model for this particular issue.

[tsimionescu]

They claim.