[dylan604]

what do you call them? there's no way possible that they can make changes to the software and have them thoroughly vetted before the OTA push. Tesla does not have enough cars owned by the company driving on public roads to vet these changes. The QA team at best can analyze the data received from the customers. That makes the customers the testers in my book.

[93po]

the fact that you don't know how tesla vets these changes, very extensively, prior to any physical car receiving the update reinforces that you have no idea what you're talking about

tesla does extensive, meaningful vetting of these updates. i'll let you do the research yourself so that maybe you can quit spreading misinformation

[dylan604]

so well vetted that issues keep happening, so well vetted that a OTA "software patch" is raised to the level of automotive recall. if you call that misinformation, then, "boy, i don't know".

[JoshCole]

I think you are just correct that customers are acting as testers. I think the issue he is having isn't on that point, where you are right.

It is well known that issues happen despite extensive vetting. The presence of vetting does not exclude the absence of issues. For example, lets say there are 1000 defects in existence and your diagnostic finds 99% of them. So it finds 990 defects. So now there are 10 defects remaining that are not found. Next you have another detector that finds all defects. How many defect does it see? 10. It is usually going to see about ten defects. Your expectation given vetting is taking place should that be you will tend to observe about ten defects in this particular situation.

So lets say you are someone who is watching that second detector. You observe that you keep seeing defects. Probably, because of selection bias on results, you observe this with an alarming 100% rate of finding defects - the few times no defects happen it doesn't get shared for much the same reason we don't pay attention to drying paint. Can you therefore claim there is no diagnostic that is detecting and removing defects?

Well, not really, because you aren't observing the defects unconditional on the process that removed them. Basically, when you observe 10 defects, that doesn't mean there weren't 990 other defects that were removed. It just means you observed 10 defects.

So the actual evidence you need to use to decide whether there is a diagnostic taking place is evidence that is more strongly conditional on whether it is taking place. You need something that can observe whether the 990 are being caught.

In this case, we have video evidence of extensive QA processes. So that is much stronger than the evidence that defects show up, because defects show up both when we do have a vetting and we don't have a vetting. And each reasonable test case ought to be decreasing the chance of a particular defect that the test case is testing for.

For a much more thorough treatment on this subject check out:

https://www.lesswrong.com/tag/bayesian-probability#:~:text=B....

So that is basically why he disagrees with you.

Tesla definitely needs to root cause the defects that were found and make improvements on the existing vetting process but it is very obvious they do have these processes.