[JoshCole]

I strongly feel people ought to have these discussions while consistently citing actual data sources relevant to the discussion.

For example, did you predict, based on the speculation of Tesla being incompetent with regard to safety, that they have the lowest probability of injury scores of any car manufacturer? Because they do.

Did you predict, based on speculation about Elon Musk's incompetence in predicting that self-driving would happen, that there are millions of self-driving miles each quarter? Because there are.

Did you predict, based on speculation about Tesla incompetence in full self-driving, that the probability of accident per mile is lower rather than higher in cars that have self-driving capabilities? Because they do.

I know this sort of view is very controversial on Hacker News, but I still think it is worth stating, because I think people are actually advocating for policies which kill people because they don't actually know the data disagrees with their assumptions.

https://www.tesla.com/VehicleSafetyReport

[justapassenger]

Unaudited (internal Tesla data), cherry-picked (comparing with average cars in USA, which are 12 years old beaters, to their very young fleet of expensive cars) data, that doesn't correct for any bias (highway driving vs non-highway driving being one of the many issues) is not exactly the magic bullet you think it is.

Also, none of that is self driving. This data talks about AP, not FSD. FSD is also not self driving by any means (it's level 2 driver assist), but that's a detail at this point.

[JoshCole]

I didn't say it was a magic bullet. So you are hallucinating thoughts about me, not responding to what I said. Being critical of the data like you are being is good thinking in my opinion. I just don't like how often people don't have beliefs that are anywhere close to the data.

For example, elsewhere in this comment thread, someone threw out a random statistic of 400:1 as part of their argument, but this seems to me to be something like six orders of magnitude diverged from a data informed estimate.

To try and contextualize how big an error that is - it is like thinking that a house in the Bay Area has the same cost as a soft drink.

I think if we have to cite our data we are less likely to do that sort of error and more likely to catch it when it is done.

I definitely don't think FSD is magically safe. So if you think that is what I'm trying to say, please update your beliefs according to my correction that I do not believe this. I think anyone driving in FSD should remain vigilant, because it can make worse decisions than a human would.

[bischofs]

A system that protects 400 people but kills 1 is not a system that I want on public roads because I don't want to be in the 1 - Elon and the children of Elon are basically making the assumption that everyone is okay with this.

The probability of an accident for any driver assistance system will ALWAYS be lower than a human driver - but that doesn't mean the system is safe for use with the general public!

People like me are not advocating for "killing people" because we aren't looking at data - it's that no company has the right to make these tradeoffs without the permission and consent of the public.

Also if this was about safety and not just a bunch of dudes who think they are cool because their Tesla can kinda drive itself, why does "FSD" cost $16,000?

[fallingknife]

> People like me are not advocating for "killing people"

If you are advocating against a system that protects 400 people and kills one, you are advocating for killing people.

[JoshCole]

> A system that protects 400 people but kills 1 is not a system that I want on public roads because I don't want to be in the 1 - Elon and the children of Elon are basically making the assumption that everyone is okay with this. > > The probability of an accident for any driver assistance system will ALWAYS be lower than a human driver - but that doesn't mean the system is safe for use with the general public!

Totally we should be wary of a system that protects 400 and kills 1. Thank you for providing the numbers. It helps me show my point more clearly.

If you are driving on a road you encounter cars. Each car is a potential accident risk. You probably encounter a few hundred cars after ten or so miles. Not every car crash kills, but lets just assume they all do to make this simpler. For the stat you propose, you are talking about feeling uncomfortable with an accident per mile of something around the ballpark of ten miles.

Now lets look at the data. The data suggests the actual miles per accident is closer to 6,000,000 miles per accident. This is six orders of magnitude diverged from the number of miles per accident that you imply would make you feel uncomfortable.

Lets try shifting that around to a context people are more familiar with: a one dollar purchase would be a soft drink and a six million dollar purchase would be something like buying a house in the bay area. This is a pretty big difference I think. I feel very differently about buying a soft drink versus buying a house in the Bay Area. If someone told me they felt that buying a house was cheap, then gave a proposed price for the house that was more comparable to the cost of buying a soft drink, I might suspect they should check the dataset to get a better estimate of the housing prices, because it might give them a more reasonable estimate.

So I very strongly feel we should cite the numbers we use. For example, I feel like you should really try and back up the use of the 400 to 1 number so I understand why you feel that is a reasonable number, because I do not feel that it is a reasonable number.

> Also if this was about safety and not just a bunch of dudes who think they are cool because their Tesla can kinda drive itself, why does "FSD" cost $16,000?

Uh, we are a on venture capitalist adjacent forum. You obviously know. But... well, the price of FSD is tuned to ensure the company is profitable despite the expense of creating it as is common in capitalist economies with healthy companies seeking to make a profit in exchange for providing value. It is actually pretty common for high effort value creation, like creation of a self-driving car or the performance of surgery, for the prices to be higher.

[prewett]

Interesting graph, I like that it's broken out into quarters. But,

1) those are statistics for the old version, the new version might be completely different. I've had enough one-line fixes break entire features I was not aware of that my view is that any change invalidates all the tests. (Including the tests that Tesla should have but doesn't) Now probably a given update does not cause changes outside its local area, but I can't rely on that until it's been tested.

2) the self-driving is presumably preferentially enabled for highway driving, which I assume has fewer accidents per mile than city driving, so comparing FSD miles to all miles is probably not statistically valid.

[JoshCole]

I agree with you. I would really like to see datasets that reflect how things actually are. I think it would be really dangerous to jump to FSD being safe on the basis of the data I shared. However I would hope that whatever opinions people shared were congruent with the observed data. I don't feel like the prediction that Elon Musk and Tesla not caring about safety is congruent with the observed data, which shows the autopilot has improved safety, best explains the observations of improved safety.

Just for context - I've been in a self-driving vehicle. Anecdotally, someone slammed on the breaks. The car stopped for me, but I was shocked: for hours before this the traffic hadn't changed, it was a cross country trip. I think I would have probably gotten in an accident there. Also anecdotally, there are times where I felt the car was not driving properly. So I took over. I think it could have gotten into an accident. Basically, for me, the best explanation I have for the data I've seen right now is that human + self-driving is currently better than human and currently better than self-driving. The interesting thing about this explanation is how well it tracks with other times where we've technology like this before. In chess playing for example, there was a period before complete AI supremacy (which is what we have now) where human + AI was better than AI.

I like the idea of being safe, so if the evidence goes the other way, advocating for only humans or only AI doing the driving, I want to follow that evidence. Right now I think it shows the mixed strategy is best and that is kind of nice to me because it implies that the policy that best collects data to reduce future accidents through learning happens to be the policy that is currently being used. I like that.

[mcguire]

As any Tesla supporter will tell you, Autopilot != FSD.

(Is Autopilot still limited to divided, limited access highways? Those are significantly safer than other roadways.)

[rootusrootus]

> Is Autopilot still limited to divided, limited access highways

No. Was it ever? All you need is a piece of road that has something which appears to be lane lines. The road to my house is usable despite having no actual paint striping because it happens to have a crack that runs fairly straight up one side and was filled with tar. So the camera thinks it's a lane line. Ta-da!

[super_flanker]

This report is for Autopilot, not FSD which everyone else is talking about on HN.

[JoshCole]

Good point.

The thing is we often have discussions about this stuff and I'm trying to advocate for citing datasets to more tightly correlate our words with the evidence that our words correspond to. I'm not trying to say this version shouldn't have been recalled for example, but that I think we should be close to evidence.

In the case of auto-pilot, it was the case that people made the same arguments that are now being made against FSD. I think that makes it somewhat relevant to the discussion, because people previously also made the same claims about safety, but now that we have the data, we can see those claims were wrong. I believe these sort of generalizations, though inaccurate, can help us to make more informed decisions, but I'm not really confident in any beliefs that are made at this greater decision from direct data.

So I think anyone who can provide datasets which correspond with FSD performance rather than autopilot performance ought to do so. That would be really great data to reflect on.

The thing I'm worried about is that no data at all is backing the conjectures - which, given that I sometimes see estimates that I calculate to be many orders of magnitude away from data informed estimates - seems to be the case on Hacker News at least some of the time.

[freejazz]

Please ignore all the times I'm wrong in favor of all the times I'm right!

[JoshCole]

I agree that people who don't cite the evidence are ignoring the evidence? Are you trying to say I'm doing that by pointing to relevant datasets which track the number of accidents and the probability of injury? If so, why are there accidents tracked in the datasets such that the rate can be calculated? This kind of contradicts the claim that I'm asking to ignore, but I definitely agree that other people are ignoring the data if that is what you are trying to say.

[freejazz]

No, your argument is just ridiculous. The standard isn't and shouldn't be how much they get right. It should be what they get wrong and how they do that. I completely disagree with your point, and phrasing it obtusely just makes you obnoxious from a conversational standpoint.

[JoshCole]

My position is that we ought to include assertions backed by the evidence. Your views probably do have evidence that supports them. I want to see the evidence you are using, because I think that is important.

I'm not sorry that annoys you, because it shouldn't.

[freejazz]

>Oh. So you don't like the data, because it disagrees with you. So you are trying to pretend I'm ignoring data, even though I'm linking to summary statistics which by their nature summarize the statistics rather than ignoring the statistics.

Oh the data is great. I like the data. I'd take the data out to dinner. It's completely besides my point, and you continuing to be obtuse and rephrasing things this way, is not only a strawman, but it's rude.

> Your views probably do have evidence that supports them. I want to see the evidence you are using, because I think that is important.

Not every policy decision is driven by data. Some are driven by reasoning and sensibility, as well as deference to previous practices. So your whole data-driven shtick is just that... a shtick.

[JoshCole]

You claim that I said that we should ignore evidence, but I didn't. I claimed that we should look at it.

You claim that I said that we should focus on the good, but I didn't. I claimed that we should look at the data.

Now I feel as if you are trying to argue that looking at data is wrong because not all decisions should be made on the basis of the data. This seems inconsistent to me with your previous assertion that my ignoring data was bad, because now you argue against your own previous position.

That said, uh, datasets related to bayesian priors support your assertions about deference in decision making. So you could, if you cared to, support that claim with data. It would contradict your broader point that I should not want to have data, but you could support it with data and I would agree with you, because contrary to your assertion I was making an argument for evidence informed statements. Your inference about whether I think the evidence leans should not be taken as an argument that I believe my positions would always be what was reached by looking at the evidence, because I don't think that is true. I'm obviously going to be wrong often. Everyone is.

Unfortunately, I think you lie too much and shift your goalposts too much. So I'm not going to talk to you anymore.