[throwaway894345]

You seem to view safety as some binary that doesn't account for frequency or severity of incidents. Your framing suggests you think it's better to continue with human drivers and the commensurate 40k lives lost each year than to use an AI that has even the slightest possibility of causing even the least significant accident irrespective of whether or not any accident occurs in practice.

I suggest it's better to compare a given AI with humans in terms of fatalities caused per million miles driven. If an AI performs a little better than humans it should be legalized and if it performs dramatically better than humans, it should be mandatory.

Of course, this is where we need more data and greater transparency so we can answer these questions.

[noirbot]

I'm not trying to say that at all. What I'm saying is that we're at an awkward time now, where this sort of quick AI-assisted breaking is especially dangerous because fallible humans are most of the rest of the drivers on the road. In an all FSD world, this wouldn't really be a problem.

You're putting a lot of words in my mouth and assuming I'm against working on AI driving because one person might ever die. All I'm trying to point out is that it's pretty worrying to have a system that could cause a highway-speed accident because of a well-known and decently common bug. I'd be equally worried if it came up that some other decently selling model of car would randomly have the ABS system engage.

I wrote my OP here because the parent poster was casually talking about "not really break checking people" as if that's just normal behavior that's a part of R&D, instead of an AI accidentally emulating dangerous aggressive driving patterns that FSD is supposed to do away with. I'm not trying to ban FSD research or anything. I want this improved! It's just scary when people excuse dangerous behaviors by FSD systems because it's otherwise safer.

The other issue is that more data and greater transparency are both not things Tesla seems to have any interest in providing anyone, so while this may get fixed, it's not really pushing the industry forward all that much if no one other than Tesla is going to benefit. There's plenty of mentions in this thread of this sort of issue happening on other cars and adaptive cruise control systems that could benefit from an improvement for the betterment of all drivers, but instead "not breakchecking people" is going to be a unexplained feature improvement in some FSD patch probably.