[angusb]

This has done the rounds on other platforms. A couple of important points:

- the failure here is that the car didn't stop for the bus on the other side of the road with the extended stop sign. (Obviously a kid running out from behind a car this late is fairly difficult for any human or self driving system to avoid)

- the FSD version for the robotaxi service is private and wasn't the one used for this test. The testers here only have access to the older public version, which is supposed to be used with human supervision

- the dawn project is a long-time Tesla FSD opponent that acts in bad faith - they are probably relying on false equivalence of FSD beta vs robotaxi FSD

Nevertheless this is a very important test result for FSD supervised! But I don't like that the dawn project are framing this as evidence for why FSD robotaxi (a different version) should not be allowed without noting that they have tested a different version.

[fabian2k]

I don't see why Tesla would deserve the benefit of the doubt here. We cannot know how well the actual Taxi software will work, I think it is fair to extrapolate from the parts we can observe.

[angusb]

re. extrapolation: I agree with that, but remember there's sampling error. The crashes/failures go viral but the lives saved get zero exposure or headlines. I don't think that means you can just ignore issues like this but I think it does mean it's sensible to try to augment the data point of this video with imagining the scenarios where the self driving car performs more safely than the average human driver

[fabian2k]

I absolutely do think that self-driving cars will save many lives in the long run. But I also think it is entirely fair to focus on the big, visible mistakes right now.

This is a major failure, failing to observe a stop sign and a parked school bus are critical mistakes. If you can't manage those you're not ready to be on the road without a safety driver yet. There was nothing particularly difficult about this situation, these are the basics you must handle reliably before we even get to alle the tricker situations those cars will encounter in the real world at scale.

[angusb]

I agree it's a major mistake + should get a lot of focus from the FSD team. I'm just unsure whether that directly translates to prohibiting a robotaxi rollout (I'm open to the possibility it should though).

I guess the thing I'm trying to reconcile is that even very safe drivers make critical mistakes extremely rarely, so the threshold at which FSD is safer than even the top 10% of human drivers likely includes some nonzero level of critical mistakes. Right now Tesla has several people mining FSD for any place it makes critical mistakes and these are well publicised so I think we get an inflated sense of their commonality. This is speculation, but if true it leaves some possibility of it being significantly safer than the median driver while still allowing for videos like this to proliferate.

I do wish Tesla released all stats for interventions/near misses/crashes so we could have a better and non-speculative discussion about this!

[ethbr1]

Zero systemic, reproducible mistakes is the only acceptable criteria.

Do you really want to trust a heartless, profit-motivated corporation with 'better than human is good enough'?

What happens when Tesla decides they don't want to invest in additional mistake mitigation, because it's incompatible with their next product release?

[Reubachi]

Caveat/preface to prevent trolls: FSD is a sham and money grab at best, death trap at worst, etc.

But, I've read through your chain of rplies to OP and maybe I can help with my POV.

OP is replying in good faith showing "this sampling incident is out of scope of production testing/cars for several reasons, all greatly skewing the testing from this known bad actor source."

And you reply with "Zero systemic reproducible mistakes is the only acceptable critera."

Well then, you should know, that is the current situation. In tesla testing, they achieve this. The "test" in this article, which the OP is pointing out, is not a standardized test via Tesla on current platforms. SO be careful with your ultimatums, or you might give the corporation a green light to say "look! we tested it!".

I am not a tesla fan. However, I also am aware that yesterday, thousands of people across the world where mowed down by human operators.

If I put out a test video showing that a human runs over another human with minimum circumstances met, IE; rain, distraction, tires, density, etc., would you call for a halt on all human driving? Of course not, you'd investigate the root cause, which is most of the time, distracted or impaired driving.

[angusb]

> Do you really want to trust...

No, but the regulator helps here - they do their own independent evaluation

> What happens when Tesla decides...

the regulator should pressure them for improvements and suspend licenses for self driving services that don't improve

[spwa4]

It should get a lot of focus from the regulator, not "the FSD team".

[ethbr1]

Tesla and this administration operate under the Boeing model: surely the manufacturer knows best.

[angusb]

agreed - I said FSD team to distinguish from "the crowds" but this was the wrong wording, should be the regulator too.

[locococo]

No! Ignoring a stop sign is such a basic driving standard that it's an automatic disqualification. A driver that misses a stop sign would not have my kids in their car. They could be the safest driver on the racetrack it does not matter at that point.

[reaperducer]

I agree with that, but remember there's sampling error.

Ma'am, we're sorry your little girl got splattered all over the road by a billionaire's toy. But, hey, sampling errors happen.

[dzhiurgis]

Also they've repeatedly tested closer and closer distances until Tesla failed aka p-hacking.

[sokoloff]

In the video (starting at ~13 seconds), the Tesla is at least 16 and probably 20 car lengths from the back of the bus with the bus red flashing lights on the entire time.

If the Tesla can't stop for the bus (not the kid) in 12 car lengths, that's not p-hacking, that's Tesla FSD being both unlawful and obviously unsafe.

[locococo]

I thin their test is valid and not in bad faith because they demonstrate the Teslas self driving technology has no basic safety standards.

Your argument that a newer version is better simply because it's newer does not convince me. The new version could still have that same issue.

[angusb]

> Your argument that a newer version is better

I actually didn't say that and am not arguing it formally - I said what I said because I think that the version difference is something that should be acknowledged when doing a test like this.

I do privately assume the new version will be better in some ways, but have no idea if this problem would be solved in it - so I agree with your last sentence.

[Topfi]

> […] I think that the version difference is something that should be acknowledged when doing a test like this.

Did they anywhere refer to this as Robotaxi software over calling it FSD, the term Tesla has chosen for this?

[angusb]

I don't think there's an official divergence in terminology other than the version numbers (which have mostly stopped incrementing for FSD vehicle owners, meanwhile there is a lot of work going into new iterations for the version running on tesla's robotaxi fleet)

[Topfi]

Then I struggle to understand what they should have acknowledged here concerning the software used? That they do not have access to a version of FSD which currently isn’t accessible to the public? I’d think that’s self-evident for any Organisation not affiliated with Tesla.

[interloxia]

It is also a failure that it does a cruise and continues to drive over the thing/child that was hit.

[angusb]

that should have been in my list, you're right

[chneu]

Everytime Tesla's FSD is shown to be lacking someone always says "well that's not the real version and these people are bias!"

[ryandrake]

Don't forget the standard "And the next version surely will be much better!"

[mxschumacher]

it's always the next version that will go from catastrophic failure to being perfect. This card has been played 100 times over the last few years.

[jantissler]

Exactly what I wanted to add. Every single time there is hard evidence what a failure FSD is, someone points out that they didn't use the latest Beta. And of course they provide zero evidence that this newer version actually addresses the problem. Anyone who knows anything about Software and updates understand how new versions can actually introduce new problems and new bugs …

[locococo]

excatly this

[bestouff]

> Obviously a kid running out from behind a car this late is fairly difficult for any human or self driving system to avoid

That's why you slow down when you pass a bus (or a huge american SUV).

[sokoloff]

In this case, you stop for the bus that is displaying the flashing red lights.

Every driver/car that obeys the law has absolutely no problem avoiding this collision with the child, which is why the bus has flashing lights and a stop sign.

[ethbr1]

I'm honestly shocked that all versions of Tesla assisted/self-driving don't have a hardcoded stopped schoolbus exception to force slow driving.

Worst case, you needlessly slow down around a roadside parked schoolbus.

Best case, you save a kid's life.

[ryandrake]

If it merely slowed down, it would still be driving illegally. Everywhere I've driven in the USA, if there is a stopped school bus in front of you, or on the other side of the road, all traffic on the road must stop and remain stopped until the school bus retracts the sign and resumes driving.

[ethbr1]

Point being, if it always slowed down (sign or no sign), then at least reaction time requirements would be lessened.

[ryandrake]

I mean, it can be programmed in any way, so why not program it to follow both rules: If the sign is extended, then stop. Else if the sign is not extended, slow down.

[pas]

if I put a stop sign somewhere is that legal? or there's some statute (or at least local ordinance) that says that yellow buses on this and this route can have moving stop signs?

[sokoloff]

Obviously, there is law that forms the basis for this. In Massachusetts, it's MGL c. 90 § 14

https://www.mass.gov/info-details/mass-general-laws-c90-ss-1...

[addandsubtract]

Yes, in the US, school buses come equipped with a stop sign that they extend when stopping to let kids on and off the bus. You (as a driver) are required to stop (and not pass) on both sides of the road while the bus has their sign extended.

[q3k]

It's almost as if we need human-level intelligence to be actually able to reason about the nuances of applicability of traffic signs :).

[Sharlin]

Exactly. If your view is obstructed by something, anything, you slow down.

[ndsipa_pomu]

> Obviously a kid running out from behind a car this late is fairly difficult for any human or self driving system to avoid

Not really - it's a case of slowing down and anticipating a potential hazard. It's a fairly common situation with any kind of bus, or similarly if you're overtaking a stationary high-sided vehicle as pedestrians may be looking to cross the road (in non-jay-walking jurisdictions).

[burnt-resistor]

Yes. And given the latency of cameras (or even humans) not being able to see around objects and that dogs and kids and move fast from hidden areas into the path, driving really slow next to large obstacles until able to see behind them becomes more important.

One of the prime directives of driving for humans and FAD systems must be "never drive faster than brakes can stop in visible areas". This must account for such scenarios as obstacles stopped or possible coming the wrong way around a mountain turn.

[ndsipa_pomu]

> never drive faster than brakes can stop in visible areas

Here in the UK, that's phrased in the Highway Code as "Drive at a speed that will allow you to stop well within the distance you can see to be clear". It's such a basic tenet of safety as you never know if there's a fallen tree just round the next blind corner etc. However, it doesn't strictly apply to peds running out from behind an obstruction as your way ahead can be clear, until suddenly it isn't - sometimes you have to slow just for a possible hazard.

[BobaFloutist]

And that should be a case where automated systems can easily beat people. It's "just" math, there's no interpretation, no reasoning, no reading signs, no predicting other drivers, just crunching the numbers based on stopping distance and field of view. If they can't even do this, what are they for?

[Zigurd]

The simulation of a kid running out from behind the bus is both realistic, and it points out another aspect of the problem with FSD. It didn't just pass the bus illegally. It was going far too fast while passing the bus.

As for being unrepresentative of the next release of FSD, we've had what eight years ten years of it's going to work on the next release.

[arccy]

sounds like a tesla problem for naming their crappy tech "full self driving"

[enragedcacti]

Some important counter-points:

- FSD has been failing this test publicly for almost three years, including in a Super Bowl commercial. It strains credulity to imagine that they have a robust solution that they haven't bothered to use to shut up their loudest critic.

- The Robotaxi version of FSD is reportedly optimized for a small area of austin, and is going to extensively use tele-operators as safety drivers. There is no evidence that Robotaxi FSD isn't "supposed" to be used with human supervision, its supervision will just be subject to latency and limited spatial awareness.

- The Dawn Project's position is that FSD should be completely banned because Tesla is negligent with regard to safety. Having a test coincide with the Robotaxi launch is good for publicity but the distinction isn't really relevant because the fundamental flaw is with the companies approach to safety regardless of FSD version.

- Tesla doesn't have an inalienable right to test 2-ton autonomous machines on public roads. If they wanted to demonstrate the safety of the robotaxi version they could publish the reams of tests they've surely conducted and begin reporting industry standard metrics like miles per critical disengagement.

[BobaFloutist]

>(Obviously a kid running out from behind a car this late is fairly difficult for any human or self driving system to avoid)

Shouldn't that be the one case where self driving system has an enormous natural advantage? It has faster reflexes, and it doesn't require much, if any, interpretation or understanding of signs or predictive behavior of other drivers. At the very worst, the car should be able to detect a big object in the road and try to brake and avoid the object. If the car can't take minimal steps to avoid crashing into any given thing that's in front of it on the road, what are we even doing here?

[ndsipa_pomu]

I agree - it's why I think some type of radar/lidar is necessary for autonomous vehicles to be sufficiently safe. I get the theory that humans can process enough info from two eyes to detect objects, so multiple cameras should be able to do the same, but it looks like it's a tough problem to solve.

[WillAdams]

Does the Tesla taxi option afford radar/lidar?

My understanding is that Tesla is the only manufacturer trying to make self-driving work with just visual-spectrum cameras --- all other vendors use radar/lidar _and_ visual-spectrum cameras.

[ndsipa_pomu]

They've painted themselves into a corner really, as if they start using anything other than just cameras, they'll be on the hook for selling previous cars as being fully FSD-capable and presumably have to retrofit radar/lidar to anyone that was mis-sold a Tesla.

[Zigurd]

"an appreciating asset" IIRC

[Zigurd]

...and radar and mics and Google's geospatial data and 15 years of experience.

[rsynnott]

... Wait, so you think that Tesla have a child-killing and a non-child-killing version, but are only providing the child-killing version to consumers?

... eh? I mean, what?

[handsclean]

I’ve seen this “next version” trick enough times and in enough contexts to know when to call BS. There’s always a next version, it’s rarely night-and-day better, and when it is better you’ll have evidence, not just a salesman’s word. People deserve credit/blame today for reality today, and if reality changes tomorrow, tomorrow is when they’ll deserve credit/blame for that change. Anybody who tries to get you to judge them today for what you can’t see today is really just trying to make you dismiss what you see in favor of what you’re told.

[aeurielesn]

Sorry, I am failing to see how any of these three points are relevant or even important.