[rjp0008]

> the autopilot was programmed to go in a straight line and at an intersection decided to steer into a crosswalk.

Agree this is a weird and very concerning bug.

> Not to mention the fact that it seems entirely legal for a company like Tesla to test such dangerous functionality on public streets with other drivers and pedestrians, with zero permission or regulation.

This driver seems to be someone who I would be ok with testing this. He's got his hand an inch from the steering wheel and obviously paying attention. I would rather this scenario happen, than Tesla be restricted to no public road access and they just unleash an untested system.

[sleepybrett]

The problem, as I see it, is that the longer driver uses the autopilot feature the less attention they will pay to it as long as it doesn't do something nuts like this. My understanding is that this occurred soon after a new 'beta' update to the autopilot system. I'm not sure how this is surfaced to the user, if you need to opt into this new version etc.

My fear is that a new version of the autopilot system could have new bugs introduced that could disrupt the navigation on a route that the user has confidence that the previous version of the autopilot could handle. They commute on route x every day and over time they've gained confidence in the autopilot on that route. This new update however is going to run them into some obstruction due to a bug. That obstruction might be those monorail pylons we see in the seattle video or it might be a crosswalk. A driver who had confidence in the autopilot might be well distracted having confidence in his automation and not be able to correct the situation before tragedy.

IMO autopilot should be banned until it can be proved to be 100% safe. I don't think we can get there until roads are outfitted with some kind of beaconing system that sensors in the car can read and cars on the road are potentially networked together cooperatively... and only then to be enabled on roads with those markers/beacons.

People in this thread deciding that the system is safe because it's no worse than a drunk driver or student driver are missing the point. We absorb those hazards into the system because they present only a handful agents in the system. Out of 100,000 drivers in a rush hour flow, how many are students and/or drunk.. probably very few. However as teslas keep selling with this feature, our new class of hazard agents keeps going up and up and up.

Hell we wouldn't even have to mandate it to be illegal at the political level. Perhaps the insurance industry will do it for us.

[frumper]

Just curious, how does one prove 100% safe? Even if Tesla's were show to be safe on every road in existence there are always too many variables. Weather, construction, road condition, other drivers, objects falling into the roadway, on and on.

There is some level of reasonable safety that should be expected, but proving 100% safe isn't a realistic goal, nor is it even a standard for existing automobiles.

[sleepybrett]

I'm not sure you could get to 100% safety. See any rail system in the world, however I'm more comfortable with our general rate of derailments I think (I haven't looked into the statistics). So I think we could get to a 'safe' system that looks like 'automated freight carrying in a special lane that is restricted, has built in 'hinting' and only if all the freight carrying vehicles are networked' Let the truck drivers get in that lane, fully engage automatic pilot and hopefully one day take a nap, to be awoken upon approach to their destination where they will take over, move out of the special lane, exit the freeway and proceed to their dropoff/pickup.