[pdkl95]

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski

Any engineer with this attitude needs to learn the lesson of the Therac-25. The issues in the Ars article are very similar to section 4 "Causal Factors" of the report[1].

> To get to that better software faster we should deploy the first 1000 cars asap.

Is that admitting that they do not have the "better software" and intend to deploy 1000 cars using "lesser software"? That's treading dangerously close to potential manslaughter charges if prove this willful contempt for safety to a court.

[1] http://sunnyday.mit.edu/papers/therac.pdf

[phlo]

> willful contempt for safety

To play Kalanick's adversary, he might be arguing for more real-world data collection. Tesla famously equipped most of their cars with more sensors than were required at the time of delivery, using the data to drive development of the Autopilot function that was later added to the cars.

[PinguTS]

Please, remember that the original technology of the so-called Autopilot was bought from MobilEye. Please remember, that MobilEye is a long-time player in this area and have deployed their technology first with BMW and later with Volvo way before it was used in Tesla. Tesla only activated features that were not ready for primetime. That was the reason why MobilEye backed off from the partnership with Tesla.

Btw. all this of collecting sensor data and improving on this, is an original idea of MobilEye that they presented at conferences before the so-called Autopilot was available from Tesla.

[PinkMilkshake]

Agile methodology applied to safety-critical systems engineering, god help us.

[pjc50]

"Move fast and break things. Also break people, but if they're homeless or ride bicycles nobody cares"

[cmsj]

To quote a friend... "disrupting traditional life expectancy models", or to quote Webshit Weekly... "A/B testing vehicular manslaughter"

[IshKebab]

> We don't need redundant brakes & steering or a fancy new car; we need better software," wrote engineer Anthony Levandowski

He is clearly right about that. Human-driven cars are safety critical and already do fine without redundant brakes and steering. How many crashes are due to brake or steering failure? I'm guessing it's well under 10%.

Most human crashes are due to bad driving, and for computers it will be the same. I mean, even this fatal crash probably could have been prevented with better software. It's not like the brakes failed. They just weren't applied.

> To get to that better software faster we should deploy the first 1000 cars asap.

This is where he is totally mad.

[davidgould]

Human driven cars have redundant brakes. There is one brake assembly for each wheel. The hydraulic system is split into two separate circuits to guard against leaks. Additionally, cars are fitted with an "Emergency brake".

[iClaudiusX]

I get the feeling based on comments here that there is a severe lack of ethical and critical thinking among engineers and developers. I recognize that this is only a vocal minority but the constant mantra of "move fast and break things", where getting rich at any cost is seen as a virtue, has made me extremely disillusioned with this brand of startup culture. Doubly so when people are trading stock tips on how to profit from tragedy by supporting the worst actors in the field.

[JKCalhoun]

I'm not sure the "self-driving car industry" can survive these possible (inevitable?) future scenarios:

• Self-driving car kills child

• Hackers send self-driving car on wild ride

• Empty self-driving semi found in warehouse lot, GPS pirates make off with entire cargo

[archagon]

The dark, unspoken sentiment behind such comments is: "the end justifies the means". These programmers treat lives as currency for their vision of the future.

[KKKKkkkk1]

"Move fast and break things" is the motto of Facebook. It means that Facebook engineers are encouraged to make user-facing changes without red tape. If you are claiming that some self-driving car company has "move fast and break things" as their motto, then you are being willfully deceptive.

[PhasmaFelis]

Have you not noticed that "move fast and break things" is essentially how the entire startup space functions? When a social media company does it, it's merely annoying. Do it with dangerous machines, and people die.

[wdr1]

Is it better for self driving cars to have a flawless record & low adoption, or to have a 100x improvement over human drivers & broad adoption?

Creating the perfect self-driving car, with redundant systems, safety everything & so on will certainly help its safety records.

But it will also drive up the cost.

And put it out of reach for a lot of people.

If the goal is to save lives, the bar self-driving cars should be held to is what humans do driving today, not perfection.

[discodave]

From the article:

> But zooming out from the specifics of Herzberg's crash, the more fundamental point is this: conventional car crashes killed 37,461 in the United States in 2016, which works out to 1.18 deaths per 100 million miles driven. Uber announced that it had driven 2 million miles by December 2017 and is probably up to around 3 million miles today. If you do the math, that means that Uber's cars have killed people at roughly 25 times the rate of a typical human-driven car in the United States.

I don't think there's enough evidence to say that self-driving cars are as safe as humans.

[jon_richards]

To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans. Yes, you lose some life in the interim, but you accelerate the creation and adoption of something that saves much more life in the long run.

But that is human experimentation, something we as a culture generally agree is abhorrent.

[AlexandrB]

> To be fair, you have to keep in mind that deploying 1000 cars would quickly make self-driving cars safer than humans.

This seems unreasonably optimistic.

First, this particular crash is an egregious counter-example. The car doesn't even seem to slow down when it first sees the pedestrian's foot. Nor does it try to swerve. This is basic stuff for a human driver, never mind more complex avoidance and risk mitigation a human driver can perform.

Second, we've had years of training various AI content curation algorithms on social networks, videos, blogs, etc - and the most advanced AI and search company in the world still can't keep adult-oriented conspiracy videos off of Youtube Kids. And while you might counter that content is a human problem, driving is too! Dangerous driving situations happen at the periphery of traffic rules, where someone is doing something the drivers around him don't expect. I've seen people run solid red lights, drive the wrong way down a one-way street, pedestrians start crossing when their light turns red, etc. Short of creating special roads for autonomous cars only - how does an autonomous car deal with all of this successfully?

[bonesss]

> the most advanced AI and search company in the world still can't keep adult-oriented conspiracy videos off of Youtube Kids

In fairness here: Google could do this so easily by throwing people at the problem. A whitelisted set of content producers could do this, with people dedicated to whitelist curation. Google can keep them off, and quite readily. Keeping that off automatically and cheaply is the issue...

On that front: automated content recognition that can parse the nuances between regular claymation Elsa and spooky claymation Elsa who talks about jamming things in her "happy spot" is AI-hard. These are not random videos, they are content tailor made and adapted to pass whatever filters are in place.

For data scientists there is a massive gap between working with concrete sensor data that can be managed reasonably, and undefined philosophical/moral/sexual boundaries in dirty noisy counter-programmed video content... One replaces our eyes and ears and reflexes, the other seeks to replace our fabulous brains.

[culturestate]

> First, this particular crash is an egregious counter-example. The car doesn't even seem to slow down when it first sees the pedestrian's foot. Nor does it try to swerve. This is basic stuff for a human driver, never mind more complex avoidance and risk mitigation a human driver can perform.

I'm not sure which side of this I fall on just yet, but something strikes me here: you're assuming a human driver with no impairment (e.g. eyesight or fatigue) paying complete attention to what they're doing. We know that this isn's always the case (and could even be a minority!), so this doesn't seem to be a great argument.

[sunir]

You shouldn’t be driving so fast that stopping distance is longer than visibility.

It is irrelevant what average humans do. That is the current rule set for all drivers.

[blt]

No. Machine learning is not magic. Machine learning has nothing to say when the objective function is not well understood.

[fourthark]

This is assuming that the learning actually happens.

[gambiting]

Absolutely not. The cars should launch perfect, not just as good as humans. If you have any other human operated machine and replace it with an automatic one that sometimes kill humans, does it matter if it does it less often than the human operated one? Absolutely not - a machine cannot have an operating mode where death is possible. Look at Therac radiotherapy machines - they surely saved hundreds if not thousands of lives from cancer - but they had an operating mode that would kill the patient. Does it matter than a manually operated radiotherapy machine would most likely kill more people due to operator errors? Again - absolutely not.