[mikeash]

The cars just need a high speed connection to the credit bureaus, then they can optimize for minimizing the sum of the FICO scores of those killed.

More seriously, I don't understand why this is always brought up in the context of autonomous cars. How often do crashes happen where killing people is inevitable but you get to choose who dies? How often will such crashes happen when autonomous vehicles are common? How far from the theoretical optimum will it be to just say "in an emergency, brake to a stop and steer to avoid obstacles"?

I'm pretty sure this is an edge case on the edge cases. If autonomous vehicles deliver on even 10% of their promise, safety will go way up. It doesn't have to be perfect.

[JoeAltmaier]

The sticky part is, the software has to be written now. After millions of miles of exercising that code, every path will be taken.

So what do you put in the "Run over the kid, or run into a wall (or oncoming traffic)?" decision point? Somebody has to die. Just braking until you hit the kid is a pretty crappy hack.

And when the kid gets hit, the software will be examined. And the comment that says "screw it, just hit the kid" will come to light.

[mikeash]

The software isn't going to say "screw it, just hit the kid." It's going to be more like what I said: brake to a stop, steer to avoid obstacles. If an obstacle must be hit, then default to going straight.

This is already going to be vastly superior to human drivers, who often don't bother to brake, or flinch and steer into oncoming traffic for no particularly good reason.

After we've cut down the 30,000/year death rate from car accidents by a couple of orders of magnitude, if we get start to get desperate about improving safety and have trouble figuring out how, then maybe we can start looking at rare and bizarre occurrences like these, if they ever actually happen.

Trying to work out the precise requirements for ethics in autonomous car crash response, when we don't have more than the vaguest idea of what kind of crashes they'll get into or what kind of responses might be available, is awfully premature.

[JoeAltmaier]

We have an immense corpus of traffic incident reports. Its disingenuous to suggest only 'the vaguest idea'.

And the Google car can already identify a person in the road. That's critical, at least for lawsuit purposes. "You mean, it knew there was a kid in the road, and it did nothing?!"

And lawsuits will happen, the first year. Nobody cares about the vast improvement to humanity; they care that a Google car kit their kid. There's where it all fails.

[mikeash]

Humans are extremely good at interpreting visual information, but have long reaction times and short attention spans.

Computers are poor at interpreting visual information, but can react essentially instantaneously and can pay full attention to all of their inputs indefinitely.

Nearly all crashes are due to those human failings of long reaction times and short attention spans. Even crashes due to equipment failure tend to be greatly exacerbated by those failings.

Autonomous car crashes are going to be due to unrecoverable equipment failures, software bugs, insufficient sensor data, and sudden unpredictable changes in the environment which can't be avoided even with instantaneous reactions.

I don't think it's strange to think that the nature of autonomous crashes will be quite different from the nature of human crashes. I'm certainly not stating that insincerely. I mean, fully a third of traffic fatalities currently happen because human drivers deliberately degrade their own senses and reactions before driving. Another big chunk are because of drivers deliberately not paying attention to driving.

The confluence of factors that need to come together to even create one of these ethically difficult scenarios is so unlikely that I wouldn't be surprised if it never comes up at all. If it does, it's going to be single digits per year. And what's the lawsuit going to look like? Alleging that some Google car should have killed a nun instead of the plaintiff's child? I don't think the law even supports this idea of "you should have taken action to kill someone else instead." Even if it did, it's not going to ruin the industry.

[JoeAltmaier]

Fair enough