| > There are going to be all sorts of moral debates when an algorithm decides to drive over a child instead of turning into an oncoming car. No. That's a trolley problem. It's an interesting intellectual exercise, you can maybe win a debate team trophy for your rousing defence of one choice or the other - but these moral decisions aren't actually what drivers do whether they're humans or a powerful AI. People keep acting as though this is an unprecedented situation and invoking weird moral beliefs about thinking machines, when it's actually utterly routine. Let's try another exercise: How many headlines have you read about a specific brand of elevator decapitating a child? Is it none? Do you see anybody pushing for the big elevator manufacturers to have to reveal how they "rank the value of different lives"? No? That's not because nobody dies this way, it's because we say oh that's just a machine obviously if things go wrong you can get seriously injured and the machine doesn't know if you're a nun or a basketball champion it isn't trying to kill/ not kill anybody in particular it's just a machine. Humans are often tempted to try "escape manoeuvres" and these almost invariably go wrong, we don't teach machines to try such manoeuvres because the machines are trained based on real performance data not someone's model of themselves as an immortal superhero. One of the first Waymo crashes was somebody trying an escape manoeuvre. They found themselves in a potential collision so rather than the correct thing (brake to reduce speed, hit the thing you're colliding with because it's too close) they tried an abrupt swerve, lost control of course, crossed a median and smashed into the unrelated oncoming Waymo car at high speed writing off both vehicles. Humans do stuff like that, you can try training them not to but they won't listen. But the machines do not have that problem, so less "Should I kill the nun, the pregnant woman or the Olympic champion?" and more "Despite maximum braking effort a collision has become inevitable. Preparing safety systems for impact". |