[neilv]

Using this as an example of a huge alignment and priorities issue in "AI"...

Consider a kind of automobile crash for which driver of vehicle A is generally considered not-at-fault. For the sake of this discussion, let's say, a driver who gets rear-ended is generally considered not-at-fault (though I'm not sure that's the case -- just a simple example).

However, if you have an interest in the safety of occupants of vehicle A, who's at-fault isn't your top priority.

And there's never any occasion on which you do the math to say how you'd prefer your loved ones be possibly crippled or killed based on who would be considered at-fault.

OK, now suppose that you're an autonomous driving product developer. The decisions of driving behavior are different, and maybe your top priorities are to avoid bad PR and big payouts. So you'd have big incentive to take the at-fault PR and payout priorities into account when you're developing your autonomous driving sytem.

For example, your vehicle moving forward to strike something is probably very bad, for PR and payouts. But stopping abruptly, likely causing a vehicle behind to strike you, doesn't get as much of your attention, and/or doesn't weigh as strongly in your balancing.

Especially if the vehicle behind you is much smaller, like a motorcycle, so maybe your occupants don't even get whiplash, and the crushed motorcyclist would be considered at-fault (and generally motorcyclists don't get much public sympathy).

(The first examples I heard for autonomous driving were things like "There's an oncoming vehicle or obstruction, so do you choose to crash into it, probably killing/crippling the occupant of the vehicle you're driving, or to swerve into that pedestrian on the sidewalk, probably killing them. The PR calculus is different there, since consumers might secretly favor a system that prioritizes the safety of the buyer while increasing threats to others, without saying it out loud (see SUVs). But at the end of the day, the answer of how the autonmous vehicle drives, and who loses in driver decisions, might come down to the developer's very typical corporate priorities.)

[CobrastanJorji]

This feels like the "self-driving car engineers are taking the trolley problem into account" fallacy. There is absolutely no code in these things taking into account the laws of at-fault traffic accidents in their locale. The basic problem of "1. get where you're trying to go, and 2. don't crash into things" is already hard enough. There is no "dive onto the sidewalk to avoid an out of control bus" algorithm, nor is there a system to count the people on the bus and the sidewalk and compare them, nor is there a physics sim examining potential future spinal integrity.

You know what the car does if there's a motorcycle in front of it? It brakes. You know what it does if there's a weird bus coming across lanes towards it? It brakes. Driving normally is a hard enough problem. Nobody's designing these things to handle movie car chases.

[JumpCrisscross]

It’s a fine philosophical point, but far from practicality when we’re still trying to keep these cars from accidentally running into things.

The trolley problem was never an argument against mass transit.

> if the vehicle behind you is much smaller, like a motorcycle, so maybe your occupants don't even get whiplash, and the crushed motorcyclist would be considered at-fault

You’re describing tailgating. This isn’t even a trolley problem.

Jamming on the brakes for no reason is bad driving. Jamming on the brakes because something is in front of you is…that’s why we have brakes.

[dmitrygr]

> since consumers might secretly favor a system that prioritizes the safety of the buyer

Secretly? I am happy to go on record: I would not get into a car that would ever consider anyone's life more important than mine. I would not let anyone in my family do so either.

[neilv]

> Secretly? I am happy to go on record: I would not get into a car that would ever consider anyone's life more important than mine. I would not let anyone in my family do so either.

Consumers in general.

Of course HN will even have people openly say things approaching:

"In my value calculus, it is only rational to swerve the vehicle to plow through a sidewalk full of photogenic preschoolers on a field trip, to maximize my chance of survival, and increase the likelihood of my genetic material propagating and out-competing that of others. Nature has always selected for behavior such as this, and it is only right. It repels me that others do not think this way, or are too weak to admit it, because they are inferior in intellect and strength of will! Hence why I and my spawn shall be the rightful leaders over them!"

Vigorous gestures and armbands are optional.

My gut feel is that most people either don't think like that, or -- to the extent they do think a little like that -- they're selective about where and how they voice it.