[dTal]

>a mildly uncomfortable criteria to put into a computer algorithm

I suspect this is going to be a fundamental issue with AI. Far from some idealized 3 laws of robotics, AIs will need to behave like humans to fit in our society. And that will force us to confront the ways in which we don't follow our own rules - indeed can't follow our own rules, the rules being impractical but a convenient fiction to allow us feel better about ourselves.

[jefftk]

This is already happening, with driverless cars getting rear ended because they come to a complete stop at stop signs.

[emodendroket]

I would call this one pretty straightforward; a driver rear ending someone is almost always liable.

[jefftk]

The driverless car is definitely not liable, but the reason it's getting hit is because it's violating an expectation that people will bend the rules in this circumstance

[emodendroket]

I just don't think this is a great example because lots of people do come to a complete stop at stop signs, as they should.