[tshaddox]

> I feel like part of the problem with the kind of autopilot crashes you describe here is how inexplicable they are to humans.

I don't see why these are inexplicable to humans. It's certainly no more difficult to explain than, say, a (non-adaptive) cruise control in a car from 2000 doing the same thing.

> Whilst humans can be dangerous drivers, the incidents they cause generally have a narrative sequence of events that are comprehensible to us -- for instance, driver was distracted, or visibility was poor.

But that is arguably a sufficient explanation for these Tesla crashes as well. The driver being distracted or inattentive or unable to see clearly is a requirement for all of these Tesla crashes, as far as I know.

[icelandicmoss]

Perhaps 'unintuitive' is a better word to convey what I mean -- as in, there isn't an easily understandable (non-technical) narrative chain of events, there's just 'opaque box malfunctioned'. The cruise-control example you give feels a bit different, as CC doesn't claim to include automated collision avoidance, whereas something labelled 'autopilot' does.