[ajb]

I appreciate the sentiment, but this is exactly like the expectation that people can be responsible for intervening when self-driving or driver-assistance goes wrong. Human brains are strongly driven to conserve energy. If nothing seems to be happening - when errors become less and less frequent - the more difficult it becomes to guarantee intervention, and the less practiced the human will be at doing so.

I have written factory tests, in which I injected errors to make sure that the factory workers didn't develop "click next" syndrome and actually noticed errors. That's what you'd have to do. It's hard to get an organisation to stick to that, when they add up the time they're paying for in detecting fake errors.

[recursivecaveat]

Yeah, suppose a self-driving car is say twice as likely to crash as the average human (which could already be considered unfair since that includes like drunk humans and people who should've already lost their license). That means to monitor it's driving you will need to wait 25,000 hours until an accident occurs. That's ~ 12 straight years of a 9-5 spent sitting, doing absolutely nothing, just watching in a state of theoretical vigilance.