[crote]

I do not believe this will happen.

What we are seeing right now is that humans are really bad at co-operating with machines. We are asked to pay full attention to everything around us and take over at a moment's notice, while doing nothing >99% of the time. Humans simply do not work like that.

Meanwhile the AI acts like a black box, giving very little information about its inner state to the human. It can fail at any moment, without clear prior warning. It will fail in situations which look completely normal to the human. It will fail in situations it seemingly has successfully driven hundreds of times before. The AI is making decisions, but the human does not know why it is making them - or even that it is making them at all.

Humans are really bad at judging an AI's performance. We have already seen people falling asleep behind the wheel of their self-driving car, and that is not going to stop happening.

Self-driving vehicles are to a large extent an all-or-nothing thing. Until we can fully rely on it, it will only make the driving experience worse. Let's just stick to adaptive cruise control for now.

[jgwil2]

This paradox is already evident in commercial flight: as automation has increased, flying has become safer, but pilots themselves understand less and less about the machines they operate, and when automation fails they are ill-prepared to get themselves out of messes.

[martindbp]

The solution is VR training, not reduced automation.

[Balgair]

I agree that this is unlikely.

I have a new model car with all the lane-assist and following whizz-bangs. I'll never use them. When the lane markers get challenging and difficult to understand is when the assistance turns off and reverts to the human. Typically about 1 second before disaster were to strike. If it were even 10 seconds beforehand, then I'd maybe try using it again. But the 1 second window is too short for my personal limits. The assistance is completely backwards to me. I don't need it when lane markings are clear and easy to understand, I need it when they are unclear. Like, I don't need a translator to Swahili for most of the day in Mombasa, but I do need one when going over legal documents.