[Freeaqingme]

I think there won't be a big bang where we go from no self driving to fully self driving vehicles at once. It will be many small changes, that taken together will slowly transition towards full self driving cars. Since quite some years we've had cruise control, now there's adaptive cruise control. Then cruise control became able to overtake other vehicles. Next up, they may be able to take a certain exit based on satnav, etc.

It's hard to say when, but chances are that at some point in time we'll simply have FSD, while nobody realized that's where we were headed because it was all marketed as small(ish) individual features.

[crote]

I do not believe this will happen.

What we are seeing right now is that humans are really bad at co-operating with machines. We are asked to pay full attention to everything around us and take over at a moment's notice, while doing nothing >99% of the time. Humans simply do not work like that.

Meanwhile the AI acts like a black box, giving very little information about its inner state to the human. It can fail at any moment, without clear prior warning. It will fail in situations which look completely normal to the human. It will fail in situations it seemingly has successfully driven hundreds of times before. The AI is making decisions, but the human does not know why it is making them - or even that it is making them at all.

Humans are really bad at judging an AI's performance. We have already seen people falling asleep behind the wheel of their self-driving car, and that is not going to stop happening.

Self-driving vehicles are to a large extent an all-or-nothing thing. Until we can fully rely on it, it will only make the driving experience worse. Let's just stick to adaptive cruise control for now.

[jgwil2]

This paradox is already evident in commercial flight: as automation has increased, flying has become safer, but pilots themselves understand less and less about the machines they operate, and when automation fails they are ill-prepared to get themselves out of messes.

[martindbp]

The solution is VR training, not reduced automation.

[Balgair]

I agree that this is unlikely.

I have a new model car with all the lane-assist and following whizz-bangs. I'll never use them. When the lane markers get challenging and difficult to understand is when the assistance turns off and reverts to the human. Typically about 1 second before disaster were to strike. If it were even 10 seconds beforehand, then I'd maybe try using it again. But the 1 second window is too short for my personal limits. The assistance is completely backwards to me. I don't need it when lane markings are clear and easy to understand, I need it when they are unclear. Like, I don't need a translator to Swahili for most of the day in Mombasa, but I do need one when going over legal documents.

[SideburnsOfDoom]

> It will be many small changes, that taken together will slowly transition towards full self driving cars.

The problem is that there is no smooth transition: there is a "valley of failure" in the middle of it, where there is enough automation that the human can disengage most of the time, but not good enough automation to avoid all errors, so the human operator has to step in to correct errors quickly, and doing do "cold" (i.e. from a disengaged state) is not good enough.

Dekker touches on this in "The Field Guide to Understanding 'Human Error'"

[2sk21]

Good point. I think it will get even worse as automation improves. Consider that problems beyond thew ability of the automated driving system are delegated to the human driver. As automation improves, the human driver will have to be even more alert to resume control.

[SideburnsOfDoom]

Yep, the sibling comment ( https://news.ycombinator.com/item?id=33370118 ) explains it well:

> humans are really bad at co-operating with machines. We are asked to pay full attention to everything around us and take over at a moment's notice, while doing nothing >99% of the time. Humans simply do not work like that.

I have been trying to find some refs.

here's a couple: https://www.skybrary.aero/articles/cockpit-automation-advant...

https://themobilityforum.net/2021/03/15/the-potential-pitfal...

Like most of the 'Human Error' research, it focuses on Aviation, but is worth a read anyway if you care about topics such as e.g. SRE and Incident response.

[harimau777]

It seems to me that the biggest obstacle to that scenario would be legal. At some point the law would need to change to allow the "driver" to be asleep/watching TV/etc. behind the wheel.

[hulitu]

The biggest obstacle is "if compiles, ship it" mentality. Just like when a human drives: cutting corners increases the risk of accidents.