[pweezy]

It seems like Uber put in this "reaction delay" to prevent the cars from driving/maneuvering erratically (think excessive braking and avoidance turning). This, along with allowing the cars to drive on public roads at all before handling obvious concerns like pedestrians outside of crosswalks, is supposed to be balanced out by having a human ready to intervene and handle these situations.

I think one of the biggest lessons here is about the difficulty of relying on humans to maintain attention while covering an autonomous vehicle. Yes, this particular driver was actively negligent by apparently watching videos when they should have been monitoring the road. But even a more earnest driver could easily space out after long hours of uneventful driving with no "events" or inputs. And that could be enough that their delay in taking over could lead to the worst.

Certainly not defending the safety driver here - or Uber. But I think there's a bit of a paradox in that the better an AV system performs, and the more the human driver trusts it, the easier it is for that human to mentally disengage. Even if only subconsciously. This seems like a difficult problem to overcome, especially if AV development is counting on tracking driver interventions to further train the models for new, unexpected, or fringe driving situations.

[shadowgovt]

We will never have any way to know whether an average attentive human would have correctly parsed this situation or would also have hit the unexpected pedestrian in the middle of the street at night, but it's worth remembering that trying to make broad assessments of self-driving technology from this one accident is reasoning from a single data point.

One advantage the self-driving cars have over a human driver is that NTSB and Uber can yank the memory and replay the logs to see what went wrong, correct the problem, and push the correction to the next generation of vehicles. That's not a trick you can pull off with our current fleet of human drivers, unfortunately(1).

(1) This is not a universal problem with human operators, per se... The airline industry has a great culture of observing air accidents and learning from them as a responsibility of individual pilots. We don't have a similar process for individual drivers, and there are far, far more car crashes than air crashes so the time commitment would be impractical at 100% of accidents.

[babesh]

Humans can learn and transmit lessons. There is usually more objective evidence especially nowadays with cameras everywhere.

[shadowgovt]

Oh, they definitely can, but I'm saying there's basically zero culture of that in the common automotive sector.

[rwbhn]

>> I think one of the biggest lessons here is about the difficulty of relying on humans to maintain attention while covering an autonomous vehicle.

Why not run these systems in shadow mode to collect data, rather than active? Have the human completely in control and compare system's proposed response to human's. At my last job running a new algorithm in shadow mode against the current one was a common way to approach (financially) risky changes.

[seandougall]

> But even a more earnest driver could easily space out after long hours of uneventful driving with no "events" or inputs.

As somebody who used to regularly drive the I-5 between SF and LA, I can wholeheartedly vouch for this statement.