| The process is definitely vulnerable to magical thinking. I think it is possible to avoid, though, by asking if humans can be generally good at the task in question, if working through the implied interface restrictions, and then evaluating whether the required skills can be reflected in an available training data set. If either of those cannot be definitively answered, it’s probably not going to work. An interesting example here is the failure of self driving vehicles based on image sensors. My take is that most of the problems are because a significant fraction of the actual required training data is poorly represented in data that can be collected from driving experiences alone. As in: If you want a car to be able to drive safely around humans, you need to understand a lot about what humans do and think about. - then apply that same requirement to everything else that occasionally appears in the operational environment. To understand some traffic management strategies expressed in infrastructure, you’ll need to understand, to some degree, the goals of the traffic management strategy, aka “what were they thinking when they made this intersection?”. It’s not all stuff you can magically gather from dashcams. |
A model might potentially be able to understand those situations, but it would need a lot of highly task specific training data and it would never be clear if the training really covered all possible situations.
The other problem I see is that a lot of situations in traffic are really two-way communication, even if it is nonverbal and sometimes so implicit we don't realize it. But usually pedestrians will also try to infer what the driver is thinking whether he saw them, etc. In those situations, a self-driving car is simply a fundamentally different kind of traffic participant and pedestrians will interact with it differently than they would with a normal car. That problem is independent of machine learning and seems much harder to solve to me.