[drewm1980]

Engineers are going to break down the problem into many subsystems and test the heck out of them.

Maybe the system can still be globally optimized, though, as long as individual subsystems are still verifiably correctly trained. i.e. lane detection, pedestrian detection could share some of the same convolutional layers and still be tested separately.

My personal prediction is that all of this 2D convolutional network stuff will be extended to 3D within a few years. The front-end will do a full 3D scene reconstruction from first principles, and then some sort of 3D features will be learned on ~that data.