The USAF is able to deal with latency issues when controlling and targeting drones on the opposite side of the planet, though I do believe they use local people with line of sight connections to handle takeoffs and landings.
I don't see why something like their systems couldn't be applied here. Well, other than the fact that you'd need the resources (see: cash and massive satellite network) that the USAF has...
Reaction time requirements are probably much more forgiving for drones -- don't need to suddenly brake or change course due to the movements of other tightly packed drones around you during the daily commute
There is also the question of trust, one of the biggest non-technical challenge to overcome would be to convince general public to get into a car which doesn't have a driver. It would be similar to getting into a rollercoaster.
No control whatsoever of what might happen next despite every bit of your trip being pre-planned to the dot.
I don't see why something like their systems couldn't be applied here. Well, other than the fact that you'd need the resources (see: cash and massive satellite network) that the USAF has...