Here's where it will be a showstopper for me; many many times the uber map has fucked up directions and so I tell the driver the route I want to take, or to pass a particularly slow driver in front, in addition I sometimes change my destination/add a stop.
Touch screen panel, make options available to adjust the route taken? Doesn't seem like a hard problem to solve, especially compared to all the other problems that need to be solved to make autonomous cars a reality.
Changing the "messed up" directions is a strawman in your head. Sure, in the beginning those complaints would be more prevalent, but as the "rate your ride" complaints get sent up to Google, I'd imagine that there will be an army of people re-training the AI to minimize these kinds of inefficient glitches to the point that they rarely if ever happen. Google does this today with an army of yellow-badge employees who fix Google Maps.
Have a big touch screen in the car that shows the destination and the route the car is going to take? And allow you to drag/drop to alter the route similar to Google Maps?