[InclinedPlane]

There are several parts here: the hardware a self-driving car uses to "see", the software a self-driving car uses to process sensor input into a representation of the vehicle's surroundings, and then the software that makes decisions and issues commands to the car to actually "drive" it.

You have a car with all of this running but a human driving while you drive around for hundreds of millions of vehicle miles. You then review the data for these trips and use it to assess both the ability of the hardware/software to maintain a meaningful degree of "situational awareness" as well as the reasonableness of the software to control the car if it had actually been doing so. From this you can determine how good of a job the system is doing and build a fairly good idea of how much you can actually trust the car to drive on its own. Then you can let the automation drive the car with a human behind the wheel and actually paying attention. From that you can further improve your assessment of how well the car operates under realistic driving conditions.

However, if the human isn't paying attention then you potentially significantly increase risk. Especially if you short-changed the previous step of monitoring the automation's performance while "side seat driving".

In the case of, say, Waymo, they've done a fairly good job here because they've been careful and thorough in each step. In the case of Uber, as is reflective of their corporate culture, they have rushed ahead and taken on a lot more risk than they should have, in this case putting bystanders in harm's way.