The utilitarian argument doesn't require that they be better than every human driver in every condition -- just that they be better enough against the average situation that outcomes are net better.
I don't think that's right, because human drivers are exceedingly good at handling the average situation. If they weren't, you'd have accidents literally all the time. The average situation is accident-free. How much can you improve on that?
The utilitarian argument for driverless cars needs to be that driverless cars are better at handling the edge cases than humans, because that's when accidents happen.
I think you’re confusing “better” and merely “good,” as well as “average” and “mode.”
The average situation is maybe 0.99 accident-free (made up number for illustration). You improve on that by being 0.995 accident-free, which is better.
I agree that “safer than some human drivers in some conditions” is insufficient.
What I'm saying is that you can't design a car to handle the average driving situation and expect to really put a dent in the accident rate, because accidents are highly correlated to particular situations. For instance, if all of a town's accidents happen in a highly foggy area, but your driverless car cannot handle the fog because it's been designed to handle the average situation (not fog), then how will it reduce accidents? I would think that to reduce accidents it would have to be designed to work in the exceptional case (fog).
Maybe I'm confusing the idea of average and mode, so if I am an example of what you have in mind for an average situation would help.
The utilitarian argument for driverless cars needs to be that driverless cars are better at handling the edge cases than humans, because that's when accidents happen.