I agree, I don't know how we could objectively measure this but I do not see anyone trying to measure this, just selling hopes that soon the AI will be better then humans.
We would need to create some tests and simulations, passing this tests will not imply that the systems are safe but failing them would prevent any broken attempt to be approved for testing.
IF the sensors would be standard maybe we could create fake inputs and test different situation in a simulation, have some basic unit tests, now I am afraid that this companies are just tweaking things, something that today works tomorrow after an update may not work.
Isn't this quite the issue with machine learning algos. There is no 'standard' as the machine is always learning and getting more data in to match against. Testing such a system would mean that you have to 'freeze' the learning portion, something many will not like to do.
I do not know how this systems work, but I assume is not a complex NN but instead is made from layers, with an expert system and the NN would be used only in some sections like identify the objects in an image.
Even if the NN evolve if it recognizes a bike in image X it should still recognize it after it learns more or is updated with new hardware that supports more neurons.
It's fairly easy to measure, fatalities per million km driven. My understanding is that the industry aims to achieve 10x lower fatality rate before releasing self driving cars.
That way of measuring is terrible, any student then can put his own system on streets, kill 100 people, then after 1 million km some agency decide he needs to try again next year.
We need a better measurement that would not involve killing people, at least as a lower bar before accepting this cars for testing.
However, that's horrific and implies that the engineers are using something of an evolutionary algorithm. Obviously, this does not cut the cheese.