[rosser]

Nope. That's not what they're testing. They're testing whether or not the thing even works — which necessarily includes "doesn't kill people", sure. But that's not the specific thing being measured; it's a consequence of the system not being ready yet.

Your argument is more like, "We're testing this new chemotherapy agent to see how many people it kills", versus, "We're testing this new chemotherapy agent for safety and efficacy" (phase I and II, respectively). We aren't specifically screening for deadliness; we're trying to determine the therapeutic index of the agent, and get a baseline on its side effects (which, yes, may include death).

It's a subtle distinction, but it's a critical one.

All that aside, it's fundamentally disanalogous, anyway, because the populations in chemo clinical trials are already sick, and have given informed consent. Dude walking down the street getting smoked by a Tesla that thought he was a wall, not so much.