[MathYouF]

From my admittedly limited knowledge of how self driving policy models are implemented, I believe the main black box AI is determining the part

`if personInFrontOfCar:`

(using object detection, masking, 3D pose estimation, et/or cetera)

And actually defined policies on how to handle certain situations given that inference are possible, in the case specifically of self driving vehicles.

My assumptions come from when I'd asked someone at NeurIPS in 2019 about what reinforcement learning methods they use as they said "I don't think any self driving companies are using reinforcement learning, it is too risky". I don't mean to imply this is hard to find information either, reading a few papers would be all it takes to clear up the degree to which most self driving policies are "controllable" in the way you describe, or at least to what degree.

My main point is that I think it is the inferring of the environment (current state) rather than the chosen policy at each time step which is more of an error prone black box.