That's a good point, although what I'm more concerned with is industry pushback against third-party self-driving upgrades, similar to how ECU tampering voids the warranty of most cars.
Autonomous software is a lot more complex than the software the ECU uses (which is still complex), so there may be a genuine argument against allowing people to modify cars not designed for autonomous driving. It definitely raises a really serious liability question.
Even with those systems working exactly as designed, there are some serious liability questions I don't think anyone has really answered yet.