[slg]

>The benefits are quite low: if the autopilot had been on since inception between 20 and 100 lives would have been saved

This is only the case if you look at the current system as the finished product. The biggest benefit is that it gets us closer to a true self driving system. That would not only save millions of lives, but it would revolutionize logistics and economics of transportation which can in turn reshape society.

>The costs could be astronomical: a simultaneous failure (security, mistraining, date bug, whatever) could result in hundreds of thousands or millions of deaths.

I have no idea what scenarios you are imaging that could lead to "hundreds of thousands or millions of deaths." Almost every Autopilot death in the US makes national news. There is no way hundreds of people could die without there being some type of intervention in the system.