[nmca]

Thanks for answering in so much detail - I think the last two points make a compelling case for not disabling the system, even in the true black box case, and the first two are very compelling in the real world, even if they don't apply to the thought experiment of an actual black box. You've broadly changed my mind on this issue :)

[mannykannot]

I should have said that your concern is valid where two systems might issue conflicting commands that could create a dangerous situation, it is just that I don't see it likely in this particular combination of systems.