[collingreen]

There are several examples already of the modeling leading to systems that all incorrectly handled faults and pointed toward nuclear war as the correct next action. Each of these times _so far_ a human has gone against the strategic planning and operational competence you're talking about and decided personally to get more information before killing millions of people (and they were all correct so far!)

Diluting or delegating decision making to committees, processes, models, or AI all have essentially the same shape.

We can either appreciate how lucky we've been so far and actually learn from these near-doomsdays or we can choose to keep rolling the dice with our eyes covered.