[SpicyLemonZest]

I don’t really understand the question. We routinely worry about insane people with nuclear weapons; that’s why almost the entire world got together in 1968, and signed a treaty agreeing not to start any new nuclear weapons programs beyond the five that existed at the time. It’s not flawless, and four non-signatories have developed nuclear weapons since then, but I think any AI safety proponent would be more than satisfied with a similar agreement.