These stories have played out before. Maybe the best example is Oppenheimer vs Edward Teller. There will always be an Edward Teller. Yet we still haven't blown ourselves up. If you ask why and how you will find hope.
Lots of people worked at making it look like "dumb luck". That point gets missed. What they did is not well understood by most people who cry about the dangers of the next pandoras box.
I don't know how much credence to give it, however it's a fascinating thought experiment, especially the "Type-1" vulnerabilities.
[1] https://nickbostrom.com/papers/vulnerable.pdf