[yboris]

There's a lovely analogy I like: driving fast.

New technology is like putting more weight on the gas pedal. Until we learn to handle the current problems, accelerating isn't a guaranteed solution. We may be able to swerve out of the oncoming problem, but now we're going event faster.

We now have technology that can end human civilization (aka "existential risk"). What is the harm in going slower? Especially compared to the harm of going faster.

[antepodius]

It's a coordination problem. Sure, you could slow down, and then the chinese will just outcompete you/spawn a paperclip maximiser- or, in general, a less scrupulous competitor will.

You'd have to get all sides to agree on an actually effective treaty.