[I_am_uncreative]

This is such a stupid take. Even if there was a 5% chance of super-human level AGI destroying the world, we're not going to reduce that risk by restricting general computation. If it is possible to create AGI, it will happen, regardless of what the law stipulates.

This is unlike pseudoephedrine or nuclear weapons in that literally anyone with a computer could potentially create it.

[coryfklein]

Yes, it is in fact going to happen no matter what. So how do we ensure that the very first super-human AGI is even remotely interested in sharing the planet with the human race?

It is not by ensuring that literally anyone with a computer has an equal chance at creating it.