[JohnFen]

> The point is that if there is even a 0.01% chance that AGI could threaten the existence of humanity then it's worth spending time and thought addressing it.

The disconnect I see is that if there is even a 0.01% chance that is could threaten the existence of humanity, then why in the world would anyone actively pursue making it happen before figuring out a way to mitigate that risk?

[6gvONxR4sf7o]

I think it’s arms race logic. “If we don’t pursue it before figuring out risk, worse people will pursue it without figuring out the risk.”

[JohnFen]

That's not logical, though. Assuming that "worse people" will develop it anyway, wouldn't it make more sense to let them, and focus entirely on how to mitigate the risk instead?

[6gvONxR4sf7o]

I mean, yeah. I phrased it in a weak way intentionally.