[adleyjulian]

I think the EA community is very open about the uncertainties around the risks of AGI and the difficulty in addressing them. The point is that if there is even a 0.01% chance that AGI could threaten the existence of humanity then it's worth spending time and thought addressing it.

NASA has the Sentry program to look out for asteroids, with a budget of around $35 million, and the odds of a "big one" hitting in our life time is less than 1 in 10,000.

That said, the majority of EA spending is on global health and development to address immediate issues, on programs like Against Malaria.

[JohnFen]

> The point is that if there is even a 0.01% chance that AGI could threaten the existence of humanity then it's worth spending time and thought addressing it.

The disconnect I see is that if there is even a 0.01% chance that is could threaten the existence of humanity, then why in the world would anyone actively pursue making it happen before figuring out a way to mitigate that risk?

[6gvONxR4sf7o]

I think it’s arms race logic. “If we don’t pursue it before figuring out risk, worse people will pursue it without figuring out the risk.”

[JohnFen]

That's not logical, though. Assuming that "worse people" will develop it anyway, wouldn't it make more sense to let them, and focus entirely on how to mitigate the risk instead?

[6gvONxR4sf7o]

I mean, yeah. I phrased it in a weak way intentionally.