[haswell]

If you were to apply this argument to the development of weapons, it’s clear that there is a threshold that is eventually reached that fundamentally alters the stakes. A point past which all prior assumptions about risk no longer apply.

It also seems very problematic to conclude anything meaningful about AI when realizing that a significant number of those examples are doomsday cults, the very definition of extremist positions.

I get far more concerned when serious people take these concerns seriously, and it’s telling that AI experts are at the forefront of raising these alarms.

And for what it’s worth, the world as many of those groups knew it has in fact ended. It’s just been replaced with what we see before us today. And for all of the technological advancement that didn’t end the world, the state of societies and political systems should be worrisome enough to make us pause and ask just how “ok” things really are.

I’m not an AI doomer, but also think we need to take these concerns seriously. We didn’t take the development of social networks seriously (and continue to fail to do so even with what we now know), and we’re arguably all worse off for it.

[hackermatic]

Although I think the existential risk of AI isn't a priority yet, this reminds me of a quote I heard for the first time yesterday night, from a draft script for 2001: A Space Odyssey[0]:

> There had been no deliberate or accidental use of nuclear weapons since World War II and some people felt secure in this knowledge. But to others, the situation seemed comparable to an airline with a perfect safety record; it showed admirable care and skill but no one expected it to last forever.

[0] https://movies.stackexchange.com/a/119598