[laratied]

I would add to your amazing list that we are really good at denial as a coping mechanism with change.

I am not a fan of the concept of AGI though. This means so many different things to people that it seems pointless to debate something when most likely we are not talking about the same thing. François Chollet has said that he believes all intelligence is specialized intelligence. From that perspective, whatever people mean by AGI, we are already there in the world of art.

The doomer argument though is coming from defending our highly affluent and privileged life as we sit at the top of 7.8 billion people when it comes to wealth and lifestyle. It would have been better for the priest class too if the printing press had been shut down at the start. Of course, it is better for my friends and I to live in a society that we can read while most of society is illiterate but it is not better for society and humanity as a whole. The printing press was an apocalyptic development for the priest class in the same way all of this is an apocalyptic development for the "digital nomad". An apocalyptic development for the US nerd that makes 2X the median salary working 15 hours a week in between posting on here and their social media.

To extend this out to humanity as a whole though is such bullshit. Humanity will benefit enormously from this huge increase in the availability of intelligence.

Smart people are just in denial that their monopoly on higher than average intelligence is over. US devs kids born in 2023 aren't going to make 2x the median US salary while living in a poorer country with 5X less the GDP per capita. To say this is the end of the world though is simply an egocentric view of things.

[pmoriarty]

"Humanity will benefit enormously from this huge increase in the availability of intelligence."

It's a near certainty that AI will be used to create more effective/destructive weapons (if it hasn't already), and will likely be used by terrorists, scammers, and others who wish to harm humans in some way.

As this technology becomes more powerful, easier, and cheaper to use, all sorts of harmful uses of it will be made. The effectiveness and scale of this harm will also increase.

And that's all before even considering what will happen if/when AI's become truly intelligent, self-motivating, indepent, and self-aware.

The jury is still out on whether the net harm will out weigh the net benefit, and if humanity will survive something that might be analogous to neanderthals encountering homo sapiens.

[Henk0]

Yes, great point

So many of the people who opine about AI, its trajectory, and its possible effects on society, have latched on to one or two possible effects - like it overtaking jobs, or massively increasing misinformation. These are both very valid concerns, but they're only a tiny part of the big picture

The thinker who I perceive as having the best holistic (in the non-wooey sense of the word) understanding of how the rapid development of AI will affect this and a number of other social and existential risks is Daniel Schmachtenberger. He lays it out well in this episode of the Theories of Everything Podcast: https://www.youtube.com/watch?v=g7WtcTATa2U&t=2373s

Highly recommend watching it, even if it's long. Some main points though: - AI will increase the rate of development of every other technology it is applied to - In fields like biotech, this can lead to cancer cures, but also to increasingly dangerous bioweapons - Our current economic system is based on exponential economic growth in a limited resource world. AI applied in the service of profit will amplify this, leading us increasingly fast towards a number of tipping points. Of course, AI can also help steer us away from that path, but that is not the natural attractor - Game theoretic multipolar traps (aka Moloch) incentivise arms races and races to the bottom just like we see now. Those who are willing to move fast and break things have an advantage in these dynamics vs. those who prefer to move slowly and carefully - Cheaper and more efficient AI models will lead to increasing decentralisation of the technology, making it very hard to control - unlike current weapons of mass destruction

List goes on, but Daniel makes a much better case. Again, I would love to hear a good critique of his thinking, but haven't come across one yet

[pmoriarty]

Also see this interview[1] with Robert Miles.

I really hope these doomsayers are wrong, but my suspicion is the risk is real. Unfortunately, I'm not sure what can be done about it, as the profit and power these AI's promise is going to be near impossible for humanity to resist.

[1] - https://m.youtube.com/watch?v=kMLKbhY0ji0

[Henk0]

Yes, Robert Miles is great at explaining the problems of AI alignment, so I'll second the recommendation!

[digging]

> The doomer argument though is coming from defending our highly affluent and privileged life

It's not at all about that.

Even if "truly general" intelligence is impossible, that's irrelevant to the actual concerns about AI apocalypse. There are multiple theories about what failure looks like, but they essentially come down to a loss of control.

Now, obviously, that means something different for the owner class and for the worker class, which can be extrapolated to have global implications as well. But this isn't an issue of the owner class ceding control to the working class. It's an issue of the owner class ceding control to an alien. Maybe that alien makes things more egalitarian and prosperous. Or maybe it makes us extinct. Any and all possibilities are options for it as far as we know because it is fundamentally an inhuman (= alien) intelligence. We can't understand it even as well as we understand humans and human organizations (that is, not very well), let alone control it as well as we do humans and human organizations (that is, not enough to prevent self-inflicted climate apocalypse).

Basically, we're opening a box with a random magical spell inside it and deciding that we'll just have to live with whatever the effects of that spell are. I'm not for the status quo, but AI is just mind-bogglingly dangerous, and I think that's why there are so many wrong arguments against its danger. We literally cannot comprehend an intelligence greater than our own.

[NumberWangMan]

Nitpick: I think we can comprehend an intelligence greater than our own, up to some point, but that's different from being able to predict its actions.

And we could contain an intelligence greater than our own, up to a point. But if there are a lot of incentives not to, because letting that intelligence act on the world gains the "handler" money/power, then once there's one, there will likely be many, many more.

[pixl97]

> Humanity will benefit enormously from this huge increase in the availability of intelligence.

I know corporations will, but Moloch doesn't necessarily represent humanity.