[snaking0776]

That’s interesting. I commented something about this elsewhere but to me part of the exponential argument that loses me though is that it can often seem like a way to distract from issues that already exist which we should be working to fix. Things like autonomous weapons or mass surveillance are already here and rather terrifying and I would hope that we would dedicate our time to fixing those rather than having industry leaders focus so much on hypotheticals. While I guess the hypothetical scenario could be so bad that we must focus on it, I imagine a world which can’t come up with a way to spread wealth more equally or prevent mass proliferation of surveillance technology through profit seeking behavior will not be able to handle a digital super intelligence. So I keep coming back to the question: why is all I hear these industry leaders talking about is the threat of extinction? Maybe it’s just news coverage but I would love to see a leading lab release research on the health effects of subaudible sound in datacenters or other immediately present issues which would build good will towards these further out concerns.

[hollerith]

>why is all I hear these industry leaders talking about is the threat of extinction? . . . I would love to see a leading lab release research on the health effects of subaudible sound in datacenters

It is straightforward for industry leaders to avoid living near data centers, but there's no way for them to insulate themselves from the extinction threat -- no way short of somehow eliminating the danger for everybody, which seems quite hard to do. Since industry leaders are as self-centered as everyone else, the extinction threat is what they think about.

Also, you describe the extinction threat as "further out". A lot of us think there is already some small amount of AI extinction risk being incurred every day. I.e., we think the period of danger has already begun.

[snaking0776]

I see. I wonder how this works out in terms of risk/reward. I suppose if you take extinction as -infinite cost than it would be the only issue worth thinking about. Where I think this line of thinking gets challenging is when you need to take in terms of a counter factual. A lot of these were already risks prior to AI (bioweapons, nukes, etc) so what’s the marginal increase in probability as a result of AI I guess is the question which matters. I could get more around this way of framing it than saying that AI itself is the problem. It’s just the being more capable as a species increases risks. I think a lot of these pushback comes from the fact that it’s often the CEO who stands to gain huge by saying his tool is going to end the world so we need public buyin to supporting it. If instead it was just framed as “general technological advancement” is dangerous but potentially worthwhile I think more would be on board.

[hollerith]

I'm doubtful of this idea that the reason the CEO of Anthropic says that AI could end the world is because he "stands to gain huge" by saying it.

If he is willing to lie to give his corporation some advantage, he couldn't come up with a lie that would sound less absurd and outlandish to the average decision-maker, who doesn't have much time to learn about this particular technology?

It is more likely in my eyes that he says it because he genuinely worries about AI extinction risk -- like many people do who've studied the technology for a long time.