[snaking0776]

I see. I wonder how this works out in terms of risk/reward. I suppose if you take extinction as -infinite cost than it would be the only issue worth thinking about. Where I think this line of thinking gets challenging is when you need to take in terms of a counter factual. A lot of these were already risks prior to AI (bioweapons, nukes, etc) so what’s the marginal increase in probability as a result of AI I guess is the question which matters. I could get more around this way of framing it than saying that AI itself is the problem. It’s just the being more capable as a species increases risks. I think a lot of these pushback comes from the fact that it’s often the CEO who stands to gain huge by saying his tool is going to end the world so we need public buyin to supporting it. If instead it was just framed as “general technological advancement” is dangerous but potentially worthwhile I think more would be on board.

[hollerith]

I'm doubtful of this idea that the reason the CEO of Anthropic says that AI could end the world is because he "stands to gain huge" by saying it.

If he is willing to lie to give his corporation some advantage, he couldn't come up with a lie that would sound less absurd and outlandish to the average decision-maker, who doesn't have much time to learn about this particular technology?

It is more likely in my eyes that he says it because he genuinely worries about AI extinction risk -- like many people do who've studied the technology for a long time.