[estearum]

Let's assume that Anthropic believes they're in an arms race to create a potentially dangerous technology, and they believe they're the best ones to win this race.

Unlike nuclear weapons, advancing in this arms race requires actually deploying the product over and over again. Deploying the product makes your advancements visible to your competitors.

It makes complete sense to try to limit the degree to which that's true.

[sobellian]

It's an interesting assumption. The idea behind this with nukes was that we'd like to nuke Germany before they could nuke us. Even after we defeated Germany, we nuked Japan even though they had no possibility of getting their own nukes.

The nuclear 'race' was based on the premise that the winner could use it to destroy all other racers (a faulty assumption, see the USSR among others). I will charitably assume Anthropic does not intend to literally destroy anyone and merely wants to become an AGI monopoly. But if AGI is so powerful, any monopoly would not be stable since the incentives for entry into the market are massive. Why would China stop developing AGI just because Anthropic has it?

[estearum]

Do you believe the current situation is more akin to the race to the first nukes, where no one could know for sure the other competitors were even racing...

or is it more similar to the Cold War, where there were obviously competitors engaged in the race?

And yes, agreed the equilibrium dynamics for AGI are very different (and far harder to predict) than nukes. That sounds like a good reason to be sure we get there first since presumably any potential advantage wouldn't go to the second or third runner-ups

[sobellian]

I can't really say I see a similarity to either the Manhattan Project or the Cold War. I don't see how one could apply either massive retaliation or MAD. These are private companies, they are not vested with the necessary authority to destroy anything. Even if they had it, they couldn't. You can't destroy China, they have 1.4B people, nukes, and a large part of the world's manufacturing. So multiple organizations want to do something first, that could be anything from nukes to railroads to lining up for communion wafers.

[estearum]

You think "arms race" is a dynamic that only applies to literal arms?

"Ability to literally destroy the other entity" is not a necessary or even typical feature of arms races.

[sobellian]

Well it's difficult to argue against something that was never specifically stated. If someone is able to state specifically how this is an arms race in any other way than that it's a race at all then I'm happy to have that conversation.

[estearum]

"Arms race" is the term used colloquially to describe the dynamic that emerges in "winner-take-all" markets.

It seems that the frontier labs believe they're participants in a winner-take-all market. Therefore they're in "an arms race."

Winner-take-all markets do not require that the winner literally destroys the losers, but only that the winner enjoys disproportionate returns compared to their actual superiority.

Whether or not this is actually true is TBD, but I think you're naive to think the frontier labs do not believe this to be true.