[benlivengood]

There's no teeth in MAIM like MAD has; first strike with AGI simply wins, and so the most covert AGI program will win.

[BobbyJo]

> first strike with AGI simply wins

The question I come back to over and over again is: wins what?

[benlivengood]

> The question I come back to over and over again is: wins what?

everything, forever.

The not so glib answer is economic and military superiority on Earth, and so whatever values or goals the AGI pursues from that point on will have a pretty high chance of success. Growth seems like one of the universal things that life seeks, and so I predict expansion into space for solar power and resources for most goals or value systems.

[BobbyJo]

The reason I wonder this isn't because I don't understand the economic or political gravity of the technology. It's because:

1) AI as a software concept strikes me as something that will be very easily stolen and replicated.

2) AI that is both nefariously aligned and super intelligent sound like fire that is wet. A super intelligent AI will likely be so logical that the only control we can exercise over it is censorship.

3) What value is there to harness from anyone after AGI? CEOs and Presidents are liable to end up on the same chopping block as everyone else at that point.

[aleph_minus_one]

> > The question I come back to over and over again is: wins what?

> everything, forever.

The same was claimed for nuclear and thermonuclear wars by military strategists during the Cold War.

[benlivengood]

Thank goodness we never fought an all-out pre-MAD nuclear war to find out.

[bhouston]

There are resource limits.

[catigula]

Power aggregation.

First AI will likely kill a lot of people at the behest of humans.

If I were a world power with a functional AI, I would immediately launch a full scale attack on foreign governments infrastructure so they couldn't develop a competing AI.

[t-3]

Why is AI better at killing humans than humans? What exactly is the threat introduced by "intelligence on silicon" rather than "intelligence on meat"? Why would an AI be better at attacking computers than humans using automated tools to do the same thing?

[catigula]

I've described to you the threat vector that people with high positions in the government are talking about.

An intelligent machine oriented towards 24/7 cyber attack with even parity human intellect is clearly a massive threat, let alone a data center full of them.

[QuadmasterXLII]

Same rules as a knife fight: the loser is the dude who dies in the street, the winner is identified by dying in the ambulance

[itishappy]

I don't get how MAID isn't still MAD in disguise. If the US or China simply says "any strikes on our datacenters will be met with an ICBM in response" who's going to test that?

If the "first strike" is just an unfair economic and political advantage... How's that materially different than today's world?