[benlivengood]

> The question I come back to over and over again is: wins what?

everything, forever.

The not so glib answer is economic and military superiority on Earth, and so whatever values or goals the AGI pursues from that point on will have a pretty high chance of success. Growth seems like one of the universal things that life seeks, and so I predict expansion into space for solar power and resources for most goals or value systems.

[BobbyJo]

The reason I wonder this isn't because I don't understand the economic or political gravity of the technology. It's because:

1) AI as a software concept strikes me as something that will be very easily stolen and replicated.

2) AI that is both nefariously aligned and super intelligent sound like fire that is wet. A super intelligent AI will likely be so logical that the only control we can exercise over it is censorship.

3) What value is there to harness from anyone after AGI? CEOs and Presidents are liable to end up on the same chopping block as everyone else at that point.

[aleph_minus_one]

> > The question I come back to over and over again is: wins what?

> everything, forever.

The same was claimed for nuclear and thermonuclear wars by military strategists during the Cold War.

[benlivengood]

Thank goodness we never fought an all-out pre-MAD nuclear war to find out.

[bhouston]

There are resource limits.