[dota_fanatic]

One cannot predict what a smarter-than-themself agent will do ahead of time, if they could then they’re just as smart. Just as a dolphin cannot predict how a human will come up with novel and utterly overwhelming ways to farm them, you and this other poster cannot predict how an AGI will achieve dominance of its environment to achieve its goals, so your request is impossible to fulfill. Lee Sedol couldn’t “take us through it step by step” how AlphaGo would beat him in Go.

That aside, afaik most safety concerns arent around a bad human actor using AGI to dominate the planet, it’s around an AGI being misaligned to begin with, it cannot be controlled, we promptly lose everything after it manifests.

[nradov]

How ridiculous. This is just sci-fi bullshit without a shred of logic or scientific evidence. You would have just as much credibility claiming that we are at risk of an alien invasion. I mean I can prove that it's impossible.

[dota_fanatic]

No evidence? There are two examples of evidence in my post: the history of homo sapiens vs other intelligent mammals and AlphaGo vs humans in Go.

Alien invasion? What in the non sequitur are you going on about? And apparently you have a proof against the possibility of AI misalignment? Pack it up everyone, nradov has the entire field of AGI alignment nailed. And a proof of the non-existence of aliens, never-mind very smart people have put out a mathematical model which seems to fit the evidence quite well.*

Apologies for the snark, but your reply was rather abrasive.

* https://grabbyaliens.com/

[Apocryphon]

How does the ability to learn how to play go translate to being able to defeat human beings in the social arena.

[dota_fanatic]

It doesn’t. I was discussing the inability of a weaker intelligence to predict ahead of time what a stronger intelligence will do to achieve its goals. AlphaGo is an example of that in the specific domain of Go gameplay. A general intelligence is generalized, which is why humans can outcompete other agents in so many different domains, just as an AGI could outcompete us in any domain given the opportunity to grow in power.

[x86x87]

You challenge them to a game of go and the loser needs to relinquish control, duh /s

[datameta]

Here's an example: our dear Czar uses the AGI to discover a 16 shell company chain in order to get resources and tech that are under sanctions. Or, he has it use real-world conditions from large batches or real-time uncurated battlefield data and intelligence reports to run simulation after simulation to inform his generals where it is likeliest that the next counterattack would be concentrated.

[blackoil]

Lawyers and narrow systems are much more efficient at it.