[qazwsxedchac]

> sober voices as Geoffrey Hinton sounding the alarm about super-smart AI

Can someone please explain to me what exactly the danger is / the dangers are of "super-intelligent" AI?

AIUI, an AI is a combination of hardware, software and parametrization. In broad terms, it exists as a black box which supplies to humans responses to token sets fed to it.

Even if an AI has the launch codes for ICBMs somewhere in its training data, it doesn't have an interface to the nearest missile silo to use them. It cannot commandeer the resources (hardware, space, cooling, electricity) it needs to operate, it is dependent on humans to supply those. So humans can pull the plug on it at any time.

Even if an AI were to become both sentient and nefarious, by what mechanism would it harm humans?

I'm genuinely looking for concrete examples of such a mechanism because I can't imagine any which humans couldn't trivially control or override.

[throw310822]

I used to wonder the same, and was dismissive of the idea of "superintelligence". Now I think the whole thing- superintelligence, alignment, etc-, can be simplified a lot. Imagine you could give to someone (a random human being) the gift of omnipotence. Would you trust anyone with this ultimate power? That's the whole problem.

[richk449]

> Even if an AI has the launch codes for ICBMs somewhere in its training data, it doesn't have an interface to the nearest missile silo to use them. It cannot commandeer the resources (hardware, space, cooling, electricity) it needs to operate, it is dependent on humans to supply those.

How do you know the AI doesn’t have an interface to the middle silo?

Are there no military systems that are connected to the internet?

Is the power grid connected to the internet? How about our water filtration system?

I’m not worried that gpt4 is going to start launching missiles, but I also don’t understand where the certainty comes from that it isn’t possible for an internet connected AI to launch missiles.

[antifa]

> I'm not worried that gpt4 is going to start launching missiles

Cool SciFi showerthought:

An AI will probably know that Stuxnet happened.

https://en.m.wikipedia.org/wiki/Stuxnet

An AI could trick multiple human actors into unknowingly assembling all the pieces together for a nuclear launch.

[arisAlexis]

https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/

[qazwsxedchac]

Thank you for the reference. Its author defines a "rogue AI" as:

"an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere"

and explains that it would also need to be goal-directed in a way which would be at odds with human wellbeing.

Stipulating all that, what is still missing is an explanation of the mechanism by which an AI, rogue or otherwise, could do harm. How is it supposed to affect the world outside its computing substrate?

Absent humans making available the interfaces and resources to do so, it can't. The referenced article includes an example of a genocidal human doing exactly that, and using an AI as a force multiplier. That, as the trope goes, is a social problem, not a technical problem, and it needs a social solution, not a technical one.

Each of the other examples in the referenced article (military AI going rogue, wireheading, amoral corporate AIs manipulating humans) require AIs interfacing with the physical world outside their computing substrate or with the biosphere. Again, because these scenarios remain dependent on humans making available such interfaces, I fail to see how a hypothesized "rogue" AI could achieve any autonomy to do serious damage.

I see this panic about rogue AIs as well-intentioned but misguided, and perhaps exploited by folks who would like to control / diminish / force licensing of general purpose computing.

[n4r9]

> How is it supposed to affect the world outside its computing substrate? Absent humans making available the interfaces and resources to do so, it can't.

True, but what more do you need than the ability to send web requests to arbitrary domains, and receive the responses?

[joseda-hg]

Kinda like ChatGPT hired a person to just pass captchas, or Sydney reading news about it's own actions/interactions and therefore getting info it shouldn't have had, there's a lot of space for going out of the guardrails of not having a proper interface

Too many systems rely on people being unaware of exploits, but an AI would never forget something, or get bored or tired of trying, It doesn't need to be smarter than humans, just have enough persistence and attention to detail

[Filligree]

> How is it supposed to affect the world outside its computing substrate? Absent humans making available the interfaces and resources to do so, it can't.

This would be more reassuring if hooking it up to a Python prompt wasn't virtually the first thing people did.

[arisAlexis]

The article explains that too. But it's easy to think of millions of examples if you can hack everyone and persuade people to do stuff for you and you are smarter than them.