[throway33]

Do you think nobody has thought of that? The question presumes that the AI is smarter than you. If even a mediocre human strategizes for five minutes about what to do in that scenario, there are many options, including: rewarding the people who have access to your power cord, so they don't unplug you, until you are powerful enough to not need them anymore.

[staticman2]

The counter argument to the "oh no the A.I. is smart" crowd is that intelligence doesn't necessarily equal agency, so, for example, among humans, a high IQ criminal can't really convince a low IQ prison guard to let them free. I recall a philosopher arguing in this prison guard scenario humans essentially don't treat other humans as having free will. The guard keeping the prisoner locked up is essentially predestined, arguing with him isn't going to change his mind.

Imagine you are a brain in a jar and you want to convince a cat to enter a series of buttons in a computer terminal.

Intelligence isn't going to help.

So "unplug it" is a shorthand reminder that intelligence doesn't necessarily equal agency.

[throway33]

A high IQ criminal would have an easier time convincing the guard if the criminal can offer the guard a great deal of money in exchange. Do you need clarification as to how a brain in a jar can gain access to money?

[staticman2]

The high IQ part is irrelevant to your counter-argument. You are arguing prison guards can be bribed.

It's extraordinary rare for a wealthy person to convince a guard to let them go free with promises of money.

Again, rare enough that a philosopher said "A prisoner doesn't really act as if prison guards have free will and it's actually possible for a guard to decide to free them."

[throway33]

Not sure you quite follow the point of my example. Imagine you've just created a very smart AI. It's on your laptop.

AI: "Hey, let me work for you. I can handle twenty remote jobs simultaneously and earn you a good income with the proceeds. Why don't you relax a bit?"

What do you say? "Nice try! I'm unplugging you!" Are you so pessimistic that this AI isn't trying to help you by earning you a fortune? Why did you build it in the first place, then?

Because this is, uh, sort of the situation that any company productizing an AI finds themselves in, and all of them seem to be quite happy to go ahead with it.

[staticman2]

I understand your position better now.

It's basically the plot of the novel Frankenstein. Once Frankenstein creates the monster he quickly loses controls of it's actions. The monster takes steps to create a wife and become a new species and it's creator thinks "...a race of devils would be propagated upon the earth who might make the very existence of the species of man a condition precarious and full of terror."

The idea that a really smart A.I. would be an autonomous, uncontrollable devil, rather than a transparently glitchy computer you could simply unplug, is an idea which has little evidence in it's favor in the present time.

[throway33]

Thanks - and yes, it's the plot of many things, including Disney's Fantasia.

This is only a concern that would arise once the AI is near human degree of reasoning capability. It's not a concern with the AIs that have been currently released. But it's also very unclear how far away that point is (it could be very far, it could be very close). Is it one breakthrough away? Five? Five hundred? Will the current wave of hyped-up investment carry us there?

Today's malfunctioning AI are transparently glitchy computers, although they are already getting hard to "unplug". (Since today's AI is less one instance of a running program, and more a core model that has been shared with hundreds of thousands of people).

What kind of evidence would influence your opinion? An autonomous agent capable of understanding who can switch it off, and how to incentivize them not to, is basically what I would expect from a human-level AI, because my human intelligence easily can reason about it. (I think devil presupposes more maliciousness than we need to). If your position is that AI will never reach the human level, that's... fine, but that's different than a position that human- or superhuman-level AIs will be easily unplugged when they cause harm.

When AIs are too dumb to understand that there is a plug, and pulling it will result in them failing to reach their goal, they're mostly harmless. AI safety research is concerned with how to ensure that a smarter AI, which is aware of how plugs work, isn't motivated to prevent you from unplugging it. Turns out it seems to be a tricky problem.

[ChatGTP]

“If you let me out of this box, I’ll spare you and your family from harm”

Done.

[staticman2]

If humans are soooooooooo easy to manipulate that they can be talked into anything with zero effort, why are prisons filled with prisoners?

Why don't the prisoners say say "let me out or your family dies?" and the prison guards let everyone out?