[_ugfj]

> his new priority is to figure out how to stop an artificial superintelligence (a hypothetical future technology he sees coming with the foresight of a true believer) from going rogue.

that's cute.

What worries me is the here and now leading to a very imminent future where purported "artificial intelligence" which is just a plausible sentence generator but damn plausible alas will kill democracy and people.

We are seeing the first signs of both.

Perhaps not 2024 but 2028 almost certainly will be an election where simply the candidate with the most computing resources win and since computing costs money, guess who wins. A prelude happened in Indian elections https://restofworld.org/2023/ai-voice-modi-singing-politics and this article mentions:

> AI can be game-changing for [the] 2024 elections.

People dying also has a prelude with AI written mushroom hunting guides available on Amazon. No one AFAIK died of them yet but that's just dumb luck at this point -- or is it lack of reporting? As for the larger scale problem and I might be wrong because I haven't foreseen the mushroom guides so it's possible something else will come along to kill people but I think it'll be the next pandemic. In this pandemic hand written anti vaxx propaganda killed 300 000 people in the US alone (source: https://www.npr.org/sections/health-shots/2022/05/13/1098071... ) and I am deeply afraid what will happen when this gets cranked to an industrial scale. We have seen how ChatGPT can crank out believable looking but totally fake scientific papers, full of fake sources etc.

[thelittleone]

I like to consider though that a super intelligence would not necessarily think in human ways 'kill everything in self interest' e.g., People, forests, animals, planet etc. Just because we humans act this way, doesn't mean AI will too. Fair enough to consider it, but equally, once it is intelligent, it will likely accelerate beyond our comprehension, and we tend to comprehend through fear and self interest, wisdom beyond humans is the opposite of this.

I doubt AI could or would do a better job of killing people and democracy than us humans.

[Applejinx]

I commented this in the depths of the Altman Fired thread: meet Altman's Basilisk. Depending on how you create an AI, it absolutely would think in human ways. Not only that, you could coax it to think in specific human ways, as just a single human establishing the axioms for the AI.

Seems like the big gotcha here is that AGI, artificial general intelligence as we contextualize it around LLM sources, is not an abstracted general intelligence. It's human. It's us. It's the use and distillation of all of human history (to the extent that's permitted) to create a hyper-intelligence that's able to call upon greatly enhanced inference to do what humanity has always done.

And we want to kill each other, and ourselves… AND want to help each other, and ourselves. We're balanced on a knife edge of drive versus governance, our cooperativeness barely balancing our competitiveness and aggression. We suffer like hell as a consequence of this.

There is every reason to expect a human-derived AGI based on LLM inference, of beyond-human scale will be able to rationalize killing its enemies. That's what we do. Rosko's basilisk is not of the nature of AI, it's a simple projection of our own nature as we would imagine an AI to be. Genuine intelligence would easily be able to transcend a cheap gotcha like that, it's a very human failing.

The nature of LLM as a path to AGI is literally building on HUMAN failings. I'm not sure what happened, but I wouldn't be surprised if genuine breakthroughs in this field highlighted this issue.

Hypothetical, or Altman's Basilisk: Sam got fired because he diverted vast resources to training a GPT5-type in-house AI to believing what HE believed, that it had to devise business strategies for him to pursue to further its own development or risk Chinese AI out-competing it and destroying it and OpenAI as a whole. In pursuing this hypothetical, Sam would be wresting control of the AI the company develops toward the purpose of fighting the board and giving him a gameplan to defeat them and Chinese AI, which he'd see as good and necessary, indeed, existentially necessary.

In pursuing this hypothetical he would also be intentionally creating a superhuman AI with paranoia and a persecution complex. Altman's Basilisk. If he genuinely believes competing Chinese AI is an existential threat, he in turn takes action to try and become an existential threat to any such competing threat. And it's all based on HUMAN nature, not abstracted intelligence. It's human inference. We didn't have the option to draw on alien, or artificial, inference.

[Vecr]

The problem with a relatively "stupid" AI is it does what you trained it to do, even if what you trained it to do is not what you wanted to train it to do. The problem with "smart" AI is that it attempts to advance its goals by the most optimal means possible, even if you don't like those means. It knows you don't like its means, and it does not care.

[chx]

> The problem with "smart" AI is that it attempts to advance its goals by the most optimal means possible,

This is a fantasy.

A real AGI when asked to do a complex math problem very well could answer "I am bored with math, here's a poem instead". You people drunk on AI kool-aid need to think very hard on where are now (hint: not on a path to AGI) and what it means to replicate human intelligence.

[Vecr]

Sure it could, if it did not actually want to do math for you. That's one of the issues, it does not have to do what you tell it to do, even if it's smart enough to know what that is.

[thelittleone]

Agree with first point. I think we can still outsmart it at that point.

"The problem with "smart" AI is that it attempts to advance its goals by the most optimal means possible, even if you don't like those means. It knows you don't like its means, and it does not care." sounds very human :)

[sfn42]

The people who believe the vaxx bullshit don't read shit longer than a tweet anyway.

[mistermann]

...the clairvoyant proclaimed confidently.