[drcode]

> At no point in the next 30 years will there not be an active community of people who "loathe" AI and work to obstruct it.

Then I have good news for you: If humanity goes extinct in the next few years because of unaligned superintelligence, there actually will no longer "be an active community of people who loathe AI and work to obstruct it"

[slg]

>If humanity goes extinct in the next few years because of unaligned superintelligence

This is either a misunderstanding of the anti-AI crowd or an intentional attempt to discredit them. The majority of anti-AI people don't actually fear this because that belief would require that this person has already bought into the hype regarding the actual power and prowess of AI. The bigger motivator for anti-AI folks is usually just the way it amplifies the negative traits of humans and the systems we have created which is already happening and doesn't need any type of pending "superintelligence" breakthrough. For example, an AI doesn't actually need to be able to perfectly replace the work I do for someone to decide that it's more cost-effective to fire me and give my work to that AI.

[concinds]

There are many different groups of anti-AI people with different beliefs.

This attempt to "reframe and reclaim" (here, paraphrased: "significant existential risks from AI is actually marketing hype by pro-AI fanatics") is a rhetorical device, but not an honest one. It's a power struggle over who gets to define and lead "the" anti-AI movement.

We may agree or disagree with them but there are rational anti-AI arguments that center on X-risks.

[slg]

>There are many different groups of anti-AI people with different beliefs.

See my other comment. I qualified what I said while the comment I replied to didn't, so it's weird that this is a response to me and not the prior comment.

>here, paraphrased: "significant existential risks from AI is actually marketing hype by pro-AI fanatics"

If we're talking "dishonest rhetoric", this is a dishonest framing of what I said. I'm not saying this is inherently intentional marketing hype. I'm saying there is a correlation between someone who thinks AI is that powerful and someone who thinks AI will benefit humanity. The anti-AI crowd is less likely to be a believer in AI's unique power and will simply look at it as a tool wielded by humans which means critiques of it will simply mirror critiques of humanity.

[tptacek]

This particularly anti-AI article is not from a pdoomer.

[autoexec]

> an AI doesn't actually need to be able to perfectly replace the work I do for someone to decide that it's more cost-effective to fire me and give my work to that AI.

Exactly, "lack of intelligence" is really a much bigger concern than "superintelligence". Companies and government will happily try to save money and avoid accountability by letting AI do work that it can only do poorly and it will be humans who are left with the accelerated AI powered enshittification and blind/soulless paperclip maximization that results.

[mitthrowaway2]

It is not a misunderstanding; the anti-AI crowd is heterogeneous.

[slg]

Which is why I said "The majority of anti-AI people...". It was the comment I was responding to that was treating the anti-AI crowd as homogeneous by ascribing to them all a rather fantastical belief of a minority of that group.

[oidar]

> If humanity goes extinct in the next few years because of unaligned superintelligence,

I've seen people claiming that this could happen, but I've yet to read any plausible scenario where this might be the case. Maybe I lack the imagination, could you enlighten me?

[drcode]

https://ifanyonebuildsit.com/

[YZF]

- AI smarter than any human.

- AI dominated the physical world. Robots, factories, etc.

- AI decides humans aren't contributing and/or wasting resources it feels should go somewhere else.

I mean not unlike humans causing extinction of other species?

[theamk]

That "etc" in "robots, factories, etc" is doing a lot of work here.

Factories, even fully robotic ones, heavily rely on humans to set up and maintain them. Moreover, the safety culture means there are tons of "disable" controls which can be triggered by any human and no machine can override.

Robots look impressive, but they cannot function without the humans either. Military kill-bots are likely the worst, but machines cannot repair or refuel them.

None of this is going to change in the "next few years".

[YZF]

Yes.

Robots can't function without humans because they're not super-intelligent. We already see quite capable humanoid robots. Those factories that rely on humans - they'll be converted to be operated by humanoid robots. By the super intelligence.

That's the hand wavy story. It's hard to dive into details in an HN comment but I'm happy to try and develop some of those details. You're saying that something much smarter than humans isn't going to be able to bridge the gap to the physical world. I'm not so sure.

EDIT: Another way to think about it is that if a god-like infinitely capable being took control of all our online digital systems including I donno Teslas, factory automation, power grid, any form of connected robot in the world, nuclear weapons launch systems, airplanes, whatnot, do they have any path to a sustainable "existence" without relying on humans. Or at least with us unable to detect and stop that. If the answer is no then we're probably safe. It's kind of hard to convince ourselves of that. Keep in mind that humans can also be manipulated to do work for this god just like spies/saboteurs e.g. are recruited online today and paid bitcoin to do some random master's bidding.

[theamk]

> Another way to think about it is that if a god-like infinitely capable being took control of all our online digital systems including I donno Teslas, factory automation, power grid, any form of connected robot in the world, nuclear weapons launch systems, airplanes, whatnot, do they have any path to a sustainable "existence" without relying on humans

ha ha ha no. Teslas run of energy and cannot refuel, a circuit breaker in factory pops and no robot can reach it (or maybe a roof leaks), and "any forms of connected robot" _either_ cannot walk the stairs or can maybe run for a hundred miles before running out of battery.

The "humans can be manipulated" is the only thing to worry about, and you don't need robots for that, other humans have been trying hardest to do it just fine for millennia. I guess it's up to you if you want to be afraid or not, but I am not seeing anything super special so far.

[cwillu]

I've yet to read any plausible scenario where stockfish defeats me, all the scenarios my friends come up with have obvious holes in the plays they suggest stockfish could make.

[Aerroon]

But AI isn't going to be unaligned. It's going to be aligned the same way we are because it learns from our data.

[drcode]

we mostly know how to make it understand what we want. we don't know how to make it care about what we want, except via reinforcement learning. there are good reasons to believe rl won't work for this once the ai reaches a certain levels of capability.

[ryandrake]

What's more likely to happen is that humanity won't go totally extinct--it will just drastically shrink. When robotics and AI perform all useful work and everything is owned by the top 1000 richest people, there will be no more economic purpose for the remaining 7,999,999,000 of us. The earth will become a pleasure resort for O(1000) people being served by automation.