[hax0ron3]

>It's all so obscene. Instead, why don't we try to protect human dignity and move towards a more humane future?

I think the Darwinian logic of reality might make this hard. If society A and society B are both developing AI and one of them stops in order to protect humans, society B may continue to develop AI and then it might either outcompete society A economically to the point of reducing it to poverty (it is theoretically possible to take most of another society's market share in something by only slightly outcompeting it in price or quality), or it might even outright conquer society A.

A solution to the problem needs to address this issue somehow.

[rajamaka]

Maybe a stupid question but aren't the Amish essentially this thought process put into practice?

They aren't able to outcompete their neighbouring societies economically yet are in no way impoverished, and in many cases actually come out on top by many QOL/health metrics.

[RobKohr]

By the protection of the host society. It the Amish were an independent nation they would be conquered.

[satvikpendem]

Don't romanticize the Amish, they have lots of issues including literal child sexual abuse and incest.

https://www.cosmopolitan.com/lifestyle/a30284631/amish-sexua...

[VinLucero]

Yeah, it’s the same problem as nuclear proliferation.

We see what happened with Ukraine when they gave up their nukes in exchange for a promise of protection.

The same could be true of countries who forgo AI development directly in exchange for promises.

[chadgpt3]

More poignantly we saw it in Iran.

[lenerdenator]

Wanting more independence from a former imperial master != building an oppressive government on theology and building a nuclear program that seems to exist to back that theology by threatening a nation that causes you theocratic issues.

[chadgpt3]

I agree, Iran != America

[thinkthatover]

this is Amodei's position in a nutshell for AI development. We have to go as fast as possible because China. It's not the only frame though. If AI models and warfare (cyber, biological) becomes easily accessible and dangerous enough, there is a strong incentive for the world's leaders to cooperate towards something akin to nuclear non-proliferation.

In fact, there's strong incentives now to slow down AI progress for multiple reasons: de-escalate tension over Taiwan and lessen China's desire to build their own advanced fabs, protect peoples livelihoods by smoothing the AI transition. Except the incentives to bring AI companies public (and maintain some twisted shred of American Hegemony) are greater.

[rustystump]

There is no reason to slow down and only reasons to speed up. AI has not had the world shatteringly obvious negative potential as atomics by a long shot.

There is a latent potential for a negative outcome but the surface is showing a relatively benign productivity boost similar to a smart phone.

Only time will tell if the negative impact is on the scale of atomics or manmade bioweapons. Sadly, humans usually need to be burned a few times before learning the lesson. Eugenics seems to be a universal no go publicly across the world. But that was a painful lesson.

[lokar]

Unless you change the balance of selection pressures…

[gverrilla]

"Darwinian logic of reality"

[ranyume]

What does outcompete economically mean and why would it matter? Or do you mean society A dominates in some form society B? This has already happened in history and is the essence of capitalism. If you want to overcome this situation you need to replace capitalism globally.

[ian_j_butler]

> If you want to overcome this situation you need to replace capitalism globally.

Not true.. you just need to replace late-stage capitalism locally. There's a very reasonable concept of circular economy that's relevant here. It breaks down only in that sometimes you need things that you can't make and must go outside the circle. Especially if you intentionally work to mitigate the main stuff requiring you to go outside, it's not some law of nature that it must continue, or even a law of capitalism. Some protectionism / local-first is part of it, but the bigger part is just being rational.

The bad kind of globalism, enshittification, dead-economy theory, and basically ALL of the really ugly stuff we could talk about here are characteristic of late-stage capitalism and the associated short-term thinking, and it's totally consistent for a real capitalist to reject it. Why? Because getting as rich as possible isn't incompatible with sustainability.. particularly from the perspective of corporations/countries that plan to last longer than a single human CEO or exist beyond a single generation of shareholders/citizens.