[wcfrobert]

> Turn three: the company that fired its workers to save money discovers that its customers were, in aggregate, other companies’ workers. Revenue growth stalls. The AI subscription that was supposed to be an investment in efficiency turns out to be a contribution to the destruction of its own market.

If we take it to the extreme, the final solution to this problem is secessionism: a fully non-human AI economy where the customers and providers are both robots. Why fund public education or research or healthcare? Just build more data centers. A billion dollars and a bunker in the Southern Hemisphere will not save anyone. Capital is not a moat in this hypothetical non-humane world. Whence do you derive your authority? How can you trust your body guard? You and what army? An army of robots/drones? What if they get hacked? What if the AI researchers get alignment right and Claude refuses your request?

It's all so obscene. Instead, why don't we try to protect human dignity and move towards a more humane future?

[hax0ron3]

>It's all so obscene. Instead, why don't we try to protect human dignity and move towards a more humane future?

I think the Darwinian logic of reality might make this hard. If society A and society B are both developing AI and one of them stops in order to protect humans, society B may continue to develop AI and then it might either outcompete society A economically to the point of reducing it to poverty (it is theoretically possible to take most of another society's market share in something by only slightly outcompeting it in price or quality), or it might even outright conquer society A.

A solution to the problem needs to address this issue somehow.

[rajamaka]

Maybe a stupid question but aren't the Amish essentially this thought process put into practice?

They aren't able to outcompete their neighbouring societies economically yet are in no way impoverished, and in many cases actually come out on top by many QOL/health metrics.

[RobKohr]

By the protection of the host society. It the Amish were an independent nation they would be conquered.

[satvikpendem]

Don't romanticize the Amish, they have lots of issues including literal child sexual abuse and incest.

https://www.cosmopolitan.com/lifestyle/a30284631/amish-sexua...

[VinLucero]

Yeah, it’s the same problem as nuclear proliferation.

We see what happened with Ukraine when they gave up their nukes in exchange for a promise of protection.

The same could be true of countries who forgo AI development directly in exchange for promises.

[chadgpt3]

More poignantly we saw it in Iran.

[lenerdenator]

Wanting more independence from a former imperial master != building an oppressive government on theology and building a nuclear program that seems to exist to back that theology by threatening a nation that causes you theocratic issues.

[chadgpt3]

I agree, Iran != America

[thinkthatover]

this is Amodei's position in a nutshell for AI development. We have to go as fast as possible because China. It's not the only frame though. If AI models and warfare (cyber, biological) becomes easily accessible and dangerous enough, there is a strong incentive for the world's leaders to cooperate towards something akin to nuclear non-proliferation.

In fact, there's strong incentives now to slow down AI progress for multiple reasons: de-escalate tension over Taiwan and lessen China's desire to build their own advanced fabs, protect peoples livelihoods by smoothing the AI transition. Except the incentives to bring AI companies public (and maintain some twisted shred of American Hegemony) are greater.

[rustystump]

There is no reason to slow down and only reasons to speed up. AI has not had the world shatteringly obvious negative potential as atomics by a long shot.

There is a latent potential for a negative outcome but the surface is showing a relatively benign productivity boost similar to a smart phone.

Only time will tell if the negative impact is on the scale of atomics or manmade bioweapons. Sadly, humans usually need to be burned a few times before learning the lesson. Eugenics seems to be a universal no go publicly across the world. But that was a painful lesson.

[lokar]

Unless you change the balance of selection pressures…

[gverrilla]

"Darwinian logic of reality"

[ranyume]

What does outcompete economically mean and why would it matter? Or do you mean society A dominates in some form society B? This has already happened in history and is the essence of capitalism. If you want to overcome this situation you need to replace capitalism globally.

[ian_j_butler]

> If you want to overcome this situation you need to replace capitalism globally.

Not true.. you just need to replace late-stage capitalism locally. There's a very reasonable concept of circular economy that's relevant here. It breaks down only in that sometimes you need things that you can't make and must go outside the circle. Especially if you intentionally work to mitigate the main stuff requiring you to go outside, it's not some law of nature that it must continue, or even a law of capitalism. Some protectionism / local-first is part of it, but the bigger part is just being rational.

The bad kind of globalism, enshittification, dead-economy theory, and basically ALL of the really ugly stuff we could talk about here are characteristic of late-stage capitalism and the associated short-term thinking, and it's totally consistent for a real capitalist to reject it. Why? Because getting as rich as possible isn't incompatible with sustainability.. particularly from the perspective of corporations/countries that plan to last longer than a single human CEO or exist beyond a single generation of shareholders/citizens.

[Henchman21]

The PKD short story “Autofac” seems highly relevant. Amazon had the HUGE BRASS BALLS to include this as an episode in “Electric Dreams”:

https://www.imdb.com/title/tt6902176/

[the_af]

"Autofac" is one of my favorite PKD short stories, though it's very depressing and somewhat different from the "Electric Dreams" episode, which (if I remember correctly) started similarly but had a very different implication and outcome.

"Autofac" as originally written seems to me where this AI "utopia" is leading us.

[20after4]

Absolutely loved that episode. I also thought it was incredible that Amazon was the company to release this.

[dguest]

That Amazon releases this seems to be consistent with the trend the original linked article mentioned: these big tech companies are happy to warn people of the apocalypse they are causing.

It helps not to think of corporations as a single coherent consciousness: hypothetically everyone who works there, from top to bottom, can believe that what they are doing is harmful, but also feel powerless to fight the hand of the market.

In practice I doubt the entire management structure agrees on that, some honestly believe they are doing good, but there's nothing stopping EvilCorp from emerging from a bunch of perfectly good people who are "just doing their job".

[tomrod]

Don't request for human dignity. Demand it!

[rambojohnson]

But apparently the only acceptable way to demand basic human dignity and freedom is to stand politely in line with a ballot. I think we’ve reached a phase in society where the real call to action makes people uncomfortable. Nobody wants rubber bullets to the face.

[Henchman21]

If we’re not willing to fight for ourselves we don’t deserve life.

[satvikpendem]

> It's all so obscene. Instead, why don't we try to protect human dignity and move towards a more humane future?

It's funny that this question is asked when the answer to why not is already in your very same comment. The logical incentives for each member game theory wise tend toward that outcome you describe.

[gom_jabbar]

As mentioned in the comments, this is essentially Nick Land's vision, which he has been developing for over 30 years. If anyone is interested in exploring this topic in depth, I have a research project on it: https://retrochronic.com (The Capital Autonomization section of the introduction is particularly relevant.)

[matrix87]

Nick Land already predicted this

[gom_jabbar]

Yes. Here's one relevant quote that touches on autonomization and secession:

> Most simply, there is the utilitarian order, in which capital establishes itself as the competitively-superior solution to prior purposes (production of human use-values), and the intelligenic order in which it accomplishes its self-escalation (mechanization, autonomization, and ultimately secession).

Nick Land (2014). Freedoom (Prelude-1a) in Xenosystems Blog. Retrieved from github.com/cyborg-nomade/reignition

[righthand]

So maybe instead use Musk-style [strike]lies[/strike] hype to build a company that claims to bring Medicare for All to the public using advanced AI within 3 years. “Which will make everyone super rich not paying for healthcare.”

Then use the money to get candidates elected and [strike]bribe[/strike] lobby politicians. That eliminates the ballot issue and it plays at the rich scum bag line tow-ers game.

[gen220]

> why don't we try to protect human dignity and move towards a more humane future?

I hear and have a lot respect for what you’re saying, but I’d like to propose that we thoroughly explore every other alternative first, just to make sure we aren’t missing out on something bigger and better and leaving anything on the table.

Sigh.

[fwipsy]

You can't have an economy with just AIs because they don't consume anything except compute. If AI can build more compute, they can definitely defend their owners.

There's a limit to how much elites can consume. Most people are happy with a few million dollars or something. The people who go past that are obsessed; they're competing with each other.

There is no reason for elites to secede. There is no reason why we can't have bajillionaires and subsistence farmers on the same planet, in the same economy, using the same dollars. (It's basically already happened. What's a few more zeroes?) If AI cannot provide security (either directly or through creating wealth used to buy security) then it will not create this level of inequality in the first place.

Places like Sudan have already been left behind and they're currently in the middle of a very bloody war which the West is largely ignoring. Now the Western middle class is making noises about violence because their prosperity is under threat. But this is what capitalism has always done. This is what we signed up for.

[BosunoB]

Ahh yes, the "human dignity" of billions of people toiling away to make Americans cheap widgets. There never was dignity to capitalism. We can strive to replace it with something better.

[rambojohnson]

human dignity is not profitable.