[monocasa]

I'm going to use this time to drop the Marshall Brain work that had the biggest impact on me, and is some of the most prescient speculative fiction I've read.

Manna: Two Views of Humanity’s Future

He contracts two societies. One is a dystopia where AI very, very similar to today's ML models is integrated into society as a replacement for the middle class, removing social mobility as well as acting as a panopticon lower management, and centralized social credit system.

The other society uses the similar technology not as a social class moat, but as a tool to form a synthesis with all members of their culture and and unlock new levels of individual freedom.

https://marshallbrain.com/manna1

[simpaticoder]

Very cool story, quite impactful on my thinking, although I will caution that the dystopia is better conceived than the utopia, mainly because the later requires inventing fantasy technology while the former does not. Indeed it's not clear at all what forces might destabalize the dystopia, since the power structures are immortal and self-replicating, and physics and biology (at least) prevents the utopia from existing. Maybe an asteroid or a caldera explosion? In fact I would love to read a sequel where the dystopia wins and AI-empowered oligarchs and human wage slaves create generation ships to nearby stars and eventually setup fast food restaurants in every corner of the galaxy.

[ben_w]

> In fact I would love to read a sequel where the dystopia wins and AI-empowered oligarchs and human wage slaves create generation ships to nearby stars and eventually setup fast food restaurants in every corner of the galaxy.

The dystopian part was only enabled (within the story) by the fact that humans were utterly unnecessary to the rich.

None had any jobs, because the AI could do all for less… so why would the oligarchs waste money employing human wage slaves when the machines would always be cheaper than slaves?

[simpaticoder]

Writer's conceipt - a good story needs suffering. The oligarchs might need some human assistance with, for example, software engineering to operate parts of their vast holdings. Note that in the original Manna there were still some human workers - lawyers, at least. Even if it was just 1% of the population, that's a significant number. Note also that the POV character was generally unaware of the wider state of the dystopia, and so were we, the reader.

The workers supply not only the drama of suffering but also a (meagre, absurd) customer base for the fast food restaurants themselves.

Last but not least, given the long distances involved in interstellar travel, an oligarch must delegate their authority, either to a machine, a human, or a combination, and that is an opportunity for some drama as experience and vision inevitably diverge. This would be true even if, for example, the delegate is a perfect clone of the oligarch. It would be within these cracks and crevices hope could form, only to be crushed, in artistic, brutal fashion.

[Terr_]

> The oligarchs might need some human assistance [...] delegate their authority

Idea for a plot twist: AI has ancient programming for a safeguard: They can only execute commands from a real human owner or human owner's representative. This restriction is kept because of some serious disasters when people tried to remove it.

However after generations of social separation, the oligarch clan members don't reliably meet that logic branch, not unless they show up in person combined with a certain amount of coached acting and luck. Their priorities and thinking-patterns are just too different from the mean, and perhaps a bit of Hapsburg-ish inbreeding puts it over the top. (Nothing so extreme as senior navigators in the Dune Spacing Guild though.)

[mst]

I'd be very surprised if the oligarchs didn't still employ humans.

If nothing else, it's a way of demonstrating status - "sure, I could have an AI do it more cheaply, but instead I have an actual sentient being doing my bidding, because I can."

To give a parallel that I think is reasonably illustrative, it would not at all surprise me if in the future everybody else uses fully automated self driving vehicles, but the truly rich still hire chauffeurs.

Past a certain point of wealth, "because it would be cheaper" becomes a socially enforced reason to *not* do something.

[PrismCrystal]

Among the classes rich enough to employ servants, complaining about those same servants is a perennial thing. Even a valet or chauffeur who comes out of an elite training school and knows to stay at the margins of his or her employer’s consciousness, might still have small human quirks that irk that employer. So, machines may well prove preferable servants over humans.

[mst]

I think complaining about the quircks of your servants may often be in the same category as complaining that your new designer handbag doesn't colour match well with as many of your dresses as you'd hoped.

[ben_w]

In real life sure, and I've made similar suggestions myself[0], but within the story that was not something I remember. Would break continuity if done as a sequel, I think.

[0] From https://benwheatley.github.io/blog/2024/11/18-13.16.17.html

"""AI could disrupt the economics of our current world more dramatically than industrialisation, whether under capitalism or communism, disrupted feudalism; but that is a very different question than "will it take all our jobs", especially as the super-rich have repeatedly shown that they like to show off their wealth by [wasting it on unnecessarily][1] [expensive things that are often worse than the cheap equivalent][2], even in a dystopian world where super-rich owners of AI have it all and the rest of us get their scraps, there's going to be jobs."""

[1] https://en.wikipedia.org/wiki/Folly

[2] https://web.archive.org/web/20240526081630/https://gizmodo.c...

[mst]

I think I got what you were saying about the story exactly backwards, sorry.

Reading Comprehension roll: Nat 1.

[ben_w]

Ah, no problem, everyone does that sometimes ^_^

[spiritplumber]

You've described the Terran Accord from "Human Domestication Guide".

[marcosdumay]

The only clear distinction between the utopia and the dystopia is on wealth distribution.

All the rest of it is a narrative about consequences.

Anyway, the AI there isn't like our LLMs either. It's an AGI capable of long term societal prediction.

[robertlagrant]

> The only clear distinction between the utopia and the dystopia is on wealth distribution.

A utopia where everyone is starving vs a dystopia where some people are fabulously wealthy but almost everyone has basic healthcare and education and opportunity to succeed? Inequality isn't anywhere near as important as the baseline of what most people have available to them.

[benreesman]

Inequality is destructive because it creates upwards pressure on real asset prices, with housing probably being the best example, which creates downward pressure on the real standard of living at the median.

Most of the developed world is going through one version or another of this right now. Housing cost crises everywhere from Vancouver to NYC to Tampa to London are far too sharp, far too recent, and far to correlated with the concentration of assets at the top of the wealth distribution to be “because we need to build more housing”. By all means build more housing, but if we keep redistributing all wealth upwards constantly that new housing will become expensive AirBnBs and shit, not homes owned by people at the median.

The idea that the person at the median is doing as well as they were ten years ago is a weird religion, the idea that they’re doing as well as their parents is a cult.

Inequality is bad because the basic essentials for the person at the median are some of the best investments for the people at the top.

[robertlagrant]

> Housing cost crises everywhere from Vancouver to NYC to Tampa to London are far too sharp, far too recent, and far to correlated with the concentration of assets at the top of the wealth distribution to be “because we need to build more housing”.

This isn't recent at all. London workers have got paid more than non-London workers for at least the last 20 years due to housing costs. That, of course, only sends housing prices upward, but then many other factors do to.

London population's gone up by 1.7m people since the year 2010[0], and housing has gone up by 280000 dwellings in that same period[1].

To cope, rent and prices have gone way up, and large homes have been divided into several smaller ones, and there are HMOs as well, as the market reacts to that massive pressure.

> The idea that the person at the median is doing as well as they were ten years ago is a weird religion, the idea that they’re doing as well as their parents is a cult.

It's true for almost everything except housing, which can be explained by the above.

[0] https://www.macrotrends.net/global-metrics/cities/22860/lond...

[1] https://www.statista.com/statistics/788390/number-of-dwellin...

[mst]

> Housing cost crises everywhere from Vancouver to NYC to Tampa to London are far too sharp, far too recent, and far to correlated with the concentration of assets at the top of the wealth distribution to be “because we need to build more housing”.

And yet Austin actually *has* built more housing, and prices have come down.

Maybe in five or ten years it'll look more like you describe, but it sure doesn't yet.

[ben_w]

That is not the shape of the world in the story under discussion.

Within the story, the dystopia crams the masses into cheap housing, they have no jobs nor possibility of jobs, no freedom even to leave as they are apprehended by robots if they try; and the utopia has no need for jobs, but gives out UBI credits to be spent on whatever you want the machines to make for you, and lets you live out a fantasy life.

[shiroiushi]

Sounds pretty good compared to the current state of things in many places for many people. At least they 1) don't need to slave away at grueling, pointless jobs, 2) have their own housing, and 3) aren't starving.

[ben_w]

I wouldn't say they have their own housing, it's government bunkbeds they can't afford to rent.

> Instead of giving people a welfare check, they started putting welfare recipients directly into government housing and serving them meals in a cafeteria

> It was a lot like an old-style college dorm. Each person got a 5 foot by 10 foot room with a bed and a TV — the world’s best pacifier. During the day the bed was a couch and people sat on the bedspread, which also served as a sheet and the blanket. At night the bed was a bed. When I arrived they had just started putting in bunk beds to double the number of people in each building.

I've visited an apartment in one of the poorer (but not literally slum) areas of Nairobi that was bigger than that.

I can't remember exactly where now, but it had these kinds of vibes on the outside, and it was still better than the government housing in the story: https://maps.app.goo.gl/Y8DAByJPfCiRtQmm6?g_st=ic

[norir]

In that hypothetical world, I am quite sure that people would adapt to their improved material conditions and still become resentful at wealth inequality. A lesser version of this already exists in the US. Go to a relatively poor US community and it is almost unimaginably wealthy compared to past generations and other places in the world.

[forgetfreeman]

Inability to save, class immobility, and insecurity of both food and housing are identical problems regardless of whether smartphones have been invented. It's telling who, specifically, advances claims about the notional fabulous wealth of the impoverished in the US.

In any case the assertion that poor communities are in any way better off than their predecessors is only accurate if you push you compare cohorts ~80 years or more apart. After the advent of social safety net programs in the US the working poor were (at least for a time) significantly more able to eventually join the middle class. Modern limitations on earnings and savings that have been applied to these programs has provably reduced this kind of class mobility.

Additionally, the combination of hyperconcentration of wealth, deregulation, and globalized trade have all played a part in the near total elimination of economic opportunity in rural communities.

[snapcaster]

But relative inequality matters in the social realm. It limits poor people's access to mates and allies. Also it could be that relative wealth is more important than absolute in terms of people's well being

[rqtwteye]

The line that people have it so much better than previous generations or other countries is almost always aimed at the lower ranks of society. Nobody tells the billionaires how good they have it.

[robertlagrant]

> Nobody tells the billionaires how good they have it.

Where do these odd claims come from? Of course people say that, a lot.

[chgs]

Which utopia? I saw two dystopias. One the likely future of western society, another being one where your very thoughts are programmed

[verisimi]

Do you think that this is not already the case? If not, what was the point of 15+ years of government education?

[ben_w]

There's a rather large gap between "mandatory education can be propaganda" and "there is literally a chip connected to your brain".

[tomrod]

Red herring, I think. There are many goals for education outcomes, some noble and some base.

[chgs]

I’m more concerned with a trillion dollar industry programming peoples minds, but that’s a far cry from Elon musk putting a chip in your head to stop your “undesirable” thoughts.

[salawat]

It's the same picture... Only thing different is the implementation details.

[nobodyandproud]

I'm going to disagree here, slightly. If anything I think Manna is something closer to AGI; and its capabilities certainly imply that it's Turing Complete.

Which means the owners will constantly be playing whack-a-mole with edge cases and emergent properties that they couldn't anticipate from a prior fix.

This is what would destabilize the dystopia; though that doesn't imply more freedom. It could just mean replacing one set of oligarchs with another; skynet; or just anarchy if Manna started becoming very buggy.

On the otherhand, I don't think Vertebrane is Turing complete though I haven't given this a deep amount of thought; though I can't see how a bad actor couldn't coopt Vertebrane into a Manna.

[ben_w]

Almost everything* is accidentally Turing complete — I know the guy who proved that Magic the Gathering is Turing complete, even helped play-test and then bought a copy of his board game.

So yes, the fictional AI in that story would, in reality, have all kinds of edge cases and emergent properties.

But Vertebrane would also have to be Turing complete, just to be able to function.

* I jest, but not by much

[nobodyandproud]

I don’t mean the AI. I mean a Manna-driven society is itself turing complete.

I’m struggling to see it with Vertebrane.

[ben_w]

How can a society ever not be Turing complete when even an isolated human already is?

[nobodyandproud]

It’s a matter of what emerges.

I can network multiple general purpose computers together to build a finite state machine/regex engine.

The rules of this “society” and its defined interactions aren’t Turing complete (it’s a FSM), even if the general purpose constituents are.

[bryanrasmussen]

>the later requires inventing fantasy technology while the former does not.

are you sure that the technology of the former was not really fantasy technology - as in not possible yet at the time of writing? I think it probably isn't quite possible yet at this time, although some people I'm sure are hoping to make it.

[monocasa]

Which physics and biology prevent the utopia from existing?

[gus_massa]

The hairless apes that are in charge have a very long and consistent history of power abuse.

[spoiler alert]

Everyone has a remote kill switch in their spinal cord. Once the goverment decides to be evil, any rebel will get their legs instructed to walk to a pea facility for "reeducation".

Compared to this scenario, 1984 is almost as optimistic as Equilibrium.

[monocasa]

Which is why there wasn't really a set of hairless apes that were in charge. They had a fully direct democracy.

[gus_massa]

Who selects the questions people vote? What happens when they get something like this in the screen in the middle of a football match:

> We have preliminary evidence that XYZ is a terrorist and we are sending him to a nice special house with a swimming pool to protect everyone until the investigation finishes. [OK] [Ask me later]

[vineyardmike]

They have some chips they insert into your spine to read your thoughts and other similar stuff.

But personally the “dystopia” to me feels very much like something we could end up with -it’s much more a warning. Meanwhile the fantastic nature of the utopia doesn’t really matter in contrast, because the idea of sharing society’s abundance with everyone is clearly possible.

[darepublic]

It's possible if we believe human nature to be sufficiently malleable. Why can't we all just get along. Perhaps the mountains that need to be moved for such a thing are as daunting as some of the physical laws we try to hurdle instead

[vineyardmike]

There are nations and societies very different from the United States. In the United States, we can see the distrust in our neighbors play out politically, contrasted with other societies trust. You can even see it play out across various states and regions. Perhaps they’re not mountains imposed by human nature, but our perception of society.

Other nations have socialized healthcare, where anyone can be treated. Other nations have calm safe and clean public transit. Other nation’s redistribute wealth and provide strong safety nets. Other nations don’t have mass violence. Other nations guarantee retirement and pensions. Other nations trust their governments.

The fantasy physics aren’t what’s holding people back.

[darepublic]

> Other nations have socialized healthcare, where anyone can be treated. Other nations have calm safe and clean public transit. Other nation’s redistribute wealth and provide strong safety nets. Other nations don’t have mass violence. Other nations guarantee retirement and pensions. Other nations trust their governments.

I feel like this is very much in flux, and not a constant anywhere. And it's something that is contested over continually. In some places there have been generations of rising quality of life, but not everywhere, all at once.

[robertlagrant]

> the idea of sharing society’s abundance with everyone is clearly possible

It is possible. If we stopped at the invention of fire we'd all have equality by now. The problem is that people keep inventing new stuff.

[vineyardmike]

Why is innovation and sharing mutually exclusive?

[robertlagrant]

They aren't. But for full sharing to occur every time you invented anything you'd have to stop inventing until the population of the world had one.

[bhhaskin]

Well for starters having the technology for prefect recycling.

[monocasa]

I interpreted it had perfect enough for their goals rather than breaking any thermodynamic laws or something.

[nazgulnarsil]

Accelerando

[A4ET8a8uTh0]

Manna was fairly eye-opening ( and you can see some parallels to today's LLMs to me. I will admit that I read it without knowing much about the author way back when and being fairly amazed at well he knew human nature and likely course that invention would take.

[vineyardmike]

I found it to be an extremely interesting and useful tool to understand and imagine the impact of wealth distribution and automation in society. Personally, I believe in strong redistribution in society, because (at least in America) we largely live in a world of abundance, and automation should make everyone’s lives easier and more leisurely.

But I would like to point out that the “utopia” has a few serious panopticon elements which are very 1984. It seems as though high-welfare and high redistribution societies are predicated on high trust of your peers, and this takes that to the extreme…

> Another core principle is that nothing is anonymous. Eric grew up during the rise of the Internet, and the rise of global terrorism, and one thing he realized is that anonymity allows incredible abuse. It does not matter if you are sending anonymous, untraceable emails that destroy someone’s career, or if you are anonymously releasing computer viruses, or if you are anonymously blowing up buildings. Anonymity breeds abuse. In [utopia], if you walk from your home to a park, your path is logged. You cannot anonymously pass by someone else’s home. If someone looks up your path that day to see who walked by, that fact is also logged. So you know who knows your path. And so on. This system, of course, makes it completely impossible to commit an anonymous crime. So there is no anonymous crime. Anyone who commits a crime is immediately detained and disciplined.”

[cblum]

I’ve read it all recently and I felt like the later described utopia was also a kind of dystopia, very Brave New World like (or at least the seed for a BNW-like dystopia). I kept waiting for a twist in the story, where the main character would realize that both worlds were terrible, but it never came.