[BoorishBears]

> you get pissed off as a software engineer lol

I think this part depends on the person. I've personally been programming since I was a kid making games for my TI-83+, and in all that time and fatigue have been the limiting factors in how much of what I wanted to build that I actually could build.

So something able to write code rigorously enough to replace SWEs would be an absolute dream! I love programming with all my heart, and it's the thing I've spent most of my life doing... but I feel in love with it because it could make things.

A way to make even more things at a greater scale I'm individually capable off is such a joyous idea that if anything, I get annoyed at the idea they'd tease that without knowing it's possible (of course, they're trying to raise so...)

The aspect of wanting to replace SWEs is completely ok with me and I think there should be a rush to see it through. Imagine if every researcher could have an army of top tier SWEs at their beck and call for example. Or even imagine learning to program alongside a personal world-class expert from day 1, after all the fact AI could do it wouldn't mean we couldn't still do it ourselves if we wanted to.

-

Unlike the "AGI in 3 years crowd" I don't actually know that it's possible, but where I agree with them is that the route there is probably not going to be a slow burn. Most companies need to demonstrate some external value along the way or they won't be able to continue, hence the chasing down of usecases that they can ship today.

Unfortunately not all of us can raise $1B on a txt file and a promise not to release our product :)

[arvinsim]

I would guess that the pushback is more about the possible economic imbalance that will happen and less about being replaced on the actual effort of coding.

[BoorishBears]

Maybe because I'm not originally from this country my view is different, but I think the fact the majority of the world's increase in population is about to happen in extremely poor places that will be subject to the worst of climate change means that the AI will have to be immensely powerful to actually increase worse imbalances than we're already headed for.

So powerful that it'd also raise the floor on quality of life for the 8 billion people on earth almost with ease, even if its owners stayed deeply profit/power motivated.

After all, it's not like the tech bros will get to make money by hoarding the AI and it's fruits after all. They need to apply to downstream tasks to actually cash in on its value. They could hoard the AI itself, but if OpenAI was suddenly able to break into every industry with a tireless AI army of top engineers and researchers they'd still be be producing real advancements for the world.

(and to be clear that's closer the worst timelines where AI advances so greatly. I think the more realistically we'd seem competition lead to something much closer to widespread advancements rather than some singular superpower emerging)

[intended]

I am almost certainly wrong, and we will find some solution, or sue the hell out of the genAI firms, but this is the economic issue I see. It competes with productivity as a core economic driver of human wellbeing:

Concentration of wealth.

GenAI consumes content, even that created in low resource languages and regions, and spits it back out, separating the creator from the traffic due to their labor.

This isn’t entirely unknown - we’ve all been inspired by someone else stuff and copied our own.

Now, genAi firms have inserted themselves into this loop. And they’re cutting out the creator.

The scaled, automated pseudo workers that these firms promise, are owned by the firms. The productivity they create accrues to a small group of foreign multi nationals.

Economically - this shouldn’t be an issue. More productivity, means more capability of people doing newer work.

I do expect this to happen. However firms are also very good at making sure they capture the greater share of the market.

[dividedbyzero]

That researcher probably wouldn't have an army of SWEs at his or her disposal but be out of a job like the SWEs. If they get AI to a point where it can be a safe and competent senior SWE, it'll be able to fill a huge breadth of other roles as well. Human creativity isn't looking like quite the moat it was supposed to be.

Our societies are not in any way equipped to deal with putting what may well be a sizable majority of working-age people out of work, possibly for good, nor are we in any way ready for the kind of power certain tech billionaires would have if their workforce were to scale with just the amount of hardware they own.

At this point I kind of hope the current breed of AIs will plateau quickly and stay there for a while so that maybe society can catch up instead of getting surprise bulldozed by a gaggle of tech giants.

[cudgy]

Question I have is if no one has any job because it’s been replaced by AI, what happens to the economy?

[blibble]

essentially it would be the end of employment and return to feudalism

likely followed by either a French Revolution (if lucky) or a Russian Revolution (if not)

[cudgy]

Of what use are the peasants in an AI driven and dominated feudal society, though. Maybe us peasants be useful in wars or battles between the different lords, but this would also probably be performed by automated drones and robots.

It would seem that the Lord‘s will have nothing to Lord over though. Who’s gonna buy their crap and for what reason will they they create anything? The whole thing seems like a massive doom spiral.

[Nevermark]

> It would seem that the Lord‘s will have nothing to Lord over though.

You have machines that can design and build mega-yachts, mansions, private space craft, ....

You can afford to purchase vast areas of land from people selling whatever they have to get by, ...

--

Asking what billionaires will do when they can't sell to the poor, is like asking why the human economy didn't crash ages ago because we have any off planet aliens to sell to.

Or how did we keep the economy going all this time, given the ants and trees couldn't afford anything we produce?

All an economy needs is someone with the means of production, who is able to get resources, and use those resources to produce something they want. I.e. you can have a working economy with just a single person. Or self-interested AI.

Hermits have an economy. Now imagine the hermit has trillions of dollars of resources and square miles of intelligent circuitry and robotic servants.

That hermit doesn't need the rest of us. Customers? Where we are going, we don't need customers.

[cudgy]

Ok. I guess they have themselves to Lord over. I’d hardly call that an economy though.

[DaleMgrh]

https://asimov.fandom.com/wiki/Solaria

[cudgy]

“The Solarians specialized in the construction of robots, which they exported to the other Spacer Worlds.”

Thanks for this, but Solarians have an economy that sells robots to other planets. Earth has no such luxury.

[BoorishBears]

> Our societies are not in any way equipped to deal with putting what may well be a sizable majority of working-age people out of work

Our societies were built before there was a technology that could replace its smartest people with machines that never tire?

I don't get why people keep trying to imagine current society + super-intelligent AI: by definition it won't be our current society if we can actually get there would it?

I mean if we have AI that can even replace the researchers (I wouldn't dream so boldly tbh), imagine how much faster the pace of scientific discovery becomes. Imagine how much more efficient we can make power generation and transmission, discover new treatments for disease, democratize learning at costs never before possible...

I don't love to spend too much time daydreaming what we could do down that because SWEs already feels like a bit of a pipedream, so all novel research being automated away is just completely in fantasy land... but realistically we're already on a pretty terrible trajectory otherwise.

Our next billion people are about to be born into some of the worst off parts of the planet. AI becoming good enough to replace researchers would be an infinitely more positive trajectory than some of the others we could end up on on otherwise.

[dividedbyzero]

Social media could have been utopian, too, yet those apps are algorithmic manipulation hellscapes that threaten to bring down even the most robust democracies. The same people who make it so are poised to be the ones in control of these AIs. I don't think they want the kind of utopia you imagine.

[BoorishBears]

What I described doesn't have to be utopian in an absolute sense, just significantly better than where we're currently headed.

I think a lot of the unchecked pessimism around super-intelligent AI is just people being a bit naive or shut off from the reality of just how terrible things are going to be over the next century.

We're waging 25% tariffs over planefuls of people, what's going to happen when it's 100 million people trampling over borders trying to escape disease, famine, and temperatures incompatible with human life?

Compared to that, even if these companies abuse their ownership of AI and monopolize the gains, an AI capable of producing novel research and development by itself would still bring us much closer to solving major problems than otherwise.

[sensanaty]

And you think the tools that are, as we speak, boiling rivers and lakes in order to power the insanely resource-hungry AIs is the solution to any of those problems, such as famine and rising temperatures? If anything, they're accelerating us towards these issues.

There's about to be 500 billion dollars invested in generating even more electricity for these monstrosities, instead of literally anything else actually useful today that we could be putting towards climate research or renewable energy. Nope, we're just gonna generate even more spam and bullshit while spinning up nuclear reactors to power it all.

[BoorishBears]

If they don't pan out we won't even reach 500 billion dollars actually invested and tbe bubble will pop. And it'll take many many many trillions before AI is even close to the biggest reason climate change is killing people.

People tend to take different pieces of mutually exclusive end states. Like my comment is speaking to a fantasy like outcome where again, we have AI that puts all researchers out of jobs.

If it can do that (which I don't think it will, but if we're dreaming it), let it boil oceans, let them abuse it, you can become unfathomably wealthy by solving world hunger.

Let them force countries to take out loans that practically put then in OpenAI's permanent debt to access the advancements they come up with.

It's still better than the alternative, and it still allows room for them to be as evil as they want.

I mean either way countries are going to become indebted to companies eventually if we keep down the current path. I don't see how the ultra wealthy can't use mass upheaval and desperation to secure extremely cheap labor and solidfy their power even without AI.

At least now, even if it's for their own personal gain and amusement, they can trivially solve

-

It's like we're talking about someone potentially developing a cure for all cancer, but everyone is worried because the company behind it is evil and will hoard it for themselves while charging $1B per dose: let's still get to that cure if it can be developed.

It'll be a truly miserable and awful world watching people we know die while there's a cure, and watching them die because they're not able to pay this villain... but today there's no cure, no proof one can exist, no hints on how to get one.

It's better to have the cure and no clue how to fix the broken situation that it creates, than to have no cure and no clue how to fix cancer, because one is a problem of people/power/ethics and the other is a problem of unknown proportions and guaranteed ongoing suffering not more damaging than having a cure, no matter how inequitable access to it is.

[DaleMgrh]

The problem emerges when the ones monopolizing the research use it for their own ends, which are to control everyone else.

So instead of being used to improve quality of life, AI gets used to improve efficiency of death.

[cookiengineer]

I agree in part with your views.

Though I think you misunderstand or underestimate human nature of self interest. Everyone that is in control of a superpower like this will abuse it. Be it on a presidential level, be it on CEO level, be it a major shareholder of a foreign NGO. That is why we had democratic splits of types of power in the first place. I say "had" because the trend globally is leading to right wing autocratic ideas due to manipulation of social media.

Human self interest and egocentric world views is what gave us this mess.

The only thing capable of evening out the odds is a federalistic decentralized approach, which we desperately need for AI. Something like a legislative system for lots of overfitted mini AI assistants that also give outliers a chance to be the social trend.

Otherwise we will land up with the ministry of truth, which, right now is Facebook and TikTok effectively. The younger generations that grew up with social media tend heavily towards populist right wing ideas because those are easily marketable in 30 seconds. Paint the bad guy, say that it is established fact, next video. Nobody is interested in the rationale behind it, let alone finding and discussing a compromise like you would in a real debate that wants to find a solution.

We need to find a way to change beliefs through rationale rather than emotions. Ironically this problem is also reflected in trained LLMs that turn into circlejerks because they've learned that from the dataset of us easily manipulateable humans.

[conartist6]

I don't think it's that simple. For one thing, AI isn't a democratizing force. If it's as good as you think it will be, it will be less like having a good education and more like having an indentured servant. Some people will have whole fleets of such servants doing their bidding, while others will have none.

For another, research isn't an end unto itself. As you note, for some people an already-unfathomable level of societal knowledge has resulted in nothing but continued poverty. Benefit from scientific knowledge requires a stable economy full of consumers who can and will purchase high-tech items. Where will that wealth come from once the value of human intellectual labor has been so undercut by cheap AI intellectual labor? Without the capitol to make AIs work for us, most if not many people will be left to live their life as servants to AIs so that those AIs are able to have autonomy in the real world.