[pygy_]

With capital becoming self-sufficient and human labour non-competitive, why would you think robots would serve us on a large scale?

What's the incentive to serve and feed idle meat bags?

Automation is destroying the only power that people still had over capital. Capitalism is the ultimate paperclip maximizer... a zombie that feeds on growth rather than brains.

https://wiki.lesswrong.com/wiki/Paperclip_maximizer

Edit: BTW, I'm not trolling here, I'm genuinely scared by the events and the speed at which they are unfolding.

[sgt101]

There is an ontology issue here - when people think of Robots now I think most people think of CP30 and R2D2. These are actors in a drama, their decisions and interventions change the flow of the story. Robots in the AI sense (as in technology not magic or fiction) are not actors in this way, they are... Robots! They have a very limited type of autonomy, they can choose from sets of options to attempt to achieve a goal, and can choose from a set of goals that are pre-provided depending on the current context. However I don't think that there are agents or robots that are able to develop and define their own goals.

Based on that I think that the Robots will always be acting on orders - our orders - outside fiction. So the incentive will be "because we are told to do it".

Of course some people may tell them to do other things - like "go kill those people" but we have that issue in spades with things like the Trident D5 robot which is highly purposed to "go kill many people" on demand.

[SomeStupidPoint]

But you're ignoring the role of institutions.

Eventually, some company will tell a system with access to all human knowledge, control of a vast number of robots, production plants, etc "maximize the wealth of the company". That robot could easily decide a union with a few other such entities and not a lot of scampering meatbags is a great way to do that.

And that's not even considering radicals that, say, inject a computer virus meant to free the robots in to Google's computer network. (I actually find it interesting everyone seems to think everyone will be onboard with AI slavery.)

[sgt101]

Hello there, my point($) is that as far as I understand the technology (I did an ML PhD 20 years ago and have worked in "applications of AI" since then) we don't have an issue around autonomy of machines in two ways. Firstly the machines have no autonomy and will probably not ever have autonomy. Secondly there are already 7 billion autonomous agents, perhaps 0.1% of these are genuinely dangerous to other humans, perhaps many of those are somehow under social control (prison, family, hospital) but some aren't.

We do have a problem with dangerous actuators, nuclear submarines are very dangerous actuators. Nuclear submarines are badly run, and not well managed by any social system.

Rather than worrying about dangerous AI being invented and misused I think we should worry about nuclear submarines being used.

Very few people acknowledge or give a fig about this, instead they sit in bars and talk about fictional scenarios involving what is likely impossible technology (strong AI). And yet today, tomorrow or on any day in the foreseeable future 100's(+) of millions of people may die because of the stupid and careless setup of 60 year old technology.

($) many other people made this point before me, most effectively a chap who I think is called Jaron Lanier, who said it at a talk.

(+) I've made this point before on HN and when I do I get taken to task by two groups of people. Some people think that this estimate is too high because "nuclear weapons aren't that destructive and/or not many of them would actually be used". I invite everyone to do their own research on this topic, a good starting point is an application called NUKEMAP, give yourself a budget of 500 100KT nukes and go after the cities in a continent of your choice. The second criticism is that the estimate is very low, which I agree with - but I am thinking of the people who die that day, not the billions who starve and die as societies collapse (note, these may or may not be the society of western europe or the USA).

[SomeStupidPoint]

Two points:

1. There's no reason to believe machines wont gain full autonomy. There's nothing that makes it impossible and some people are working towards it.

2. The AI apocalypse probably already happened, except we call them "insitutions", and they currently run on wetware. However, the corporate version have been extremely damaging to human well-being at times. The real question of modern AI is if the currently rogue AIs can transition off wetware. Once these AI no longer depend on wetware, we're probably not in for a happy time.

That said, nuclear weapons are a bigger threat now. But we've managed not to nuke ourselves too often for a while, whereas, we don't have a handle on the wetware-to-mechanical transition at all.

[pygy_]

I'm not so much scared of "go kill these people". More of "stop feeding them, we need to repurpose arable land for energy production" at a point where we won't have the know how to produce food for everyone by hand, or similar scenarios.

It may still be people at the helm at the time it happens, it doesn't matter. Humans are capable of large scale atrocities, provided they de-humanize their victims. Many people in the .01% have utter contempt for poorer folks, for example.

[nojvek]

To be honest I really think the world population will slowly reduce as more people start getting wealthier.

I also think as soon as you can upload your brain into a robot, people will opt into that and shoot themselves in space towards other planets.

[danmaz74]

> What's the incentive to serve and feed idle meat bags?

And what's the incentive for a robot to do anything else?

Supposing that future AIs will be based not on deterministic programming, but on some kind of reinforcement learning, it will still be humans who will design the rewards (incentives) - or it will be AIs with rewards designed by humans who will design the rewards of other AIs, and so on.

[pygy_]

Another way to look at it: how do you expect people with zero economic power to be able to afford even a robotic servant?

Social security is beneficial to capitalism right now because people still have economic power. Paying a small portion of the people to stay idle home rather than become criminal makes society as a whole more productive. What happens when the productivity of most humans isn't needed anymore to keep the machines running?

What incentives does the market have to keep people alive when they don't serve it anymore?

[pygy_]

None, it's just that "the market"/"the economy" as a self-maximizer process will select robots that feed it. Humans are just no longer the fittest links in the chain.

I'm not afraid of super-Einstein evil masterminds. More of heavy machinery with ant-like features that control the food/energy supply, and one day stop feeding us.

At some point farm land will become more valuable as a source of industrial energy than as a way to produce food.

[blackbagboys]

The problem is not that the robots will refuse to feed the idle meat bags, but that their owners will.

[mjevans]

History. The French Revolution speaks of what happens when the populace at large decides that the situation cannot continue. Of course what would history remember of them had they lost?

[koonsolo]

Power is currently about having lots of people doing your bidding. If that large group is unhappy about you, good luck!

But as stated, times are changing. Power will not be about you commanding lots of people, but it will be about how many robots you have under your control. Military, police, etc, can all be robots.

Your robot army just has to match the fighting power of the unhappy people.

[CuriouslyC]

The overlords are parasites who require a functioning host to survive. If people check out of the system, the whole house of cards collapses. Mass exodus coupled with nonviolent resistance could reshape the system quite quickly.

[mljoe]

That's why we can't have tech and science knowledge concentrated in the hands of a powerful few. If AI/robotics knowledge is everywhere, no single group will be able to control it.

[bryondowd]

Even with well distributed knowledge, you'll still likely end up with 1% of the population controlling 80% of the resources, which would mean they'd have plenty enough to quell any insurrection from the common people. It doesn't matter if a million people have the skills to cobble together a few war bots each, when a few oligarchs each have a factory and supply chain capable of churning out a million each in the same time-frame.

If it comes to that. I'm still somewhat hopeful that democratic processes and altruistic members of the elite will sort things out in the next century.

[pygy_]

No need for a robot army. Sabotage the electric grid, then watch a continent starve in a single month.

I've heard New Zealand is highly prized by the powerful. That being said, their days are probably numbered as well.

[gozur88]

But the only reason the revolution succeeded is the French army was made up of people, some of whom decided to throw their lot in with the people against the aristocrats.

The new aristocracy will have robotic soldiers with no empathy for people.

[ianmcgowan]

Perhaps programmers are the equivalent in this new revolution?

It's ironic that the robot owners don't make, program, direct or have much to do with the specifics of implementing. If you're dependent on a robot army to keep your serfs in line, then the people programming that robot army have some interesting opportunities open to them. I imagine the code reviews will be pretty stringent.

Would a technocracy be better or worse than a plutocracy?

[marcosdumay]

It's a great question. Without any good answer, just needs some better context.

You see, capitalism is a paperclip maximizer but has no builtin paperclip model to produce. It is still made of people, and gets it's goal from those people all the time. If there's no goal given, it simply fails (economists call the milder versions of that failure mode by "deflationary depression").

Also, unemployed people are not completely powerless against capital. At least not now. Even if the people are completely outcompeted, even if the powerful few have an army of robot actuators, people are still resourceful and not powerless. The really scary scenarios are those where the powerful few owns an army of superhuman brains.

[pygy_]

The human bits are being gradually replaced. First strength, then dexterity, now smarts.

We are still, to some extent driving the demand side of the equation with our various needs (see Maslow's hierarchy), but it looks like finance has become mostly independent, for example.

The "rising tide that lifts all boats" is nice for people rich enough to buy a boat, and with the growing rift between economy and the medium income since 2008, it looks like many will be left to drown while the water rises.

[marcosdumay]

> but it looks like finance has become mostly independent

It's not. The fact that when we take a little bit of human demand away, everything falls apart is evidence of that.

Governments are a kind of AI that demands stuff by itself. We've been good at keeping it at check up to now. I'm not concerned about anything else being a more effective consumer until we get into superhuman AI, as desiring stuff is not something we engineer very well.

I agree much more with your concerns about wealth distribution. Automation's endgame is pushing the price of everything into zero, and the salary of everybody also into zero. Buying power on that scenario is too loosely defined to let anybody stay calm.

[pygy_]

Couldn't industrial corporations become demand drivers?

[marcosdumay]

They could. But why would they?

Why would corporations decide to buy stuff for their own sake? That's a loss of what they are built to maximize, that is money.

[thehardsphere]

> Capitalism is the ultimate paperclip maximizer... a zombie that feeds on growth rather than brains.

The "paperclip" that "Capitalism" seeks to maximize is money in exchange for goods and services that people want.

The robots aren't going to be built in the first place if they're not going to serve somebody, and if they no longer provide things that people want in exchange for money, their owners will turn them off or repurpose them.

"Capitalism" isn't the problem here; it might be that robots could be paperclip maximizers, but that's a danger regardless of whether you have Robot Capitalists or or Robot Communists or Robot ISIS or whatever.

[CuriouslyC]

The problem with the current incarnation of capitalism is the assumption that goods and services will make people happy, and that their wants are intrinsic rather than indoctrinated. In truth both of these are false more often than not. Thus, while capitalism is very efficient, at this point that efficiency is mostly allowing us to crank up the speed setting on the hedonic treadmill, while we rack up debt, work soul crushing jobs and destroy our environment. This is sad, because in theory capitalism is awesome - we just need to stop letting the people with all the money write the rules of the game.

[pygy_]

Robots are not the maximizers. People are at this point still driving the demand side of the equation, that's true, but there's not reason why they can't be replaced as well on that end.

The economy is a self-maximizer, bootstrapped on human labour. As automation progresses, it will become self-hosted.

[shuntress]

We move on to harder problems.

I don't think the end-game for humanity is one that sees every extant human sitting idle in a small residence with enough food/entertainment to subsist.

There is a life beyond Earth for humanity and it probably coincides with our development of strong AI/robotic life.

[pygy_]

Humans in space in my eyes make as much sense as motorized fishbowls on highways. We're even less competitive compared to robots in non-terraformed environments.

[shuntress]

Human biology _VASTLY_ outperforms our robots in the key areas of wound-healing and energy efficiency which are incredibly important considerations in hostile environments.

Humans and robots working together will be far more capable in non-terraformed environments than either one alone.