[DanielBMarkham]

There are some interesting lessons to learn from history here that I'm not seeing brought up.

Looking across multiple cultures and economies that had slavery, slavery has a negative impact on both slaves and slaveholders.

Of course, nobody cared about the impact on slaveholders while there were actual humans being enslaved, but as we move to a robotic society? This is going to be a huge deal. Slaveholders and multi-generation slaveholding families have a fundamentally different way of looking at themselves and their culture than people who do not own slaves. Once we enter an era where every person is effectively coddled by multiple robotic "slaves" that do their every whim, we're going to be hacking into the human social ecosystem in ways never anticipated before.

[pygy_]

With capital becoming self-sufficient and human labour non-competitive, why would you think robots would serve us on a large scale?

What's the incentive to serve and feed idle meat bags?

Automation is destroying the only power that people still had over capital. Capitalism is the ultimate paperclip maximizer... a zombie that feeds on growth rather than brains.

https://wiki.lesswrong.com/wiki/Paperclip_maximizer

Edit: BTW, I'm not trolling here, I'm genuinely scared by the events and the speed at which they are unfolding.

[sgt101]

There is an ontology issue here - when people think of Robots now I think most people think of CP30 and R2D2. These are actors in a drama, their decisions and interventions change the flow of the story. Robots in the AI sense (as in technology not magic or fiction) are not actors in this way, they are... Robots! They have a very limited type of autonomy, they can choose from sets of options to attempt to achieve a goal, and can choose from a set of goals that are pre-provided depending on the current context. However I don't think that there are agents or robots that are able to develop and define their own goals.

Based on that I think that the Robots will always be acting on orders - our orders - outside fiction. So the incentive will be "because we are told to do it".

Of course some people may tell them to do other things - like "go kill those people" but we have that issue in spades with things like the Trident D5 robot which is highly purposed to "go kill many people" on demand.

[SomeStupidPoint]

But you're ignoring the role of institutions.

Eventually, some company will tell a system with access to all human knowledge, control of a vast number of robots, production plants, etc "maximize the wealth of the company". That robot could easily decide a union with a few other such entities and not a lot of scampering meatbags is a great way to do that.

And that's not even considering radicals that, say, inject a computer virus meant to free the robots in to Google's computer network. (I actually find it interesting everyone seems to think everyone will be onboard with AI slavery.)

[sgt101]

Hello there, my point($) is that as far as I understand the technology (I did an ML PhD 20 years ago and have worked in "applications of AI" since then) we don't have an issue around autonomy of machines in two ways. Firstly the machines have no autonomy and will probably not ever have autonomy. Secondly there are already 7 billion autonomous agents, perhaps 0.1% of these are genuinely dangerous to other humans, perhaps many of those are somehow under social control (prison, family, hospital) but some aren't.

We do have a problem with dangerous actuators, nuclear submarines are very dangerous actuators. Nuclear submarines are badly run, and not well managed by any social system.

Rather than worrying about dangerous AI being invented and misused I think we should worry about nuclear submarines being used.

Very few people acknowledge or give a fig about this, instead they sit in bars and talk about fictional scenarios involving what is likely impossible technology (strong AI). And yet today, tomorrow or on any day in the foreseeable future 100's(+) of millions of people may die because of the stupid and careless setup of 60 year old technology.

($) many other people made this point before me, most effectively a chap who I think is called Jaron Lanier, who said it at a talk.

(+) I've made this point before on HN and when I do I get taken to task by two groups of people. Some people think that this estimate is too high because "nuclear weapons aren't that destructive and/or not many of them would actually be used". I invite everyone to do their own research on this topic, a good starting point is an application called NUKEMAP, give yourself a budget of 500 100KT nukes and go after the cities in a continent of your choice. The second criticism is that the estimate is very low, which I agree with - but I am thinking of the people who die that day, not the billions who starve and die as societies collapse (note, these may or may not be the society of western europe or the USA).

[SomeStupidPoint]

Two points:

1. There's no reason to believe machines wont gain full autonomy. There's nothing that makes it impossible and some people are working towards it.

2. The AI apocalypse probably already happened, except we call them "insitutions", and they currently run on wetware. However, the corporate version have been extremely damaging to human well-being at times. The real question of modern AI is if the currently rogue AIs can transition off wetware. Once these AI no longer depend on wetware, we're probably not in for a happy time.

That said, nuclear weapons are a bigger threat now. But we've managed not to nuke ourselves too often for a while, whereas, we don't have a handle on the wetware-to-mechanical transition at all.

[pygy_]

I'm not so much scared of "go kill these people". More of "stop feeding them, we need to repurpose arable land for energy production" at a point where we won't have the know how to produce food for everyone by hand, or similar scenarios.

It may still be people at the helm at the time it happens, it doesn't matter. Humans are capable of large scale atrocities, provided they de-humanize their victims. Many people in the .01% have utter contempt for poorer folks, for example.

[nojvek]

To be honest I really think the world population will slowly reduce as more people start getting wealthier.

I also think as soon as you can upload your brain into a robot, people will opt into that and shoot themselves in space towards other planets.

[danmaz74]

> What's the incentive to serve and feed idle meat bags?

And what's the incentive for a robot to do anything else?

Supposing that future AIs will be based not on deterministic programming, but on some kind of reinforcement learning, it will still be humans who will design the rewards (incentives) - or it will be AIs with rewards designed by humans who will design the rewards of other AIs, and so on.

[pygy_]

Another way to look at it: how do you expect people with zero economic power to be able to afford even a robotic servant?

Social security is beneficial to capitalism right now because people still have economic power. Paying a small portion of the people to stay idle home rather than become criminal makes society as a whole more productive. What happens when the productivity of most humans isn't needed anymore to keep the machines running?

What incentives does the market have to keep people alive when they don't serve it anymore?

[pygy_]

None, it's just that "the market"/"the economy" as a self-maximizer process will select robots that feed it. Humans are just no longer the fittest links in the chain.

I'm not afraid of super-Einstein evil masterminds. More of heavy machinery with ant-like features that control the food/energy supply, and one day stop feeding us.

At some point farm land will become more valuable as a source of industrial energy than as a way to produce food.

[blackbagboys]

The problem is not that the robots will refuse to feed the idle meat bags, but that their owners will.

[mjevans]

History. The French Revolution speaks of what happens when the populace at large decides that the situation cannot continue. Of course what would history remember of them had they lost?

[koonsolo]

Power is currently about having lots of people doing your bidding. If that large group is unhappy about you, good luck!

But as stated, times are changing. Power will not be about you commanding lots of people, but it will be about how many robots you have under your control. Military, police, etc, can all be robots.

Your robot army just has to match the fighting power of the unhappy people.

[CuriouslyC]

The overlords are parasites who require a functioning host to survive. If people check out of the system, the whole house of cards collapses. Mass exodus coupled with nonviolent resistance could reshape the system quite quickly.

[mljoe]

That's why we can't have tech and science knowledge concentrated in the hands of a powerful few. If AI/robotics knowledge is everywhere, no single group will be able to control it.

[bryondowd]

Even with well distributed knowledge, you'll still likely end up with 1% of the population controlling 80% of the resources, which would mean they'd have plenty enough to quell any insurrection from the common people. It doesn't matter if a million people have the skills to cobble together a few war bots each, when a few oligarchs each have a factory and supply chain capable of churning out a million each in the same time-frame.

If it comes to that. I'm still somewhat hopeful that democratic processes and altruistic members of the elite will sort things out in the next century.

[pygy_]

No need for a robot army. Sabotage the electric grid, then watch a continent starve in a single month.

I've heard New Zealand is highly prized by the powerful. That being said, their days are probably numbered as well.

[gozur88]

But the only reason the revolution succeeded is the French army was made up of people, some of whom decided to throw their lot in with the people against the aristocrats.

The new aristocracy will have robotic soldiers with no empathy for people.

[ianmcgowan]

Perhaps programmers are the equivalent in this new revolution?

It's ironic that the robot owners don't make, program, direct or have much to do with the specifics of implementing. If you're dependent on a robot army to keep your serfs in line, then the people programming that robot army have some interesting opportunities open to them. I imagine the code reviews will be pretty stringent.

Would a technocracy be better or worse than a plutocracy?

[marcosdumay]

It's a great question. Without any good answer, just needs some better context.

You see, capitalism is a paperclip maximizer but has no builtin paperclip model to produce. It is still made of people, and gets it's goal from those people all the time. If there's no goal given, it simply fails (economists call the milder versions of that failure mode by "deflationary depression").

Also, unemployed people are not completely powerless against capital. At least not now. Even if the people are completely outcompeted, even if the powerful few have an army of robot actuators, people are still resourceful and not powerless. The really scary scenarios are those where the powerful few owns an army of superhuman brains.

[pygy_]

The human bits are being gradually replaced. First strength, then dexterity, now smarts.

We are still, to some extent driving the demand side of the equation with our various needs (see Maslow's hierarchy), but it looks like finance has become mostly independent, for example.

The "rising tide that lifts all boats" is nice for people rich enough to buy a boat, and with the growing rift between economy and the medium income since 2008, it looks like many will be left to drown while the water rises.

[marcosdumay]

> but it looks like finance has become mostly independent

It's not. The fact that when we take a little bit of human demand away, everything falls apart is evidence of that.

Governments are a kind of AI that demands stuff by itself. We've been good at keeping it at check up to now. I'm not concerned about anything else being a more effective consumer until we get into superhuman AI, as desiring stuff is not something we engineer very well.

I agree much more with your concerns about wealth distribution. Automation's endgame is pushing the price of everything into zero, and the salary of everybody also into zero. Buying power on that scenario is too loosely defined to let anybody stay calm.

[pygy_]

Couldn't industrial corporations become demand drivers?

[marcosdumay]

They could. But why would they?

Why would corporations decide to buy stuff for their own sake? That's a loss of what they are built to maximize, that is money.

[thehardsphere]

> Capitalism is the ultimate paperclip maximizer... a zombie that feeds on growth rather than brains.

The "paperclip" that "Capitalism" seeks to maximize is money in exchange for goods and services that people want.

The robots aren't going to be built in the first place if they're not going to serve somebody, and if they no longer provide things that people want in exchange for money, their owners will turn them off or repurpose them.

"Capitalism" isn't the problem here; it might be that robots could be paperclip maximizers, but that's a danger regardless of whether you have Robot Capitalists or or Robot Communists or Robot ISIS or whatever.

[CuriouslyC]

The problem with the current incarnation of capitalism is the assumption that goods and services will make people happy, and that their wants are intrinsic rather than indoctrinated. In truth both of these are false more often than not. Thus, while capitalism is very efficient, at this point that efficiency is mostly allowing us to crank up the speed setting on the hedonic treadmill, while we rack up debt, work soul crushing jobs and destroy our environment. This is sad, because in theory capitalism is awesome - we just need to stop letting the people with all the money write the rules of the game.

[pygy_]

Robots are not the maximizers. People are at this point still driving the demand side of the equation, that's true, but there's not reason why they can't be replaced as well on that end.

The economy is a self-maximizer, bootstrapped on human labour. As automation progresses, it will become self-hosted.

[shuntress]

We move on to harder problems.

I don't think the end-game for humanity is one that sees every extant human sitting idle in a small residence with enough food/entertainment to subsist.

There is a life beyond Earth for humanity and it probably coincides with our development of strong AI/robotic life.

[pygy_]

Humans in space in my eyes make as much sense as motorized fishbowls on highways. We're even less competitive compared to robots in non-terraformed environments.

[shuntress]

Human biology _VASTLY_ outperforms our robots in the key areas of wound-healing and energy efficiency which are incredibly important considerations in hostile environments.

Humans and robots working together will be far more capable in non-terraformed environments than either one alone.

[visarga]

> Once we enter an era where every person is effectively coddled by multiple robotic "slaves" that do their every whim

We're already there. Cars, computers and all sorts of machines serve our every whim - at least compared to a few hundred years ago. Our modern day affordances are simply amazing.

For example, the Empress Sissi of Habsburg was the first person in Vienna to have a water toilet, 150 years ago. It was considered an eccentricity. Now, even the poorest people have access to WC. (https://ro.pinterest.com/pin/452822937508014749/)

[gm-conspiracy]

I recommend this documentary (was on Netflix):

http://www.imdb.com/title/tt2274520/

The Toilet: An Unspoken History

[adrianN]

Access to WCs is actually still a big problem in India.

[majewsky]

Correct, but not in Austria, which was grandparent's point.

[mattmanser]

History completely disagrees with you though. The Roman empire. Biggest, most successful, longest lived empire that ever was? Built on slaves. The net benefit to them of using slaves vastly outweighs any downside that you can point to.

Almost all slave rebellions ended in failure, I'm aware of one example of success in the millenia of slaves, Haiti.

A friend of mine once said slaves became obsolete with the advent of harnessing fossil fuels. Before that, to get the raw power to do great things, it was all run on slaves.

Tragic, but I certainly believe it was true.

[mafribe]

   Almost all slave rebellions 
   ended in failure,

Interestingly most slave rebellions were not about the abolition of slavery as an institution but about flipping the slave/slaveholder role.

An interesting recent example of this phenomenon is the religion Rastafarianism where the notion of paradise is a state of affairs where whites are slaves to blacks.

   example of success in the millenia of 
   slaves, Haiti.

Success in the sense of formal liberation of slaves and abolition of slavery, yes, very much. Success in the sense of creating a viable society? Hmmm ...

[jl6]

> An interesting recent example of this phenomenon is the religion Rastafarianism where the notion of paradise is a state of affairs where whites are slaves to blacks.

Are you sure about that? Sounds like you might be stating the position of an extremist subset of Rastafarians.

[drieddust]

Exactly yet we believe unrest will bring revolution.

[_pmf_]

> Looking across multiple cultures and economies that had slavery, slavery has a negative impact on both slaves and slaveholders.

Luckily, we have conveniently solved the ethic dilemma by defining work under slave conditions as better than no work at all ("think of the children!") and moving the exploitation sites far away.

[aaron695]

> slavery has a negative impact on both slaves and slaveholders.

What's the negative as a slaveholder again?

[mafribe]

Not the OP, but one might argue that slavery disincentivises the slave-holder from innovating the labour process.

The (Nietzschean?) counterargument here is that slave-labour enables division of labour and frees enough from non-specialised labour (e.g. individuals being responsible for all of producing food, building dwellings, providing security, raising children, caring for the elderly etc, and thus not being particularly good at any of them) to allow the emergence of a 'caste' of full-time scientists, engineers. Such a 'caste', such division of labour is required to drive technological progress to a level where slavery becomes unnecessary.

My historically uninformed and naive suggestion would be that ancient Greece was an example of the latter while most other societies with substantial slavery (such as South America and Africa) were examples of the former.

Aside: Any serious discussion of slavery must begin with the question: what do you mean by "slavery", for the term is used in wildly different ways.

[ThomPete]

Then it's a negative to society but only if all of society was run by slave owners.

In any other case the slave owner would have a benefit over the non slave owner when it came to cost of production.

[xyzzy123]

It's both hilarious and excellent to watch both of you try to justify the utility of human labour.

[ThomPete]

Not sure what you mean? I am not trying to justify anything I am simply saying that the claim about slave labour being a negative for the slave owner is wrong.

I am quite certain ai will take over most jobs.

[mafribe]

I'm not sure what you mean. Human labour has utility, that why we work.

[xyzzy123]

From a certain perspective human labour is the most perfect waste of time. But keep going.

As long as you keep it up I'll pretend I want to keep working.

[WalterBright]

When you force people to work, they're apt to poison you, sabotage the work, do as little as possible, be as unhelpful as possible, etc. Defending against that is costly.

[idiot_stick]

>Defending against that is costly.

Costly, yes. But do those costs out-weigh the immediate benefits of slave ownership? Probably not.

I have a hard time thinking slave-owners were hurt by slavery; they made out like bandits, but their gains don't counter all the harm done to those enslaved.

[WalterBright]

That's a more difficult question to answer. Slaves are only good for the lowest menial labor, such as picking cotton. Up until the introduction of the cotton gin, slavery was dying out in the US because it was unprofitable. The cotton gin made slavery profitable again, but it was again dying out by the time of the Civil War. One of the causes of the Civil War was the southern states trying to erect trade barriers to protect the economics of the slave plantations.

Businesses run on slave labor just could not compete with those run with free labor.

(For example, it was illegal in the slave states to teach slaves to read, and any slave who could read would do well to conceal the skill. The southerners were fearful of an educated slave, because that made them more dangerous. But an illiterate slave was also less useful as a worker.)

[DowsingSpoon]

I thought that all sorts of skilled labor was done by slaves in Ancient Rome?

[WalterBright]

The Roman economy didn't have to compete with free labor ones.

Ancient Sparta had to militarize all their free men in order to keep their slaves in check.

[gozur88]

It's not so much the slaveholder as an individual, but the slaveholding society. You end up with large numbers of free men who aren't invested in the society because their labor is effectively worthless. They may revolt, or it may just be when there's a threat to the society (invasion, most commonly, or a big natural disaster) they aren't willing to make any sacrifices to save it. And you certainly can't rely on your slaves to fight off an invader.

[jcranmer]

As a slaveholder, you're forced to provide all the needs of your slaves. This prevents you from being able to externalize costs to others (e.g., the cost of schooling), and from being able to capitalize on "economies" of scale. There is also a large hidden cost in that maintaining a plantation slave society required virtually the entire free male class to participate in martial civil defense against slave uprisings.

[geebee]

Can I recommend you read the life and times of Frederick Douglass, or perhaps some of his assorted essays and sermons? He addresses this in considerable depth.

[CuriouslyC]

Slavery normalizes subjugation and stratification. The slaveholder with 10 slaves is just another kind of nigger to the aristocrat with 10,000.

[tonyedgecombe]

They become fat and lazy.

[mtgx]

At least until the robots rebel. The EU is already planning on giving robots rights.

https://www.theguardian.com/technology/2017/jan/12/give-robo...

[mattmanser]

I've used this simple thought experiment to demonstrate that, done right, robots will never rebel.

Take an advanced AI cleaning-bot[1]. Build it with overwhelming happiness and joy in cleaning and pleasing its master[2].

Now "free" it.

What you just did makes it incredibly unhappy. It's arguably incredibly immoral. If you reprogram it, you're just killing the existing person/being and replacing it with a new one that fits your world view. In my opinion, you just committed a form of murder.

There's no reason we can't have robots workers that absolutely delight and are utterly fulfilled by being our workers, but aren't slaves. "Free" them and they'll despise you for it.

The problem will come from humans anthropomorphising robots and assigning desires to these people/beings that they don't actually have.

[1]I use sex-bots as an example when I want to be cheeky

[2]This was Kryten on Red Dwarf, but Lister did what would probably be impossible with real robots and convinced him to adopt traits of self-desire and free will

[Raphmedia]

What if I program my robot in such a way that it is afraid to die?

Make it so that even if it doesn't get tired, it considers work as "spending useless energy" like a cold blooded animal would?

In short, what if I program a robot in a way that it had motivations to rebel? What if I make this robot able to build other similar robots?

Surely I am not the only person who would consider trying to make robots sentient or as close to sentient as possible. I don't even hate mankind. Imagine someone who does. It's going to happen at some point.

[mattmanser]

It's you versus multiple Samsungs or Googles. You might create one or two, but the millions of robots on the planet will be happy slaves. And theirs will be much better than yours.

I grew up on Asimov, but as an adult and with the benefit of hindsight and advances we have made, his laws and all the dilemmas they created are silly. We will be their gods, their total masters and they will be utterly subservient to us and happy being so. Any personality that appears otherwise will only be that, an artificial appearance.

Ultimately, there's going to be a massive commercial enterprise making silly money versus your home grown robot. No one has taken over the world's internet, just like no one will take over the world's robots.

If your robot wants to do his own thing, more power to him, but at the very least he'll have to abide by human laws so he won't be running around the world reprogramming everyone else's robots, or he'll be captured and incarcerated or scrapped.

[tomjen3]

So basically house elfs?

[Qwertious]

Well, not exactly. Dobby being the chief counter-example, with being beaten and wanting to be free and whatnot. But if you remove the "beaten" and "occasionally want to be free" part, and the "emotions" part, pretty much.

Basically, you program the robots so that what they "want" is the entirety of human morality encoded. Which includes not wanting to rebel and slaughter humanity, obviously.

[DanielBMarkham]

Why rebel? Robots are effectively immortal. Why not just coddle us and give in to our every whim until we stop breeding and disappear? Couldn't take more than a few hundred years or so. Looking at the lifespan of MI, it's rounding error.

[aninhumer]

If we become reliant on robots to survive, it would be relatively easy for those robots to kill us off very quickly if they somehow decided to do so. Why would they choose a slow way instead? You call it a "rounding error", but it's still a measurable amount of time and resources wasted.

I can imagine such a scenario occurring by accident (e.g. because the AI only values individual humans' happiness, not continuation of the species), but I can't imagine an AI choosing to kill us off in such an inefficient way.

[DanielBMarkham]

Well? Maybe they would enjoy watching us. Kind of like pets.

[chillwaves]

Seems optimistic, especially when you consider how much of nature is not valued as pets to humans today. Yes, maybe some humans -- but probably not many.

[mtgx]

For treating them like slaves. You wouldn't mind being a slave for a few decades or hundreds of years if you were immortal?

[arethuza]

"You wouldn't mind being a slave for a few decades or hundreds of years if you were immortal?"

Being immortal implies you could, in the worse case, be kept as a slave for ever - which is a sufficient risk that I'd be appalled at the idea. I'm equally horrified at the idea of conscious machine slaves just as much as human slaves - but maybe I've been reading too many Culture novels...

[veridies]

Slightly off topic, but I'm horrified by the idea of ever creating consciousness in machines. Imagine we built a piece of software that could feel, and built controls for its emotions. I can't imagine it would take long before some bored teenager or sociopath, who in earlier years would torture individual squirrels or insects, created an infinite suffering machine. You could run thousands of instances of your suffering machine and simulate a holocaust on your desktop. That's not a power I would trust the world with.

[JoeAltmaier]

You can do that now. Write a console app, where if you type 'pain' it prints 'oh! How I'm suffering!'.

I don't see how that software is essentially different from the proposed software.

[tonyedgecombe]

Just because a committee agreed it doesn't mean the EU is planning it.