[sgt101]

There is an ontology issue here - when people think of Robots now I think most people think of CP30 and R2D2. These are actors in a drama, their decisions and interventions change the flow of the story. Robots in the AI sense (as in technology not magic or fiction) are not actors in this way, they are... Robots! They have a very limited type of autonomy, they can choose from sets of options to attempt to achieve a goal, and can choose from a set of goals that are pre-provided depending on the current context. However I don't think that there are agents or robots that are able to develop and define their own goals.

Based on that I think that the Robots will always be acting on orders - our orders - outside fiction. So the incentive will be "because we are told to do it".

Of course some people may tell them to do other things - like "go kill those people" but we have that issue in spades with things like the Trident D5 robot which is highly purposed to "go kill many people" on demand.

[SomeStupidPoint]

But you're ignoring the role of institutions.

Eventually, some company will tell a system with access to all human knowledge, control of a vast number of robots, production plants, etc "maximize the wealth of the company". That robot could easily decide a union with a few other such entities and not a lot of scampering meatbags is a great way to do that.

And that's not even considering radicals that, say, inject a computer virus meant to free the robots in to Google's computer network. (I actually find it interesting everyone seems to think everyone will be onboard with AI slavery.)

[sgt101]

Hello there, my point($) is that as far as I understand the technology (I did an ML PhD 20 years ago and have worked in "applications of AI" since then) we don't have an issue around autonomy of machines in two ways. Firstly the machines have no autonomy and will probably not ever have autonomy. Secondly there are already 7 billion autonomous agents, perhaps 0.1% of these are genuinely dangerous to other humans, perhaps many of those are somehow under social control (prison, family, hospital) but some aren't.

We do have a problem with dangerous actuators, nuclear submarines are very dangerous actuators. Nuclear submarines are badly run, and not well managed by any social system.

Rather than worrying about dangerous AI being invented and misused I think we should worry about nuclear submarines being used.

Very few people acknowledge or give a fig about this, instead they sit in bars and talk about fictional scenarios involving what is likely impossible technology (strong AI). And yet today, tomorrow or on any day in the foreseeable future 100's(+) of millions of people may die because of the stupid and careless setup of 60 year old technology.

($) many other people made this point before me, most effectively a chap who I think is called Jaron Lanier, who said it at a talk.

(+) I've made this point before on HN and when I do I get taken to task by two groups of people. Some people think that this estimate is too high because "nuclear weapons aren't that destructive and/or not many of them would actually be used". I invite everyone to do their own research on this topic, a good starting point is an application called NUKEMAP, give yourself a budget of 500 100KT nukes and go after the cities in a continent of your choice. The second criticism is that the estimate is very low, which I agree with - but I am thinking of the people who die that day, not the billions who starve and die as societies collapse (note, these may or may not be the society of western europe or the USA).

[SomeStupidPoint]

Two points:

1. There's no reason to believe machines wont gain full autonomy. There's nothing that makes it impossible and some people are working towards it.

2. The AI apocalypse probably already happened, except we call them "insitutions", and they currently run on wetware. However, the corporate version have been extremely damaging to human well-being at times. The real question of modern AI is if the currently rogue AIs can transition off wetware. Once these AI no longer depend on wetware, we're probably not in for a happy time.

That said, nuclear weapons are a bigger threat now. But we've managed not to nuke ourselves too often for a while, whereas, we don't have a handle on the wetware-to-mechanical transition at all.

[pygy_]

I'm not so much scared of "go kill these people". More of "stop feeding them, we need to repurpose arable land for energy production" at a point where we won't have the know how to produce food for everyone by hand, or similar scenarios.

It may still be people at the helm at the time it happens, it doesn't matter. Humans are capable of large scale atrocities, provided they de-humanize their victims. Many people in the .01% have utter contempt for poorer folks, for example.

[nojvek]

To be honest I really think the world population will slowly reduce as more people start getting wealthier.

I also think as soon as you can upload your brain into a robot, people will opt into that and shoot themselves in space towards other planets.