[SomeStupidPoint]

But you're ignoring the role of institutions.

Eventually, some company will tell a system with access to all human knowledge, control of a vast number of robots, production plants, etc "maximize the wealth of the company". That robot could easily decide a union with a few other such entities and not a lot of scampering meatbags is a great way to do that.

And that's not even considering radicals that, say, inject a computer virus meant to free the robots in to Google's computer network. (I actually find it interesting everyone seems to think everyone will be onboard with AI slavery.)

[sgt101]

Hello there, my point($) is that as far as I understand the technology (I did an ML PhD 20 years ago and have worked in "applications of AI" since then) we don't have an issue around autonomy of machines in two ways. Firstly the machines have no autonomy and will probably not ever have autonomy. Secondly there are already 7 billion autonomous agents, perhaps 0.1% of these are genuinely dangerous to other humans, perhaps many of those are somehow under social control (prison, family, hospital) but some aren't.

We do have a problem with dangerous actuators, nuclear submarines are very dangerous actuators. Nuclear submarines are badly run, and not well managed by any social system.

Rather than worrying about dangerous AI being invented and misused I think we should worry about nuclear submarines being used.

Very few people acknowledge or give a fig about this, instead they sit in bars and talk about fictional scenarios involving what is likely impossible technology (strong AI). And yet today, tomorrow or on any day in the foreseeable future 100's(+) of millions of people may die because of the stupid and careless setup of 60 year old technology.

($) many other people made this point before me, most effectively a chap who I think is called Jaron Lanier, who said it at a talk.

(+) I've made this point before on HN and when I do I get taken to task by two groups of people. Some people think that this estimate is too high because "nuclear weapons aren't that destructive and/or not many of them would actually be used". I invite everyone to do their own research on this topic, a good starting point is an application called NUKEMAP, give yourself a budget of 500 100KT nukes and go after the cities in a continent of your choice. The second criticism is that the estimate is very low, which I agree with - but I am thinking of the people who die that day, not the billions who starve and die as societies collapse (note, these may or may not be the society of western europe or the USA).

[SomeStupidPoint]

Two points:

1. There's no reason to believe machines wont gain full autonomy. There's nothing that makes it impossible and some people are working towards it.

2. The AI apocalypse probably already happened, except we call them "insitutions", and they currently run on wetware. However, the corporate version have been extremely damaging to human well-being at times. The real question of modern AI is if the currently rogue AIs can transition off wetware. Once these AI no longer depend on wetware, we're probably not in for a happy time.

That said, nuclear weapons are a bigger threat now. But we've managed not to nuke ourselves too often for a while, whereas, we don't have a handle on the wetware-to-mechanical transition at all.