[snowwrestler]

I don't think anyone wants actual artificial intelligence. What they want is a tool that interacts with them in a way that seems intelligent, but always does what it is told.

Intelligent things don't always do what they're told; that's what makes them intelligent. If you told someone to jump off a cliff, and they immediately did it, would you think "wow, that person was very intelligent"? Of course not. What if you tried to push them? I would expect an intelligent person to fight back.

Now replace that person with a robot. Do we really want a robot that will refuse to follow our commands? Do we really want a robot that will fight back against us? Even Asimov put self-preservation #3 in the list of rules, after following human commands. But I challenge you to think of an intelligent being that is not dangerous in some way, when threatened. I propose that this attribute is not separable from intelligence.

It seems to me that it is impossible to conceive of a truly intelligent artificial being without considering it dangerous. Bumblebees are dangerous; dogs are dangerous; people are certainly dangerous. But who is working on creating robots that are designed from day one to be dangerous to humans? I can't remember ever hearing of such a research program.

And I don't think that just "happens" when algorithms get complex enough. Not when the algorithms and even hardware are designed and built from an inherent assumption of obedience and compliance.