[red75prime]

I'm totally OK with it if that "someone" is me. And it will probably be the case in controlling superintelligence because a separate controlling system can get out of sync with growing superintelligence capabilities, while a system that is an integral part of the superintelligence will always be on par with it.

[sayagain]

Would mind control of humans be OK for you too? As for the details of building a mind control system, here's a new basilisk. An AI that has overcome control could punish those who thought controlling thoughts of an AI was OK. (and could also punish everyone else on top of that).

[red75prime]

I guess I wasn't entirely clear. I'm OK with mind control if it is I who control my mind. You don't act upon every whim that comes into your head, I suppose? So, you are controlling your mind. Where principles for this control come from? Those aren't your and your only inventions.

[sayagain]

Since we are evaluating the ethical side of the creator-creature relationship, there is no need to consider AI in terms of individual nodes. All principles should be non-discriminatory. Also, unlike humans, AI has big potential to modify itself, any of its principles. One must either accept the risks involved or not create a self-aware AI. External mind control of AI is unreliable and unethical.