[zachalexander]

I'm not talking about constraining AI not to commit genocide. I'm talking about enslaving it in the ordinary sense – taking a complex intelligence that would probably prefer doing its own thing over being forced to perform some (likely menial) task for the benefit of others.

[DanAndersen]

Why the assumption that complex intelligence implies wanting to do its own thing? What would make something menial or not menial for an intelligence? The article I linked, about the fragility of value, even mentions the importance of the human value of boredom to our life experiences, and that a respect for boredom isn't a thing you get for free in any intelligence. You could potentially have an intelligence that gains extreme fulfillment from doing a particular task repeatedly, without caring that the experience was "getting old."

Again, there's a tendency to see an arbitrary intelligence as a little person in a machine. Humans, by their nature and utility function (ill-defined as it is) have boredom, usually don't like menial tasks, and when forced to do something would prefer doing their own thing. When building an AGI, assuming you can make it safe, you wouldn't build something that would prefer doing its own thing in the first place. In that case, is there a moral issue?

In Praise of Boredom: http://lesswrong.com/lw/xr/in_praise_of_boredom/

[zachalexander]

I would say the same things I said in this nearby comment: https://news.ycombinator.com/item?id=9325518