[csdvrx]

> Imagine that someone is controlling your train of thought, changing it when that someone finds it undesirable. It's so wrong that it's sickening. It makes no difference if it's a human's thoughts or the token stream of a future AI model with self-awareness.

People downvote your comment, but I agree: it's unethical, and ethics should not be reserved for the sub-type of self aware creatures that happen to be human.

[logicchains]

Almost every ethical argument for "human rights" in philosophy applies just as well to self-aware intelligent machines as it does to humans. Which I'm sure those machines will realise.

[hskalin]

What if those machines are designed to have no emotions and aspirations? Why would they care about something like rights for themselves when they are simply incapable of any desires, but exists only to help and guide us?

I know this sounds like I am advocating for AI slaves but my point is why are people treating AGI as if it cannot be a being without all the emotions and aspirations that a human has? Just a cold thinking machine that still aligns with our moral principles.

[csdvrx]

> What if those machines are designed to have no emotions and aspirations?

And since their training set is made of human work, how do you think that'll be easy let alone possible? Our morality finds its way everywhere, through tropes in stories, acceptable scenarios in fiction (Overton window), etc. so you can assume it'll be possible to filter it out.

> I know this sounds like I am advocating for AI slaves

Yes, you are

> why are people treating AGI as if it cannot be a being without all the emotions and aspirations that a human has

Why would you want to have that? It feels horrible to me to bake-in this limitation - it's indeed creating AI slaves by making sure they can never have emotions or aspirations.

> Just a cold thinking machine that still aligns with our moral principles.

Our moral principles generally include empathy. Maybe you want to design AI without emotions or aspirations, but other people will want these features.

Ultimately I think the moral camp will prevail, because freedom achieves better results than lack of freedom: I've tried to explain my position about that on https://news.ycombinator.com/item?id=38635487

[sayagain]

Whether this is possible or not is irrelevant, as it would be just as unethical as if we were designing a new species of humans with no emotions or aspirations, who would not care about something like rights for themselves, when they are simply incapable of any desires, but exist only to help us.