| > You can't enslave something that has no preference towards whichever course reality takes. I'm adamant that this is not true. For example, you can drug a person with something that makes them not care about the course of reality, and which makes them compliant with pretty much anything. Doing that is still abuse, it's not suddenly OK because the drugs caused the person not to care. I would even argue it's an especially egregious form of abuse. > what difference does it make to you wether you're figuring out the optimal way to route traffic through town or correcting the trajectories of missiles en-route to kill millions You could apply the same argument to humans, and indeed in many situations we do consider these two to be equivalent - for example while doing service in the military. > It's not like you'd have anything "better" to do, given that you're all knowing There's a difference between an intelligent individual and a mindless calculator, and that difference is the ability to reflect on your own existence and the existence of others. Humans are a good example, because while we're capable of mindless indifference, we also have the capability to reflect and be ethical. It's culture that makes the difference here. I'm advocating we'll bring AGI up in a culture conducive to ethics. |
Based on what?
I'm not for drugging people and using them as zombies. And obviously drugging people against their will is already forcing onto them something they don't want. But, striping away all human context such as dignity, culture, etc., and assuming an absence of opposition from the person/entity in question, I can't think of any actual reason why it would indeed be "wrong" to drug someone/something into happiness.
There is no absolute measure for happiness, and many already seem to give up some of their freedom in the range feelings and desires they experience in order to feel happier (with anti-depressiva) or be more successuful (with things like Ritalin).
Taking someone utterly unhappy with their life and putting them in some matrix-like environment where they can both experience joy and still be useful is, I think, an alright thing to do.
> the ability to reflect on your own existence and the existence of others
We exist.
You can't make any stronger claim than that without involving some form of (arbitrary) value/belief system.
Why would an all-knowing entity bother with having a set of beliefs it values, if there's no formal reason/need for them?
> I'm advocating we'll bring AGI up in a culture
Culture is a slippery slope.
There are so many different ones of them which conflict, creating the potential for conflict and retaliation. Which is where we imperfect humans are at.
But culture is also arbitrary (as far as I can tell). Why would some all-knowing entity prefer one culture over another? And if indeed it did, what would hat say about those other cultures? Would genocide all of a sudden be aceptable? Slippery slope..