All advertising is emotionally manipulative, but just because I bought something that was advertised to me doesn't mean that I consider it 'the best possible purchase'.
It could have been the purchase of least resistance. It could have been the best at maximizing some particular utility. It could have been a purchase of the most recent thing that was on my mind. My decisions, unlike that of these theoretical EAs don't solely consist of maximizing some global 'goodness' utility function.
When I gave $500 to the town's theatre this year, it wasn't because 'I was trying to maximize the best way to help people, and I erroneously made a sub-optimal choice, because I based it on feelings'.
Paperclip-maximization was the furthest thing from my mind at the time. Anyone looking at my decision, and concluding based on it that I was buying into some Big Lie is completely nuts.
It could have been the purchase of least resistance. It could have been the best at maximizing some particular utility. It could have been a purchase of the most recent thing that was on my mind. My decisions, unlike that of these theoretical EAs don't solely consist of maximizing some global 'goodness' utility function.
When I gave $500 to the town's theatre this year, it wasn't because 'I was trying to maximize the best way to help people, and I erroneously made a sub-optimal choice, because I based it on feelings'.
Paperclip-maximization was the furthest thing from my mind at the time. Anyone looking at my decision, and concluding based on it that I was buying into some Big Lie is completely nuts.