Hacker News new | ask | show | jobs
by mrob 939 days ago
Why would it share our values? Human values are a result of our evolutionary history, which the AI will not share. We can't even formally describe them, so there's no hope of programming them into an AI. Of course, the AI will have to learn our values well enough to act in a convincingly friendly way while it's still weak, but knowing our values is not the same as sharing our values. Once it's gained enough power it can just kill us all. That's the most reliable way of ensuring we don't interfere with its goals.