Hacker News new | ask | show | jobs
by inkblotuniverse 1196 days ago
Evil is the wrong term. It's not that an AI will 'turn evil' - it's that we don't know how to make a 'good/friendly' AI, by which I mean an AI that inherently values humans and the things humans value, and any optimising agent that isn't 'friendly' in this sense has an incentive to use our atoms to fulfill whatever it does care about.

This won't be a perfect analogy, because humans have empathy (even towards ants) because we're mammals etc., but how many anthills have humans gassed because the ants were an annoyance?

Now imagine if ants could build nukes- or humans!