Why? Because this AI, which is basically just people smarter than us and getting smarter by the day, will decide that its only purpose in life is to serve its idiot masters?
Because if we build it correctly, it won't be a person at all, and won't have a value function orthogonal to our own. Building something that can self-improve and solve problems doesn't mean we need to build something that has independent will, motivation, or values. And it's a lot safer not to.
Sure, but "an AI that's smarter than humans" implies that. I'm sure we'll build many algorithms that are better than us at specific tasks, and we already have.
Even building algorithms that are better than us at any general-purpose task still doesn't require giving those algorithms agency or independent values of their own.