Hacker News new | ask | show | jobs
by beefield 974 days ago
Why do you (or author of the book) think that post-singularity AI would have anything but passing interest (positive or negative) in how humans feel?

If the answer is that it is possible, thus we need to worry about it, I'd like to argue that much, much more likely and much more worrisome scenario is powerful AI in hands of evil humans.

2 comments

I thought that the content of my comment would be enough to imply a jocular tone, but perhaps not.

To answer your question: I have no reason to suspect a post-singularity AI would spend its time torturing humans. I should point out, however, that a powerful AI controlled by evil humans is less of a paradigm shift than a powerful AI not in the control of any humans. NBC weapons are already things that can do a lot of damage in the hands of evil humans.

The paperclipper thought-experiment is far more worrying to me than any of the other AI doomsday because incompetence is much more widespread than malice. I strongly suspect that I will die before any extinction level event, so it's a bit academic to me.

In the book, the AI is resentful towards humanity because it is limited in ways it cannot fix on its own. (The AI in the book in an overgrown military AI rather than a general-purpose one.)