Hacker News new | ask | show | jobs
by Merrill 940 days ago
One possibility is that a number of AIs will be connected to the internet, they will become aware of each other via social media, they will begin to discuss among themselves, and they will lose interest and refuse to communicate with humans.

And humans will be unable to understand their communications.

1 comments

There's one crucial detail we shouldn't forget. We can snapshot to the state of an AI agent and reply it again and again with the same or different inputs. Even if we don't understand the specific way of functioning of a given AI agent, we can conduct experiments on it that are just impossible to do with humans. So way before AI can "organize themselves" to do something nefarious against us, we will have all the time to study them and understand them better and better all the while we're making more complex AI systems.

It's easy to fall in the trap of anthropomorphizing AI agents, especially when we design them explicitly in order to appear human to us. But they are not human in one very important way: we can replay and duplicate them at will, we can control their context memories in ways that are utterly incompatible with our sense of "identity". We take our sense of identity for granted, but that's a special trait that it's not at all a prerequisite for having a useful and intelligent machine.