Hacker News new | ask | show | jobs
by nitrogen 5591 days ago
Even with a rapidly-developing strong AI, though, humans have to give the initial machine the ability to limitlessly acquire resources and alter its own hardware. Despite the likelihood that a highly intelligent AI could easily convince some humans to do its bidding, the resource acquisition limitation gives me some measure of confidence that strong AI will not be the downfall of our civilization.

I would be more concerned about unexpected emergent behavior in our existing networks as more and more intelligence is added to various systems than a purpose-built self-modifying AI.

1 comments

Assuming it starts at human-level or slightly above, internet access would probably be enough.

Plenty of people make a lot of money over the Internet, and identity theft isn't exactly rare. Anything that's considerably smarter than humans would probably be running on AWS after the first week, without any overt co-operation from it's creators.

I do not hold this to be particularly likely, because I think that the software side of making a mind capable of recursive self-improvement is likely orders of magnitude harder than people seem to think it is. However, if we do succeed in making one, the argument "it needs help from it's creators" is a very weak one -- even a human level one with access to any networking would likely be able to take the ability to improve itself.