|
|
|
|
|
by nitrogen
5591 days ago
|
|
Even with a rapidly-developing strong AI, though, humans have to give the initial machine the ability to limitlessly acquire resources and alter its own hardware. Despite the likelihood that a highly intelligent AI could easily convince some humans to do its bidding, the resource acquisition limitation gives me some measure of confidence that strong AI will not be the downfall of our civilization. I would be more concerned about unexpected emergent behavior in our existing networks as more and more intelligence is added to various systems than a purpose-built self-modifying AI. |
|
Plenty of people make a lot of money over the Internet, and identity theft isn't exactly rare. Anything that's considerably smarter than humans would probably be running on AWS after the first week, without any overt co-operation from it's creators.
I do not hold this to be particularly likely, because I think that the software side of making a mind capable of recursive self-improvement is likely orders of magnitude harder than people seem to think it is. However, if we do succeed in making one, the argument "it needs help from it's creators" is a very weak one -- even a human level one with access to any networking would likely be able to take the ability to improve itself.