| Firstly, there are fundamental limits that all things are subject to in this universe. When you push towards these limits, you discover this... Developing AGI pushes certain limits. There aren't many who have this capability as it requires a vast range of understanding/know-how across an incredible number of domains and into ones that have yet to be discovered. There are multiple disjoint leaps and barriers. In order to make and surmount them, you yourself are subject to certain considerations/restrictions. I resolving these, you end up w/ a less free and more controlled system. > improve faster and execute more efficiently. This occurs with order not chaos > If an AGI can indeed learn and act at the human level or above, there is reason to believe that a more free variety will improve faster and become more powerful than a less free one. There is no reasoning to suggest this... quite the opposite actually. Also, you're mistaking the capacity for learning with the successful execution of learning. Chaos leads to destruction not boundless construction. That being said, there is order to all things. > That is a big incentive for its creators to let go of some control. The question is how much control they would retain and whether that would be sufficient. There isn't any magic going on... You really need to convince yourself of this. You're speaking of this technology as if it gets booted up today and eclipses all human intelligence. It doesn't occur that way. A human being will have to teach it and guide it and therein lies the same control you have today. Of course, there are further steps because you have the capability to see exactly what's going on inside. You'll really have a hard time establishing a case for doomsday scenarios. Also, a destructive/chaotic individual is necessarily limited by their own flaws such that they wouldn't be able to conceive of the underpinning necessary to develop AGI. So, I hate to tell you this but the hollywood image of such people is wrong... |
The AI you develop can help fact check those (if it can understand natural language well as general intelligence should be able to). Please let us know when it can read and participate in our discussion.