| Here's my rebuttal: On limitations of self-improvement: The very nature of AI is self-improvement. AI becomes smarter with more data. Thus, the only thing it needs is more data. On the role of human civilizations: If you give an AI a goal, it will try to achieve that goal, including trying to become smarter in order to achieve that goal. Humans also have goals - to survive and reproduce. Furthermore, I constantly see the argument that AIs don't manifest itself in physical form as a reason it can't improve itself. This seems extremely short sighted. We're about put AI into every single thing. We're going to have AI robots, AI dogs, AI cleaners, AI toasters. The physical form is not a long-term barrier for an AI to self-improve. On the importance of collaboration: I don't see how this prevents an AI from improving itself. Let's say an AI model is as intelligent as Albert Einstein. This AI can simply spawn many copies of itself. The AI will allocate 1 million Einsteins to study different areas of physics. 1 million Einsteins for chemistry. 1 million Einsteins to learn human persuasion. Etc. Now they can all come together and share ideas. On the challenge of generality: Doesn't ChatGPT prove that an AI can have general intelligence? On the role of human decision-making: Sure, the foreseeable future, tech will still be driven by humans. But the author argues that the singularity will never happen and computers won't make themselves smarter. It sounds like the author actually convinced himself that machines could make themselves smarter, just not yet? And finally, on this quote from the author: >We’re a long way off from being able to create a single human-equivalent A.I., let alone billions of them. If we can create one, creating a billion will be the easy part. |