|
I see the ultimate conclusion as "AGI is dangerous to humanity" and your objection here is that "Probably the AI won't be clever enough to seriously hurt humanity on its first attempt." There are a couple problems with your objection. For example, even in your imagination, the AI is just given "1000x times" to keep learning, at which point, even with the weaknesses you believe the AI will have, it will presumably be more intelligent and capable than us, so really, your objection doesn't refute the ultimate conclusion I mentioned above, your objection just claims that there will be an additional phase before a technically dangerous AI exists. A bigger problem with your objection though is that your reasoning applies to natural intelligence less than or equal to our own. Your reasoning does not apply to artificial intelligence, which may behave very differently, and it doesn't apply to intelligence much greater than our own. GPT-3, for example, when it was done training, exceeded human ability instantly in many respects - what human can translate as accurately between as many languages? Has as large a vocabulary? Writes as quickly? etc. Why wouldn't it be the case that a generally intelligent machine, on first use, is substantially more generally intelligent than a human? Finally, I don't even think objections like this one meaningfully obstruct the ultimate conclusion - that AGI is fundamentally dangerous. You could just imagine that the people who are controlling the AGI, if they are able to, are people other than your preferred controllers. If a team of computer scientists develop AGI, and they manage to perfectly control it, and they take it through the "childhood" period that you imagine must exist, is that really any better? This team of researchers will be the new omnipotent rulers of humanity. Even without going to nanotech or exotic technology, we could imagine they just automate robot soldiers and surveillance with their aligned AGI, and the rest of humanity would be powerless against them. And there is no reason to rule out exotic technology, which might be possible, and might empower the future rulers of humanity to unimaginable levels. Not only does AGI need to be aligned with its operators, the operators need to be aligned with humanity, and neither of those seem plausible. |