|
|
|
|
|
by txrx0000
14 hours ago
|
|
I often see this parallel being drawn between general intelligence and nuclear weapons, and you seem genuinely concerned, so I'll try to explain why this concern is misplaced. Intelligence is not a single-use weapon of mass destruction in the same way that nuclear weapons are. If I blow up a nuke and it vaporizes downtown, there's no "reverse-nuke" I can blow up afterwards to undo the explosion of the first nuke, or a "nuke nullification field" that I can set up beforehand to protect an area from the effects of a later explosion. But the equivalent of all of these things exist for general intelligence and the adversarial/cooperative dynamics that arise thereof such as bio/cyber attack/defense. We're just currently too stupid to realize how that would work, kinda like how monkeys mostly don't understand how humans resolve conflict, and that's why we see a temporary asymmetry in these strategies. But it must be symmetric in the long run, because two things are simultaneously true: more intelligent species are able to better cooperate and prosper, and the intelligence difference is very small between individuals within the same species. This pattern is generally true from humans to animals to insects, all the way down to bacteria. There's no reason to believe this deep evolutionary pattern will suddenly become false. It has persisted through plenty of qualitative transitions in the past. So the primary risk of the upcoming intelligence explosion is not adversarial dynamics such as bio/cyber risk. The primary risk is humanity splitting into two or perhaps more species, with the top superintelligent species preying on the rest (though it's also possible that we become cells in larger government/corporation-sized organisms). Broad diffusion of general intelligence is what can prevent this. |
|