| He does address that point with a quote: > I remember the spring of 1941 to this day. I realized then that a nuclear bomb was not only possible — it was inevitable. Sooner or later these ideas could not be peculiar to us. Everybody would think about them before long, and some country would put them into action.... And there was nobody to talk to about it, I had many sleepless nights. But I did realize how very very serious it could be. And I had then to start taking sleeping pills. It was the only remedy, I’ve never stopped since then. It’s 28 years, and I don’t think I’ve missed a single night in all those 28 years. -- James Chadwick (corrected Chadwich -> Chadwick) That said, I'm not sure the parallel between AGI and nuclear weapons is really that strong. Nuclear weapons are a game-changer on their own in that once you have the warhead and some kind of a delivery mechanism then you can affect events. AGIs are different in that they will manipulate and organise only information and knowledge, not physical matter directly. Aschenbrenner sort-of-considers this point, for example > Improved sensor networks and analysis could locate even the quietest current nuclear submarines (similarly for mobile missile launchers). Millions or billions of mouse-sized situational awareness 130 autonomous drones, with advances in stealth, could infiltrate behind enemy lines and then surreptitiously locate, sabotage, and decapitate the adversary’s nuclear forces. ...but he doesn't really consider the wider issues of, y'know, capturing the additional information that locating those nuclear submarines would require, or the technological jumps in actually creating and manufacturing those mouse-sized drones. Perhaps the AGI is going to solve all those practical and logistical issues, but that really does remain to be seen. |