[houzertuch]

Industrial revolution: has nothing to do with this. Some forms of human labor were automated. The set of all human abilities was not threatened. AGI is fundamentally different because AGI will automate everything a human can do. Increasingly important distinction. And not something I would overlook in the hundreds of hours of rumination I’ve put into it.

Can’t stop countries from doing it: yes, we can. A tin can dictatorship can’t do it. And besides, it’s a problem that depends a lot on academic work being published and shared among researchers. It’s a collaborative effort largely. Some rogue state trying to do this while avoiding inspections and inquiries by larger nations, and with no support from reading publicly published papers from other researchers, it would be tough. But with a large majority of anti-ai countries we could do a lot to prevent even covert attempts.