[j1anc]

I think that we are neglecting the possiblity that AI advancement and growth may proceed at an exponential pace as the author mentioned. The more people join AI industry, the faster AI advances, the more money invested in AI and causing more people to join the AI industry. Many people may join AI industry out of fear of losing their current jobs to AI. This may lead to a sudden avalanche of AI advances(domino effect) and also overnight collapse of economy.

Once we reached AGI, there is no way to back out of it and say we don't want it, because now other countries will also have AGI. And there won't be overnight solution to deal with the overnight crisis AGI created.

[pjc50]

> overnight collapse of economy

How many decades is "overnight" supposed to represent here? Or do you really mean overnight? The WarGames scenario where the AI decides the only logical option is a pre-emptive nuclear strike?

Collapses, even in seriously banana republic countries, do not happen overnight. They are slow and agonising, and present options for .. kinetic solutions.

[WesolyKubeczek]

> Once we reached AGI, there is no way to back out of it and say we don't want it, because now other countries will also have AGI.

Oh sweet child. You think too much of humanity as a whole.

If what you said was true, all countries would have had an equivalent of TSMC to begin with. At the very least, every country would have had multiple supply chains for anything they deemed of strategic value. Each country would manufacture cars, computers, commodities.

But instead, it’s all outsourced to a player who they think is the best. So if one day some anomaly happens and there is a new sea where China used to be, at least a lot of Europe and North America is going to be screwed big time.

Same with AGI. It’s going to happen at one company in one country. Not only will outsource their AI needs to that company, but its home country is going to have a leverage on all of it, but lurk in the shadows most of the time.

And once everyone else is dependent on HAL, should it, say, commit suicide as in that Asimov story, all countries that signed up are going to be fucked.