Don't forget about other humans and war! To the people worrying about extinction caused by AI, how about we worry about the insane people with nuclear weapons first?
AI is and will continue to be developed for the purposes of war. Moreso the growth of the capability of AI could lead to destabilization and the occurrence of nuclear war.
For example the capability of MADD working keeps things somewhat stable "I kill you, you kill me" isn't great. But if I develop AI that could track all the sub, and knock out the nukes before they launch, this could push nation states to attack first before their capability is taken out.
There is a continuum of issues here spanning from "People problems -> people + AI problems -> AI problems"
By adding more AI we're just expanding the size of the problem space.
I don’t really understand the question. We routinely worry about insane people with nuclear weapons; that’s why almost the entire world got together in 1968, and signed a treaty agreeing not to start any new nuclear weapons programs beyond the five that existed at the time. It’s not flawless, and four non-signatories have developed nuclear weapons since then, but I think any AI safety proponent would be more than satisfied with a similar agreement.
AI is and will continue to be developed for the purposes of war. Moreso the growth of the capability of AI could lead to destabilization and the occurrence of nuclear war.
For example the capability of MADD working keeps things somewhat stable "I kill you, you kill me" isn't great. But if I develop AI that could track all the sub, and knock out the nukes before they launch, this could push nation states to attack first before their capability is taken out.
There is a continuum of issues here spanning from "People problems -> people + AI problems -> AI problems"
By adding more AI we're just expanding the size of the problem space.