With nanoplastics, ocean acidification, global heating, rising sea levels, falling crop yields and the related Holocene extinction at work, I think AI is going to have to play a serious game of catchup.
I'm afraid global warming and plastics in the ocean might make our quality of life much worse, but if I'd had to bet on what risks human extinction, "bad people with superweapons" would be higher, and "autonomous agents a bit smarter than any human that are built to compete with it" higher still, if they were options on the page.
And how reasonable is that we would have autonomous agents a bit smarter than humans in the nearish future? Very low, I'd say, about the same as the odds I'd give a year ago for GPT4? Not odds I'd be happy to take for the _destruction of humanity_.
Don't forget about other humans and war! To the people worrying about extinction caused by AI, how about we worry about the insane people with nuclear weapons first?
AI is and will continue to be developed for the purposes of war. Moreso the growth of the capability of AI could lead to destabilization and the occurrence of nuclear war.
For example the capability of MADD working keeps things somewhat stable "I kill you, you kill me" isn't great. But if I develop AI that could track all the sub, and knock out the nukes before they launch, this could push nation states to attack first before their capability is taken out.
There is a continuum of issues here spanning from "People problems -> people + AI problems -> AI problems"
By adding more AI we're just expanding the size of the problem space.
I don’t really understand the question. We routinely worry about insane people with nuclear weapons; that’s why almost the entire world got together in 1968, and signed a treaty agreeing not to start any new nuclear weapons programs beyond the five that existed at the time. It’s not flawless, and four non-signatories have developed nuclear weapons since then, but I think any AI safety proponent would be more than satisfied with a similar agreement.
And how reasonable is that we would have autonomous agents a bit smarter than humans in the nearish future? Very low, I'd say, about the same as the odds I'd give a year ago for GPT4? Not odds I'd be happy to take for the _destruction of humanity_.