[ben_w]

"[W]e're less than 20-30 years away making humans intellectually obsolete" is neither necessary nor sufficient to get to the conclusion "20% chance of killing literally everybody".

A super-virus that blends the common cold with rabies would kill approximately everybody; that doesn't need human-level intellect to happen.

Conversely, humans are human-level intellect, and we're mostly sympathetic to each other's plights, which motivates many of us to give to charities and support those that can't support themselves.

The biggest problem with AI is that we have only marginally more idea of what we're doing than evolution did, so there's a good chance of us ending up with paranoid schizophrenic super-intelligences, or dark-triad super-intelligences, or they're perfectly sane with regard to each other but all want to "play" with us the way cats "play" with mice…

20-30 years to get there would make people like Yudkowsky, one of the most famous AI-doomers, relatively happy as it might give us a chance to figure out what we're even doing before they get that smart.

[uoaei]

It's still a lack of imagination to assume that AIs will display behaviors that align whatsoever with pathologies we identify in humans. AIs could be completely incomprehensible or even imperceptible yet have strong influence on our lives.

[ben_w]

> AIs could be completely incomprehensible or even imperceptible yet have strong influence on our lives.

To an extent both of those are already true for current systems.

That said, many people are at least trying to make them more comprehensible, and I guess that being sufficiently inspired by human cognition will lead to human-like misbehaviour.