[backpropaganda]

You're piling strawman on top of strawman, while suffering from confirmation bias. You think the AI risk guys are making well-hedge arguments, because you believe in that thesis.

Andrew Ng, Yann LeCun, and many other people who have ACTUALLY worked in AI are the ones who scoff at it. They don't need to make arguments because what do you respond to a young earth creationist?

All of the arguments in Supreintelligence or otherwise are simply that AI will eventually exist. The only argument for that it will come by 2050 is a badly conducted survey of non-experts.

Should we worry about all sorts of existential risk which could arrive in any undetermined time in the future?

The whole project is so absurd, it's hard even to begin to make any counter arguments, because none of the arguments make any sense.

[nbouscal]

Is this a badly conducted survey of non-experts? http://sophia.de/pdf/2014_PT-AI_polls.pdf

Edit: In particular, the TOP100 subgroup.

[backpropaganda]

Yes. The other 3 groups were not AI researchers. And the TOP100 group had a response rate of 29%, which adds self-selection to the process. People who are interested in AI risk et al. are more likely to respond to such a survey, adding bias. Andrew Ng or Yann LeCun or anyone actually working in AI would have refused (and probably did refuse) the invitation. Also, this TOP100 group of people are also more likely to be GOFAI folks who pretty much have no idea about the current data-driven deep learning-based AI.

Even Stuart Russel, the only CS guy in the AI risk camp, doesn't actually believe that AGI is anywhere near. But he works on it simply because he thinks we can do solve some of the problems like learning from demonstrations instead of (possibly faulty) rewards. That's actually a core AI research topic, not a AI ethics/values/blahblah topic. Oh, and also because this allows him to have a differentiated research program, and thus directs any funding on this niche to him.

[nbouscal]

Seems pretty clear neither of us will convince the other in this venue, so I'm going to leave it here. Thanks for the discussion :)