[icebraining]

Are they actually experts, though? From that paper:

  “Concerning the above questions, how would you describe your own expertise?”
  (0 = none, 9 = expert)
  − Mean 5.85

  “Concerning technical work in artificial intelligence, how would you describe your own expertise?”
  (0 = none, 9 = expert)
  − Mean 6.26

Also, the whole methodology of aggregating the opinions of random conference attendees seems suspect to me. Attending a conference doesn't make you an expert.

[nbouscal]

You can restrict your attention to the TOP100 group if you prefer.

[icebraining]

Yeah, but then you just have 29 responses in total.

[nbouscal]

The word "just" feels awfully out of place in this context. If you were doing some kind of broad-based polling of public opinion, of course you'd want a bigger sample size, but 29 of the top 100 researchers in a field sounds like a hell of a good sample to me, and well worth listening to.

[icebraining]

If they were 29 random researchers, I'd agree, but since they were voluntarily selected, not really. They try to check if the sample is biased, but it's not convincing.

[backpropaganda]

From the first paragraph of Turing's famous essay on AI:

I propose to consider the question, "Can machines think?" This should begin with definitions of the meaning of the terms "machine" and "think." The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous, If the meaning of the words "machine" and "think" are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, "Can machines think?" is to be sought in a statistical survey such as a Gallup poll.