I like this, but I feel it's a little optimistic (or pessimistic depending on your view). Isn't asking ML researchers when AI will dominate human performance a bit like asking a barber if you need a haircut?
I'm trying to see the analogy ... if I ask a barber if I need a hair cut and they say "Of course you do." they may be lying because they want the $15 (well, I'm bald so ...)
How does the ML researcher gain from lying "In year X" if they don't believe that to be true? It's a tenuous connection.
Ha, that's fair, I take your point. I was being a little glib. I didn't mean to suggest conscious malice on the barber's part -- only that the barber may be incented to provide more haircuts than are strictly necessary.
Perhaps the analogy works better if the barber is a friendly, honest person, who takes professional umbrage when they see other people with longer hair :)
One aspect of qualification is domain knowledge, which experts certainly have. Another aspect of qualification is calibration, which can only be proved & adjusted over time with a track record. A number of academic studies of prediction markets and other forecasting systems have shown that well-calibrated non-experts, with no skin in the game, often do better than actual experts, who often have poor track records as a result of incentives (or selection) to hype and extremize.[1]
Philip Tetlock has written on this topic for years. Two of his books are Expert Political Judgment and Superforecasting.
Edit: So to directly answer your question, rather than AI experts, I'd prefer technology experts (AI or otherwise) with a track record of well-calibrated predictions.
A historian of science, business professor, or futurist, might provide perspective on when people automating previous trades estimated they'd be automated; how long until they were; how long previous innovations spent in various stages of translation; and how this compares to what is known so far about the AI pipeline.
It has less to do with title than systematized knowledge of models, but people with those titles are more likely to have invested in acquiring this knowledge and these models.
Perhaps labour economists, demographers, or sociologists?
I don't believe ML researchers are unqualified -- it's more of a potential incentive problem. I don't think it's unreasonable to suggest that people involved with / employed by a technology may have a tendency to exaggerate its benefits.
Ideally I'd trust the numbers here more if there was at least a cross-section of knowledgeable people being surveyed across a few different disciplines (or at least more than one).
How does the ML researcher gain from lying "In year X" if they don't believe that to be true? It's a tenuous connection.