[rch]

At a conference I attended last month, one of the keynotes estimated that there might be 250 people in the country with the skills need to build non-trivial, ontology-based data systems. Even if that is an wild exaggeration, it is at least evidence of a perceived shortage.

Also note that an ability to transfer domain experts' knowledge into working models is at least as important as the Stats+ML bits.

[PaulHoule]

I'd say the current academic research in ML is not oriented towards producing people who can use ML in real applications.

I've hovered around the periphery of a world-leading ML research group, and the first takeaway I have is that 7 years ago I thought the stuff they were working on was going to take the world by storm, but looking back, I can say it hasn't.

This group does a number of research projects on narrowly defined topics. 4 out of 5 of these projects try out some refinement of the method that doesn't really work. Maybe 1 out of 5, if that, point to a real improvement.

The big thing that's lacking are serious attempts to push the state of the art by attacking a problem holistically and "taking no prisoners" -- yet this is exactly the kind of thinking necessary to commercialize ML.

The leader of the group got tenure so he thinks everything is going OK. He won't even offer an analysis of why this technology hasn't been widely commercialized. PhD students from this group usually interview at Google, Microsoft and Facebook but these three employers are the only ones they consider as an alternative to academic employment.

[tgflynn]

There's definitely a need for people to continue working on developing new models and ideas. I think that's where academic research fits. That said the academic world could do a much better job of effectively evaluating and comparing models so that practitioners have a clearer view of what works where. There also seem to be a lot of biases that get perpetuated in the academic world, like the bias against neural networks.

However I think what's really needed for this technology to develop to its true potential is figuring out how to apply it to real problems and I think that's more a role for industry practitioners than academics. The problem is that for people to make a living at this there needs to be a market. I think what we are seeing in this area is a shortage of both supply and demand with the supply side hindering the demand side and vice versa.

[NyxWulf]

I agree with Paul's comment. The 250 number feels low to me, but that is applying a specific model. Typically people come with some set of favorite models, and many of them provide that vast majority of the benefit a business needs. Especially when the current model in use is slipshod and busted at best.

[rch]

The 250 number refers to the medical field. So the requisite background includes at least biology, and more preferably clinical experience. That diminishes the pool somewhat.

A similar phenomena certainly manifests in other highly specialized fields though. There are far fewer people with both the skills we're talking about, and deep water E&P or big 3 audit experience, for instance.