[miles7]

There is a lot of evidence for what Paul says once you dig into a specific field. Taking two fields I know well: in physics, there was decades of work on fundamental questions on systems in equilibrium, while many obviously important open questions in out-of-equilibrium systems went neglected until the last 10 years or so when there's been a huge upsurge. These questions were known to be open 20 and 30 years ago, but just weren't as fashionable as a topic. Anyone senior enough in those earlier decades knew there were tons of open questions but also that relatively few people were working on them for whatever reason.

In machine learning, there are currently a lot of people working with neural networks, but relatively fewer people exploring alternative model architectures. So much so that issues specific to neural networks sometimes get framed as fundamental to machine learning itself. I'm personally exploring an alternative class of models called tensor networks with many possibilities for research directions and lots of open questions but only a handful of people work on them. One reason for working on a popular idea is that it's nice to work on a topic where you have many colleagues and know in advance that your model is likely to give good results on challenging datasets.

[nl]

I know next to nothing about physics, but I do know some about ML.

I think the reasons tensor networks are unexplored is interesting: The tools and techniques for dealing with them build on those for neural networks and the theoretical benefits are not clear cut enough to gain them a foothold over the practical results of neural networks.

[miles7]

Forgot to mention, but here is a recent theoretical result (prediction of generalization performance giving size of training set) based on tensor networks: https://itensor.org/miles/GenerativeMPS.pdf

[miles7]

Agreed. There is still a lot of work left to do!