[yonkshi]

The sample efficiency of ML systems are increasing rapidly in the theoretical realm, though the available data are growing magnitudes faster than the progress in ML sample efficiency. The article has a valid point: It's far cheaper to hoard data than to invent a more efficient ML system, thus we will see people race towards data rather than technical complexity.

[fooker]

>The sample efficiency of NN are increasing rapidly in the theoretical realm.

Can you provide some references for this?

[yonkshi]

Sure. I don't have any holistic survey to prove my point, but an example of recent progress in terms of sample efficiency is this paper[0]. Derivatives of this paper have been used to solve Sudoku[1], Starcraft II[2] and more [3]. This paper enabled more efficient use of data by creating a probabilistic graphical model between logical sets.

[0]https://arxiv.org/abs/1706.01427 [1]https://arxiv.org/abs/1711.08028 [2]https://arxiv.org/abs/1806.01830 [3]https://arxiv.org/abs/1806.01822

[state_less]

I like the 3D datasets in these papers since, like a scientist in a lab, you can setup the experiment and explore the domain. Adding time in would be cool too (e.g. Are the blue and red ball going to collide in 10 seconds?)

It also helps to be able to show you can answer some of these questions in principle with your model. It gives you hope that it might be able to cover real world images.