[_0ffh]

There is actually a way to get really amazing sample efficiency out of a learning setup, and that's engineering in a load of appropriate inductive biases, which personally I am convinced evolution has done for us. Explains a big chunk of the "how are brains so sample efficient" problem really easy, but unfortunately without handing us an easy way to replicate it, which makes it unpopular. Also, it's something that we don't really want to do in the same way evolution has, as all those biases do even further reduce sample efficiency for all the things for which they are not appropriate.

[usernametaken29]

In a nutshell this is what statistical learning theory says. For any dataset there is an optimum given a prediction task. It follows from entropy. As the commenter pointed out “evolution has this backed in”. There once was a research direction of evolutionary distribution estimation algorithms but basically we know nothing about evolution, and scaling ede to multidimensional data is much harder than optimising objectives and trying to squeeze the inductive bias. For all it’s worth I think much of the current AI research is focused entirely on the wrong questions. Can machines learn? Sure, inductive bias FORCES them to learn. Given basically unlimited data can computers pick appropriate inductive biases to do anything useful, “survive” if you want to call it that… probably not, at least no one has really asked these questions for a couple of decades