[visarga]

Some of the learning to learn models do much more than fine-tuning parameters, they can even discover novel architectures. On the other hand, using meta-learning can be a way to check if human generated solutions are up to par, because random search can be more thorough and even try absurd ideas that might work out.

In programming we have tons of automation as well and we haven't ditched the programmer yet. Programming is auto-cannibalizing itself since its inception, each language automating more of our work. Even in ML, 10 years ago it was necessary to create features by hand. This required a lot of expertise. Today it's been automated by DL, but we have more AI scientists than ever and the jobs are even better paid.

So I don't think meta-learning is a fluff idea, and we don't have to fear it replacing humans yet. Instead, it will make AI more robust. The only minus I see is that it requires a lot of compute, but we can rent that from the cloud (make an architecture search for a few thousand dollars), we don't need to fork millions of dollars like the big labs who own their hardware. And we don't need this kind of intensive DL all the time, just once maybe, for a project. After we find the best architecture and hyperparameters, we can use that and train normally. By collating meta-learning data across many projects, we can make training faster and cheaper, reusing insight gained before.

[lemondrops]

Creating features has not been automated by Deep Learning at all. Even for image recognition tasks, where your "features" are simply the pixels of an image, there's still lots of preprocessing work to get those images into a form that NNs can deal with well.

Feature engineering is actually still the hardest part of most ML tasks, because it can not be optimized by a simple grid search like the hyperparameters of a model.