[317070]

There have not really been a lot of major breakthroughs in meta-learning in the last year, as far as I am aware. The paper is basically saying there was not a lot of progress in the 3 years before that either.

All in all, nobody really has a clue on how to do meta-learning right (or I am not aware of their work). There is progress being made on benchmarks, but some argue that progress is not really tackling the real issue at hand, i.e. learning to learn. Moreover, the current common benchmarks are not really good at untangling the progress in deep meta-learning from the progress in deep learning in general.

[devalgo]

Isn't GPT-3 exactly the kind of meta learner you're thinking of?

[317070]

I would say it is exactly the opposite. :)

It is showing how you can get drastically better at deep meta-learning by being better at deep learning. But it does not really show how you can be better at deep meta-learning outside of the improvements in deep learning.

You can take any deep meta-learning algorithm, take the deep part in it, apply the improvements in deep learning from the last year and claim that you have improved on the deep meta-learning problem this year. Well yes, but actually also no.

It's like trying to find a new antibiotic, and the solution is throwing more existing antibiotics into the same pill. Well yes, it works, but it is also not exactly the problem.

Don't get me wrong though, GPT-3 is amazing work.