I am very skeptical of the current approaches (supervised learning) making the quantum leaps being promised. This needs a paradigm shift of weakly-supervised learning and fine-tuning for specific tasks (a la human learning).
It is indeed where a lot of the current research is heading to if you count self-supervision as part of weak-supervision. Self-supervised learning brought massive improvement in NLP and is bringing state of the art result in vision after some time where purely supervised learning showed better result.
Weakly-supervised and self-supervised methods are extremely mainstream today. There is a flood of papers on these topics. Conferences are full of them.
This comment was perhaps insightful 5-6 years ago, today it's absolute orthodoxy.
The thing is though, supervised just works way better. It's not that people don't want to do weakly-supervised, but that it's very hard!
My point exactly. Weakly/self-supervised learning is "very hard" because it's still an active area of research and breakthroughs are needed. I feel those a pre-requisite before we can credibly approach the promise of "doing everything"