[md2020]

My situation is the same as yours, CS junior heading into my capstone project next semester, and my opinion is a resounding yes. The deep learning obsession is almost certainly a hype bubble. I have observed the same here at my university, the "But what if we did it with deep learning?" projects are almost reaching meme status. It's rather disheartening as someone who actually is interested in AGI, but I've been driven away from wanting to pursue the field since the current research seems lacking in ambition and substance. My previous summer internship had me reading a lot of deep learning papers on arXiv, and the vast majority of them seem to be tweaking a single parameter in a DNN, achieving a 0.3% increase in score on an arbitrary benchmark, and calling it a meaningful result. I'd personally like to see more people doing work like the kind DeepMind does that seems to actually achieve breakthroughs informed by knowledge from neuroscience, but I have a feeling we won't see that anytime soon since DeepMind gets their pick of the best researchers in the world. I'm just an undergrad though, would love to hear the opinions of more knowledgeable people! Specifically, I'd like to hear arguments against the sentiment that "Deep learning right now is pretty much alchemy". How is the work in deep learning helping us understand the nature of intelligence, rather than just helping Facebook and Amazon better target advertisements and product recommendations?

[visarga]

> How is the work in deep learning helping us understand the nature of intelligence

Neural networks performance on a problem is a benchmark of its real difficulty. It gives us insight, a new perspective.

In millennia of deliberations what have philosophers have discovered about the nature of intelligence? And then .. a neural net beats us at all board games, another can solve differential equations, another can translate, another can see, and so on. Have we really not learned anything by these inventions?

Another advantage of DL is that it frames the problem of intelligence in mathematical concepts and rigorous evaluation.

I, for one, have reconsidered all my spiritual beliefs after learning about the agent-environment-reward model of reinforcement learning. A new way of framing the agent and life, so parsimonious and powerful. And it does not require a soul, or a god, or anything outside out real environment, and yet can explain so much.

The whole machine learning paradigm is another powerful concept through which we can understand how we might function. Previously you might wonder how emotion, thought, sensation, imagination and will relate to each other. Now we can understand how they might be implemented and wired together, and what principles support their function.

[md2020]

> And then .. a neural net beats us at all board games, another can solve differential equations, another can translate, another can see, and so on. Have we really not learned anything by these inventions?

I would still argue no, we haven't learned anything about intelligence from these. They are impressive achievements, but strictly in the sense of "We found a way to use computers in a way we were not using them before".

1.) Neural nets + MCTS beat us at all board games--board games that humans invented and can achieve mastery at. If a human Go player was born who could beat that version of AlphaZero, we would not say that person has solved intelligence.

2.) Differential equations: also invented/discovered by humans, can also be solved by humans

3.) Translate and see: See above, with the additional caveat that humans actually are better at translating and seeing than deep learning systems are.

In addition, these were all achieved individually by systems with different architectures and massive amounts of training data that would amount to several human lifetimes. An 18 year-old human can play board games, drive a car, do differential equations, and learn multiple languages, with a single brain using a generalized structure and a fraction of the "training data" afforded to DL. This indicates to me that ML as a whole is still very far off the mark of General Intelligence.

> Previously you might wonder how emotion, thought, sensation, imagination and will relate to each other. Now we can understand how they might be implemented and wired together, and what principles support their function.

Previously? This is still an unanswered question. Show me where deep learning research has even come close to producing a system that can learn and adapt like a human mind does.

[sifar]

>> Have we really not learned anything by these inventions?

Not really, and that is the problem. We can create something that sort of works - but we don't understand it. The only thing we have learned is that we don't need to understand intelligence to build something that works on some tasks.