[eivarv]

Deep learning is not just neural networks, but rather the application of these in deep (i.e. many-layered) architectures, broadly speaking.

This enables hierarchical learning of increasingly complex concepts – building new concepts upon less complex concepts from previous layers. Deep architectures are thus able to learn high abstractions, as in [1], for instance.

If you have not yet done so, I would strongly urge you to read some papers on the subject from the last decade (e.g. Hinton, Bengio or LeCun), or even just skim through the Wikipedia entry [2].

[1] http://www.technologyreview.com/view/532886/how-google-trans...

[2] http://en.wikipedia.org/wiki/Deep_learning

[return0]

Deep learning is a large scale application of Restricted Boltzmann Machines, of which Hinton (among others) was a pioneer. But that was in the 80s, not in the 2000s.

http://en.wikipedia.org/wiki/Restricted_Boltzmann_machine

[eivarv]

I don't believe the term "deep learning" is restricted to RBMs only – at least that's not the way I've seen the term used in literature (e.g. Deep Convolutional Neural Networks, various deep Autoencoders, etc.).

[return0]

Convolutional networks were also developed in the 80s as well as backpropagating algorithms (autoencoders). The way i see it used, "deep" usually means many layers, indicating a difference in quantity, not in quality.

Point is, the science was there since the 80s, and not much has changed.

[eivarv]

Sure, but these types of deep architectures haven't really been practical until relatively recently.

Well, then we're in agreement about the meaning of the term. Deep Learning, then, would be Machine Learning using any of these deep architectures – be they Restricted Boltzmann Machines, or otherwise.

[Houshalter]

I sometimes wonder if in the 2030s, people will be complaining about how all the interesting stuff was really invented back in the 2010s.

But yes, the available computing power has been a huge limitation for much AI research.