[modeless]

The idea of deep learning is to eliminate the need for domain experts to write heuristics, ontologies, feature detectors, etc. In deep learning you feed the learning system raw data and it automatically creates feature detectors to model the data. Then once you have a good set of features you can train the system to perform a specific task using those features.

As the network gets bigger and deeper the feature detectors become more abstract and capable of higher-level tasks. For example, given a bunch of images, a small network might learn to distinguish straight lines from curvy lines, while a large network might learn to distinguish humans from cats.

So if deep learning actually works, then the main constraint on the capabilities of the learning system is compute power, not the cleverness of the domain experts writing your feature detectors. The main problem becomes scaling.

[klipt]

It also relies on having a lot of data (unlabeled is fine for learning features).

By contrast, manually chosen heuristics assumes you've seen a lot of data and you're bootstrapping the model with features deduced by your biological brain.