[fsavard]

That article is pretty light on details. I wonder if he pointed towards a specific form of unsupervised learning.

Anyway it's pretty funny in light of an intro I remembered from one of his old papers:

"It would be truly wonderful if randomly connected neural networks could turn themselves into useful computing devices by using some simple rule to modify the strength of synapses. This was the hope that lay behind the original Hebb learning rule and it is the vision that has driven neural network modelers for half a century. Initially, researchers tried simulating various rules to see what would happen. After a decade or two of messing around, researchers realized that there was a much better way to explore the space of possible learning rules: First write down an objective function [...] and then use elementary calculus to derive a learning rule that will improve the objective function." [1]

ie. backprop

So actually backprop was the solution to all that initial "messing around" with unsupervised rules. Though of course to be fair (if I understand correctly) those rules had very little to do with modern "unsupervised learning" methods (e.g. autoencoders, which still rely on backprop or similar optimization).

[1] http://www.cs.toronto.edu/~fritz/absps/hebbdot.pdf published in 2003