[closetnerd]

Andrew Ng himself said that he was inspired from the ideas that Jeff Hawkins put forth in his book "On Intelligence" which could explain some of the similarities with Deep Learning [1]. But honestly, event still, Deep Learning doesn't seem to model many of the understood principles of the neocortex at all. I read a relatively recent paper by Andrew Ng [2] about Deep Learning optimized for GPU and though it resembles some of the hierarchical aspects of the neocortex, it doesn't really go any further.

I recommend that, if you're interested, you read the [3] CLA white paper for more details but the main difference I see is that the CLA tries to model the concept of storing as sparse distributed representations by modeling neocortical columns. The problem there is that even today, neuroscientists don't agree on any one theory of its structure and function. And frankly the CLA's theory neocortical columns seems to be the most sane. This is based on some of [4] Gerard Rinkus's research on the functions of neocortical columns.

Basically, in my opinion, there is A LOT more neuroscience in HTM-CLA then there is in Deep Learning. And I'm pretty sure that Deep Learning will converge on much of the concepts put forth by the CLA. It really shouldn't be seen as a competition in the first place I suppose, but the theories in AI and theoretical neuroscience are converging pretty fast already.

[1]: http://www.wired.com/2013/05/neuro-artificial-intelligence/a...

[2]: http://web.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgC...

[3]: http://numenta.org/resources/HTM_CorticalLearningAlgorithms....

[4]: http://people.brandeis.edu/~grinkus/Analog_Devices_Lyric_Tal...

[georgewfraser]

There is a much bigger problem with basing your model of the brain on cortical columns, which is that they don't exist outside visual cortex and the whisker region of sensory cortex in rats and mice. The idea of a repeating functional unit was so appealing that many neuroscientists have just refused to give it up, in a kind of collective wishful thinking. There was an excellent review paper in 2005, "The cortical column: a structure without a function", which is basically the emperor-has-no-clothes of this field.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1569491/

[closetnerd]

Yup, read that a while back actually. And it makes some good points. It should be titled "The cortical column: a structure without an agreed upon function".

The cortical column are a lot more well define, referred to as ocular dominance columns, in the visual cortex. The problem is that the structure of cortical columns are very malleable and plastic. So it makes it very difficult to see them consistently throughout the neocortex. So there isn't much definitive proof for cortical columns throughout the neocortex but there is convincing theory, very much pushed by Hawkins.

There is a large consensus that the neocortex stores and acts on information in a distributed way. Most of the well defined theories propose some kinds of neural engrams. But there wasn't any theory about how the neocortex stored information in a distributed way. The function of neocortical columns, as proposed by Rinkus, seems to explain very convincing one such way of creating Sparse Distributed Representations.

In terms of theory, my opinion is that cortical columns seem to be integral to a unifying theory of the neocortex.