[noskynethere]

> They try, usually with simple probabilistic techniques and input element-wise transforms, to mimic some function that produces approximations for a given set of inputs and outputs

It's my understanding that this is basically how the brain works. My personal theory is that enough of these "dumb" inputs, wired correctly together, leads to emergent behavior that is consciousness.

[crazypyro]

I imagine the brain more like hundreds (thousands, millions, I'm not sure the magnitude) of different specialized neural networks. So you have a specific neural network for picking out colors and that feeds (along with a bunch of other inputs) into the neural network for picking out object boundaries and that feeds into the neural network for object recognition and so on. In comparison, most neural networks that are used in computer vision are generally trying to do the entire process in a single network (although they also use feedforward, so the difference is more complex than just composing the various layers). I think there is something to the idea that we need the neural network to have points where it can spit out a partial piece of the eventual goal model, things like object boundaries before recognizing the object, recognizing eyes before the entire face, etc. The key is being able to get those logical partial model results at various layers of the network.

[noskynethere]

I'm outside my depth here, but isn't that what hierarchical learning is? (I think it's popularly called "deep learning", which I assume means the neural nets have depth?)

From what I've read, we aren't going more than a few dozens of levels deep. But it also sounds like this technique is very successful in image recognition.

Am I incorrect in my understanding?