[kbenson]

It's classic emergent behavior. While you may understand how the algorithm works, and even be able to step through and see how each neuron affects the whole, that doesn't mean you know why the answer is correctly achieved through all of them combined.

The classic example is facial recognition. Training a neural network for facial recognition will result in lots of neurons contributing a very small part of the whole, and only when all (or most) are involved is the answer correct.

To most people, this (emergent behavior) is "magic".

[tripzilch]

But it's not emergence. Using Gaussian elimination to solve a gigantic system of equations isn't emergence either, even if the numbers are a bit too many to carry in your head at once. (as a matter of fact, solving systems of linear equations is part of the RBM training algo)

And even then, if it was emergence, doesn't automatically imply we don't understand it or that it's "magic". The famous Boids flocking simulation is a classic example of emergence. It's not very mysterious. Yes large-scale behaviour emerges from simple rules, this is amazing that it happens, but it doesn't hold up a barrier for us to understand, analyze and model this large-scale behaviour. Crystallisation is emergence, again we model it with a bunch of very hard combinatorial math.

But in this case, neural networks are not an example of emergence. They are really built in a fairly straight-forward manner from components that we understand, and the whole performs as the sum of the components, like gears in a big machine.