[pjc50]

> classifying pixel patterns in isolation isn't sufficient for robust visual perception

This seems to be only a very small step forward from Minsky's negative result about "perceptrons".

[AstralStorm]

That's because DNN are only a small step removed from multilayer perceptrons as well. (Few more layers, a tiny bit of internal structure, more advanced nonlinear activation function, better training schedule. Much more training data.)

They're not even close to structural or training algorithm complexity of natural neutral networks yet.

[_0ffh]

That result was not about multilayer perceptrons, but perceptrons. But, whatever.

[AstralStorm]

Multilayer perceptrons share many of the same problems single layer perceptrons have, such as trouble with high level structure and generating weird nonrobust features. They are much more nonlinear through and thus somewhat more powerful. (I'm imprecise here but it is easy to find papers on this ancient tech from before AI winter.)

DNN is essentially one of these with more layers than just typical 4 for MLP, because we figured out a way to propagate error and training gradients. (Plus a few important but interesting details.) They are not really qualitatively different according to math they use... The main difference is use of gated or not differentiable activation functions with various ways to compute approximate gradients when faced with this feature. Especially convolutional nets are similar to MLP.