[tianlins]

it is true that most recent success of deep neural network are in the regime where n, d are large. And we surely shouldn't fantasize general AI solved in this way. However, the very appealing aspect of deep neural network is end-to-end training: for image recognition, we can map from raw-pixels to output. This is very different from other ML techniques. In some sense, deep neural networks learn "algorithms", not just "models". This formulation can be richer especially when given lots of data.