[rspeer]

Fair enough. That's a useful comparison to know about.

But I'm wondering how you get around that with the neural net. In the post, you said there are only a few hundred labeled examples, right? How can a neural net with hundreds of parameters set those parameters to anything reasonable, and not overfit, when there are about as many parameters as examples?

[bowlesbe]

Great question and I share your intuition but I think its all properly regularizing your model. I guess for neural networks, Dropout works really darn well as a regularization strategy. I could have tried to see whether performance dropped significantly without dropout.