[stillmotion]

> And there's been a strong reaction from those who are heavily invested in SVMs/kernel methods, bayesian methods, etc to claim that deep learning isn't theoretically well grounded (which is not really that true any more, but is also beside the point for those that just want to get the best results for their project.)

Could you elaborate more on what has changed recently in the theoretical grounding of deep learning?

[jph00]

It's not so much that there's just one or two specific results, but rather that there's just far more researchers working on this now, and quite a few are working on more theoretical stuff. Sometimes, that theory results in really practical outcomes - a good example would be https://arxiv.org/abs/1701.07875 . Or another by the same researcher: https://arxiv.org/abs/1506.00059 .

We've seen, in the last year or two, interesting results in nearly every area of theoretical research of deep learning, including generalization, optimization, generative modelling, bayesian models, and network architecture.