[jhartmann]

I have a machine learning startup that is using deep learning neural networks, so I'm probably biased here. I really think there is something that is worth the hype here, this is the first time we can solve significant problems without lots of feature engineering to make the neural network be able to solve the problem. While I'm sure there are going to be tons of things that deep belief neural networks can not do well even with these new capabilities and breakthroughs, there is a crapload of data out there that is begging to be analysed. Being able to get reasonable performance without a ton of feature engineering and years for a black arts team to build something that can get the data into a state where problems can be answered is SUPER exciting. The Neural Networks we are using are more specialized and more like Yann Lecun's, and we aren't using dropout like Hinton but we already have something that gets very good accuracy in our problem domain. There are some new techniques that are just coming out of Montreal, one in particular I'm very excited about called Maxout that looks like it will be another significant advance. One of the problems networks like this usually have is that the activation functions saturate above a certain level and once a neuron is in the saturated state the gradient training process will not move it anymore. Maxout is different in that it doesn't have this property, and it seems to maximize the benefit of the random selection process of dropout.

While I don't have the math credentials to match Hinton I think as more 'normal' folks like me get into the game there will also be some interesting things going on. We are trying some interesting things that seem very promising, and I'm sure there are lots of other folks beginning to play with these things that will have some interesting ideas and approaches as well.

So I personally think this is super exciting, and while it might not be applicable for every problem Deep Learning will definitely have a big impact.