[spikels]

While deep learning is a very cool technique and is currently getting the best results in a few domains I think all the hype may become a problem. I was around for the prior round of neural network excitement and much time, effort and money was wasted. In that case it turned out that other techniques were more tractable and thus easier to use and improve upon.

It must be the association with the human brain that just makes neural networks more exciting than other techniques. But dispite the appeal of imitating nature has this usually been the easiest way to make progress in the past? Seems like it would be harder to achieve both goals at the same time.

So far the results are looking pretty good but it is probably best to keep the hype at a reasonable level unless it is crucial of your business model. ;)

[jhartmann]

I have a machine learning startup that is using deep learning neural networks, so I'm probably biased here. I really think there is something that is worth the hype here, this is the first time we can solve significant problems without lots of feature engineering to make the neural network be able to solve the problem. While I'm sure there are going to be tons of things that deep belief neural networks can not do well even with these new capabilities and breakthroughs, there is a crapload of data out there that is begging to be analysed. Being able to get reasonable performance without a ton of feature engineering and years for a black arts team to build something that can get the data into a state where problems can be answered is SUPER exciting. The Neural Networks we are using are more specialized and more like Yann Lecun's, and we aren't using dropout like Hinton but we already have something that gets very good accuracy in our problem domain. There are some new techniques that are just coming out of Montreal, one in particular I'm very excited about called Maxout that looks like it will be another significant advance. One of the problems networks like this usually have is that the activation functions saturate above a certain level and once a neuron is in the saturated state the gradient training process will not move it anymore. Maxout is different in that it doesn't have this property, and it seems to maximize the benefit of the random selection process of dropout.

While I don't have the math credentials to match Hinton I think as more 'normal' folks like me get into the game there will also be some interesting things going on. We are trying some interesting things that seem very promising, and I'm sure there are lots of other folks beginning to play with these things that will have some interesting ideas and approaches as well.

So I personally think this is super exciting, and while it might not be applicable for every problem Deep Learning will definitely have a big impact.

[rm999]

>I was around for the prior round of neural network excitement and much time, effort and money was wasted. In that case it turned out that other techniques were more tractable and thus easier to use and improve upon.

And before 1980s style neural networks there were 1950s perceptrons. That was a much bigger mess, it took more than ten years for someone to point out how 'dumb' perceptrons were (they couldn't even model an XOR), which led to a collapse in AI funding that lasted more than 25 years.

[spikels]

Can we be a little more thoughtful this time and avoid the boom and bust cycle that so often leads to problems?

You would think that since it already happened with neural networks before it would be less likely to happen again. However it may be that the same factors that lead to the last cycle are still in operation and it is actually more like to happen again. Something like the reasons for the seemingly endless series of real estate bubbles.