Hacker News new | ask | show | jobs
by benanne 3971 days ago
I feel like this guide comes about 5 years too late - RBMs as density models have been shown to be relatively weak, except in the case of binary data. For continuous data, you can often do better even with a simple Gaussian mixture model. Other than that they are cumbersome to train (the gradient needs to be approximated), and the for continuous variants training can be unstable unless you use very low learning rates.

They were very popular for unsupervised pre-training a while ago, but the utility of pre-training has greatly diminished. Unless you have a ton of unlabeled data and very few labels, it's not worth the effort. And if it is, you are better off using autoencoders for pre-training anyway. They are conceptually much simpler and easier to understand, and you'll get roughly the same results.

If you want to get started with deep learning, focus on feed-forward and recurrent neural nets instead, you'll get much more useful knowledge out of that. For most of the common deep learning use cases there is no need to bother with RBMs anymore.