Hacker News new | ask | show | jobs
by jghrng 4865 days ago
All that being said, anyone care to weigh in on the pros/cons of RBMs vs something like a contractive autoencoder?

For some problems, it may be nice to have a generative model as offered by RBMS (although Rifai et al. published a sampling method for contractive auto-encoders recently: http://icml.cc/2012/papers/910.pdf). I feel like with RBMs, you can design models which incorporate prior knowledge more "easily" (you may end up with pretty complex models...), e.g. the conditional RBM, the mean-covariance RBM or the spike & slab RBM. Additionally, there's the deep boltzmann machine that consists of multiple layers that are jointly trained in an RBM-like fashion.

Auto-encoders are straightforward to understand and implement. With contractive terms or denoising, the are powerful feature extractors as well.

But as you already noted, if you "just" want to have a good classifier, I think it pretty much boils down to personal preference since you're going to spend some effort on making these techniques work well on your problem anyway.