Hacker News new | ask | show | jobs
by oergiR 3488 days ago
The "model" in the title is the model of the world, as a probabilistic model. The good thing about such a model is that it explicitly states your beliefs about the world. Once you've defined it, in theory reasoning about it is straightforward. (In practice a lot of papers get written about how to do approximate inference.) It's also straightforward to do unsupervised learning.

This is a different perspective from (most uses of) neural networks, which do not have this clear separation between the model and how to reason about it. It's funny that Chris Bishop in 1995 wrote the textbook "Neural Networks for Pattern Recognition" and now is effectively arguing against using neural networks.

You can use both by using neural networks as "factors" (the black squares) in probabilistic models.

2 comments

It's funny that Chris Bishop in 1995 wrote the textbook "Neural Networks for Pattern Recognition" and now is effectively arguing against using neural networks.

I haven't read "Neural Networks for Pattern Recognition", but his "Pattern Recognition and Machine Learning"[1] is the text for ML work including Bayesian approaches.

I don't think one should view this as "arguing against" neural networks - it's more that Bayesian approaches give you something different.

[1] http://www.springer.com/gp/book/9780387310732

One of the most popular ways of using techniques like this is the "Variational Autoencoder". I've been working on using some alternate distributions with them as of late - it's very interesting, and quite powerful.
How does this work? You use the VAE to model variables and then somehow get the distribution from them?

Got a link? (I know the basics of VAEs, but I'm missing how to link them to this)

The VAE "coder" is modelling a distribution p(z|x), and the decoder is modelling a distribution p(x|z).

I like these slides: https://home.zhaw.ch/~dueo/bbs/files/vae.pdf