Hacker News new | ask | show | jobs
by lqr 1119 days ago
How can your prior be uniform if the hypothesis class is unbounded?
3 comments

How can the hypothesis class be unbounded?

Anyway, you can define a sequence of solutions with bounded uniform priors and calculate the limiting solution. For any given data set when the endpoints of the intervals go to +/-infinity the solution will converge to the uniform prior one - if it exist.

I discussed this in another reply in this thread, but I can't personally think of any examples where the hypothesis is unbounded. The laws of physics, computation, the human mind, measuring instruments, etc. all impose bounds on real world problems.

As someone that does Bayesian Inference a lot for my work (computational biology), I very often use uniform priors, but the structure of all real world problems I have ever encountered allows me specify hard bounds to the edges of non-zero probability.

This isn't always possible, but sometimes you can define what's called an improper prior p(µ), such that even if ∫p(µ) is not finite, the posterior distribution p(µ|x) is.

A common example is when p(x|µ,v) is a Gaussian dist. with a prior on the mean set to p(µ)=1.