Hacker News new | ask | show | jobs
by Gravityloss 3134 days ago
How Bayes kinda works, or how I see it.

Conditional probability (with some caveats that someone in the comments can fill in on):

    P(a,b) = P(b,a)
    P(a|b) * P(b) = P(b|a) * P(a)
    P(a|b) = P(b|a) * P(a) / P(b)
a can be model and b can be data so it becomes

    P(model | data) =
    P(data | model) * P(model) / P(data)
We have or can estimate the things on the right side. We want to ultimately get the thing on the left side.
2 comments

To clear up your first set to have conditional probabilities for everything, Bayes' theorem is just a restatement of the product rule:

    p(a and b | context c) = p(a|b,c) * p(b|c)
                           = p(b|a,c) * p(a|c)
    or = p(a|c)*p(b|c) = p(b|c)*p(a|c) if a and b are independent of each other

    so Bayes only matters when there is dependence:
    p(a|b,c) = p(a|c) * p(b|a,c) / p(b|c)

    otherwise it's just p(a|c) = p(a|c)
I like to put things in that order because p(a|c) is the "prior belief" and with some handwaving say things like "updated belief = prior belief and new evidence about belief".
Mathematically trivial, but great notation to explain the point of Bayes. Brilliant!