[boostedsignal]

For those unclear on the concrete (rather than philosophical) difference between Bayesian and frequentist statistics in the first place, I hope it's not inappropriate for me to share this 5-minute example that I wrote a while back: https://news.ycombinator.com/item?id=11096129

[wodenokoto]

You write that the frequentist doesn't answer the question, but it does. It answers

    P(H') = (H/H+T)^H'

You also write that the frequentist solution fails to give an error estimate, yet you don't show that the Bayesian solution does give one.

If the goal of the article is to show that Bayesian is more correct than frequentist then it leaves the reader unconvinced. If the goal is to show 3 ways of finding a probability, you should either say each is fine under its own paradigm, or argue why only one paradigm is correct.

[justinpombrio]

> You write that the frequentist doesn't answer the question, but it does. It answers > P(H') = (H/H+T)^H'

That's not the probability of getting H' heads in a row. It's an estimate of the probability of getting H' heads in a row based on a Maximum Likelihood estimation.

It doesn't make much sense if you take it to be the probability of getting H' heads in a row. For example, if {H=1, T=0}, then P(H'=100) = 1. You looked at one flip, and then decided that every subsequent flip was guaranteed to be heads?

It becomes even more clear that the question isn't really being answered if you take {H=0, T=0}.

[wodenokoto]

> That's not the probability of getting H' heads in a row. It's an estimate of the probability

Well, the true probability is unknown. You might assume a fair coin and do some Bayes rules, but how is that a guarantee of anything?

[justinpombrio]

> You might assume a fair coin and do some Bayes rules

Huh? Read the post, no one is assuming a fair coin.

[boostedsignal]

> You write that the frequentist doesn't answer the question, but it does. It answers: P(H') = (H/H+T)^H'

The question was asking for P(H' | H, T), not P(H').

> You also write that the frequentist solution fails to give an error estimate, yet you don't show that the Bayesian solution does give one.

Because there is no error? In the proof I assume P(p) is known and then after that every step follows from a law of probability. There is no error to be accounted for in the procedure. The only caveat is that we need to know P(p) to be able to perform the procedure, which is a caveat that I point out at least 3 times in the page.

[zAy0LfpBZLC8mAC]

> The only caveat is that we need to know P(p) to be able to perform the procedure

I think this is a very confusing way to put it. P(p) is not an objective value that you can know or not know, it is rather a model of our subjective knowledge, and therefore it doesn't really make sense to say "the caveat is that we need to know what our knowledge is" ... yeah, we do, but that is always the case by definition, so pointless to bring up.

[badminton1]

Another problem for the less familiar with the Bayes theorem is what is described as the "Bayesian trap", explained by the youtuber Veritasium: https://www.youtube.com/watch?v=R13BD8qKeTg

[WhitneyLand]

I hope people agree it’s totally appropriate, and appreciated, thank you for reposting it.

This is most of the reason I come here, because people show the good will to share bits of knowledge and experience.

Then a whole other benefit, is that when people are willing to do this, their contribution might be critiqued or corrected, which can then sharpen or polish your knowledge and thinking even in areas where you might be very qualified.

For some people this would be a nightmare, if they can easily feel angry or hurt when their intellect is challenged, especially when they are an “expert” on the subject.

But I suspect most people here feel the opposite. You found a flaw in my results or reasoning? Fucking awesome, you have just make me stronger.

edit: I don’t know many other online forums where this dynamic exists, so if anyone does please don’t keep it a secret.

[taserian]

You might want to check out lesswrong.com

I'd recommend https://www.readthesequences.com/ as something to test the waters; if this is your style, then you'll enjoy lesswrong.com .

[mycat]

For me, this book's first chapters explained nicely about ML, MAP and Bayesian using real computer vision problems. The author included helpful visual aids (gaussian plots, contour plots, filters output, etc) http://www.computervisionmodels.com

This is a rather unusual book where it gives primer on probabilistic method that is actually applicable in non computer vision problems. It is Bayesian heavy and rarely touches neural networks; the book is released in 2012, the year deep learning boom started.

[boostedsignal]

Yeah, there are lots of good books out there. My goal was to get the point across in a 5-minute read (give or take).

[yalph]

This is great thanks for posting.