[amalcon]

The most obvious thing that nobody seems to ever explain is that a statistician can use both frequentist and Bayesian methods. In fact, most good ones do. Frequentist methods are generally better for finding the needle in the haystack, while Bayesian methods are generally better at proving that it's actually a needle and not a piece of painted hay.

Re: Where did the difference come from, that's down to different interpretations of probability. The frequentist interpretation says that probability describes the world, whereas the Bayesian interpretation describes our beliefs. Here's another common misconception: You don't need to subscribe to one interpretation to the exclusion of the other. People who use the Copenhagen interpretation of quantum mechanics (a frequentist formulation if ever there was one) will also speak of fractional belief (the definition of Bayesian probability). It is important to be clear about which interpretation you're using at any one time, but you don't need to tie yourself to one interpretation, and it doesn't need to be part of your identity or world-view.