[rwilson4]

There is less of a conflict than many would have you believe. In many situations, both approaches yield the same answer. There are some edge cases. For example, in A/B testing, is early peeking bad? From a frequentist perspective the answer is "yes, either use a sequential method, or don't early peek at all". From a Bayesian perspective the answer is "early peeking is fine".

It boils down to what properties you want your analysis to have. Cox and Hinkley's "Theoretical Statistics" has a great discussion (section 2.4). Basically, you might want your analysis to have a certain kind of internal consistency. But you might also want your analysis to be replicable either by yourself or by another researcher. Those both seem like pretty important things! But there are edge cases (like the early peeking example) where you can't have it both ways. So you have to pick which one you want, and use the corresponding methods.

[zozbot234]

The likelihood principle actually supports the Bayesian perspective on these issues of experiment design, and is regarded as foundational by many frequentists.

[rwilson4]

Agreed. But as Cox and Hinkley discuss, the likelihood principle is sometimes at odds with the repeated sampling principle, so in any particular application, you need to identify if there is a conflict, and if so, which principle is more important. In my domain (simple A/B tests), you can claw the repeated sampling principle from my cold, dead hands.

[timy2shoes]

> From a Bayesian perspective the answer is “early peeking is fine”.

That’s definitely not true. Early peeking increases your type I error rate, even with Bayesian methods. For a full explanation see http://varianceexplained.org/r/bayesian-ab-testing/