[cmansley]

May I ask how this is Bayesian in anyway? I understand that using the term Bayesian is good for directing clicks to a site, but this seems like good old fashion frequentist math. None of the hallmarks of a Bayesian approach to the problem are here: having a distribution over hypothesis, having an explicit prior, computing the posterior of the distributions.

I have some experience with the medical trial literature and specifically bandit algorithms and using cumulative regret verses other statistical measures like PAC frameworks. And regret is most certainly not a Bayesian idea. Instead you are explicitly modeling the cost of each action (providing an A or B test to a user) instead of assuming all costs are equal.

Yes, this is a better approach because it explicitly models the costs associated with the exploration/exploitation dilemma. But, it is not Bayesian.

[aaronjg]

In the clinical trial literature Anscombe's approach is considered Bayesian, and Armitage is frequentist. From Armitage's 1963 response to Anscombe's paper:

'Anscombe takes the Bayesian view that inferences should be made in terms of the likelihood function... An immediate consequence is that stopping-rules are irrelevant to the inference problem.'

Page 6 of the Anscombe paper that I cited may be helpful in your understanding of the approach.