[duhruh]

A better alternative to your simple a/b test would be the multi-armed bandit algorithm. This will give you your answer the fastest, though it's slightly harder to wrap your head around check it out here: http://stevehanov.ca/blog/index.php?id=132

[codyb]

I liked the article because it's really fun to stick with something like that, wrap my head around the math, and enjoy the exploration but the multi armed bandit algorithm was supremely satisfying to read. Simple, intuitive, logical, and with easy to tweak parameters. It doesn't get much better than that.