[mortuus]

fta: "One area where we are seeing real-time analytics improve content companies is in article headlines. For most of us, an article headline is all we use to decide whether or not to read on, so having a good one is definitely important. The Huffington Post is ahead of the game here. They use analytics to run A/B split tests on their important articles – in real-time. The Huffington Post initially shows 2 headlines for the same story, after 5 minutes of testing they discard the less popular one."

How common is this kind of behavior on the web? Are many news outlets doing this kind of split testing?

Can anyone recommend a 'getting started' with A/B testing guide (for blogging)?

[physcab]

5 minutes seems sort of arbitrary. How do they know they have a representative set that describes the whole population? What if in the next 5 minutes they get a rush of users who prefer the other headline? On the other hand, if they are getting second by second updates and they are looking at the standard deviation of the entire minute, then maybe it would be easier to tell which dataset was more statistically significant...no?

[jrich]

It all depends on the amt of traffic they get. I'm sure they've experimented enough to realize that 5 minutes is enough time to get a representative sample of their general audience.

[jrich]

Here's a pretty good A/B testing article. I'm sure some it can be applied to blogging: http://www.smashingmagazine.com/2010/06/24/the-ultimate-guid...