[patio11]

It is easy to believe the things you want to believe. A/B testing is useful for convincing you of the things you don't want to believe.

Example: I'm releasing my Rails A/B testing framework this Sunday, with a case study writeup taken from my site. My site currently allows people to defer signup for the trial by means of a guest login. That sort of option is popular here. I'm told it increases usability and user engagement, right?

Without ruining the surprise: that is a testable hypothesis. ;)

[onreact-com]

A/B testing is great, no doubt. The lesson of solely simplifying is just not a great outcome of such a test. It's like driving a car for the first time and then telling everybody that it's faster than walking.