[jspash]

I couldn't agree more. Simply spending the time to understand why you are testing certain elements is valuable on it's own.

The author doesn't mention what kind of tests were performed, but often when I read about the "futility" of A/B testing, it's usually due to lack of up front preparation and discussion of the tests true objectives. So the classic "red button vs. green button" might get you a result in the testing software, but it doesn't necessarily translate to more sales/leads or whatever the ultimate goal is.

[ac132]

The post wasn't about the "futility" of a/b testing. It's about the costs of a/b testing that are rarely considered. There's plenty of "pro" a/b testing advice in the world that leads startups to mistakenly waste resources on it.

A solid a/b testing process doesn't just happen without time & effort. You need someone smart analyzing user behavior and forming hypotheses as well as a talented design team to develop concepts that test them.

We simply acknowledged the costs of doing a/b testing well and decided our resources are better spent elsewhere.