[gk1]

I've shared some thoughts on this previously: http://www.gkogan.co/blog/test-big-changes/

TL;DR - Test big, drastic changes instead of fiddling with button colors and headlines. Examples of drastic changes:

- Entirely different homepage with different messaging.

- Change "Features" page to "Benefits" page and change its content accordingly.

- If you're offering a free download or free product, test asking for an email first.

- If your SaaS has a long signup form, test allowing people to jump right in with just an email address.

- If you're sending a robotic "welcome" email, test sending a very short personalized email instead.

[ssharp]

This has been my mantra as well. Test big and you'll likely fail or win a lot faster.

I'd also suggest making sure you approach testing by starting with a hypothesis and then testing that hypothesis. By voicing or writing down your hypothesis, you will be more likely to avoid the "stupid test trap", a phrase I use to describe running tests like changing button colors -- what hypothesis is changing button colors trying to prove? That people like red more, "for reasons"?

As mentioned above, each A/B test you run has a serious opportunity cost. I'm fortunate enough to have thousands of people going through my sales funnel daily but, even then, a test will still take several weeks or months to validate and anything approaching a change of 5% or less will really eat away at your ability to test more meaningful things.