| True, most A/B tests fail. However looking at the listed tests, they mostly seem to be general branding tests. Changing your branding, including fairly major redesigns, reliably fails to make a difference. But lots of other kinds of changes reliably do make a difference. Big button. Putting signup link in multiple places. The word "Free". Streamlining your form. (No, you don't want to ask for first/last name.) Email headlines. Providing relevant results first. However they make small differences. You need to plan on tens of thousands of successful conversions in your test. You can make this easier by running multiple tests at once. If you're a small company, you can make conversion be "to the next step in the conversion funnel" instead of "to the end". (But be warned, companies that test long enough and get large enough tend to find examples of cases where that short cut burned them.) However it is very important to set expectations. Expect to find few big wins. Expect most to be modest wins. Keep track. For an established company, if A/B testing is adding 5-20% to your bottom line each year, it is totally worthwhile. (If you have traffic and have not been testing, then odds are that your first year will find more than a 20% boost. But this declines in future years.) A/B testing is a valuable tool. It isn't magic pixie dust. |