[pg]

Does anyone out there have an answer from actual A/B testing?

[blauwbilgorgel]

http://www.conversiondoctor.com/conversion-blog/coupon-codes...

  TEST: (Straight A/B test for an online retailer in the women’s clothing market)

  Control: Coupon code on the first page of the checkout process.

  Variation 1: Coupon code removed.

  Results:

  Control: 3.8% conversion rate. (967 sales / 25,489 unique visitors)

  Variation 1: 5.1% conversion rate. (1,276 sales / 24,991 unique visitors)

[nrao123]

@pg- see Page 2 & 3 of this paper for actual A/B testing results on the coupon code issue

http://cginsights.posterous.com/hippo-ab-testing

[marksbren]

Good find. Here is a summary for those who don't want to dig though the Scribd document.

Here are the two versions: http://dl.dropbox.com/u/493143/ABTest.png (A = Old Page, B = New Page)

1. Doctor FootCare updated their checkout page to include a coupon code field (and a few other minor changes)

2. With the updated version, Doctor FootCare saw a 90% decrease in revenue.

3. Once Doctor FootCare took the coupon code out of the new version (B) the conversion rate of the page was 6.5% higher than the old checkout page (A). The new page without the coupon code field is not pictured.

That is a huge change, but I would not blindly assume that this result proves that the coupon code field should not be included in your checkout form. I would also like more data about the test.

[yahelc]

Haven't tested this specifically, but, every test I've ever done (that had statistically significant results) has shown "Less fields in a form is always better".

[efsavage]

"Less fields in a form is always better" is not entirely accurate. I'd agree with "Less fields in a form increases the number of people that complete the form", but this is not necessarily "better".

It does not mean that people will actually read the newsletter or complete the checkout process or come back and spend more money, those are the real metrics to track and rarely have anything to do with how many people submit the form.

[davidwparker]

I think this is about the time that Patrick (patio11) chimes in :)

[patio11]

Sorry to disappoint. I have done a few A/B tests in my time, but that doesn't necessarily mean I have a comparable that I can share for any element which could possibly be A/B tested.

I would also caution you that, hypothetically assuming I had a comparable here, I might be a little hesitant to say "An A/B test in June 2010 of the behavior of elementary school teachers on summer holiday ready to get their Back to School Bingo started is a great way to predict the behavior of poor startup geeks contemplating a recurring billing arrangement."

The point of A/B testing isn't to anoint someone as the local genius on design and conversion optimization. The takeaway -- over and over and over again -- is that your local genius is routinely wrong about applying their old experience or intuition to new problems.

People pay me good money for advice on things like this question. I give answers, appropriately couched as "My best guess as to how this will play out". A lot of them whiff when exposed to actual customers.