A/B Testing: Can you afford to ignore three-digit percent improvements?

Most analysts agree that Obama’s 2008 presidential campaign was won due to a series of successful A/B tests that helped improve voters’ perception of his candidacy. For marketers looking to increase conversion rates and revenue, A/B testing has proven itself to be the ultimate tool. Low cost and easy to use, the optimization method has earned a reputation for delivering three-digit percent improvements on website performance and for being applicable to a wide range of scenarios, from retail to political campaigns.

As our global village becomes more virtual and connected every day, the importance of doing A/B testing  also grows. Customers are more empowered than ever as product choices seem infinite and stand just a click away, removing switching barriers and search costs. Because of that, retailers’ only solution for consistent sales is to differentiate themselves by providing an outstanding customer experience that incites repeat purchases, building brand loyalty and increasing customer lifetime value.

In the past, brands relied on customer surveys to assess consumer preferences and shape their offerings and strategy. We now know, however, that purchase decisions are driven not only by rational factors that can be externalized in surveys, but also by emotional and irrational complexities that cannot be accurately predicted. That’s where A/B testing comes in, allowing marketers to experiment with all aspects of their websites and assemble the ones that appeal the most to customers to create the ultimate virtual experience.

So how can ecommerce retailers take full advantage of A/B testing to boost conversions and sales and what should they keep in mind before they start A/B testing? Here are best practices:


  • Familiarize yourself with your website’s problem areas

Before you start testing, you need to know what to test. Spend some time auditing your website to identify which pages are underperforming – those are probably the ones that need a makeover and would benefit the most from A/B testing.


  • Know your base line (key metrics)

Record how the page you wish to test is performing before you start running tests, so that you are able to compare the test results to your initial base line. It is important to know which key metrics are relevant to your objective. For retail websites, it is useful to track the number of page views, the conversion rate, the add to cart rate, the average order value and/or the revenue per visitor.  Other useful metrics might be cost of acquisition per new customer and life-time customer value. For further reading on analytics, check “How E-Commerce Has Reshaped the Black Friday/Cyber Monday Weekend


  • Test pages that have high traffic

The key to a successful A/B test is to have a large enough audience so that your results are statistically relevant and you are able to draw meaningful and actionable conclusions. The more traffic your page attracts, the faster you will be able to gather statistically significant results to base your decisions on. A rule of thumb is you should have a minimum of 100 visitors to each landing page and at least 20 conversions.


  • Don’t test minuscule changes

Focus your efforts on meaningful, high-visibility elements that can drive significant change and produce impactful results. Minor details will likely result in statistically insignificant test results.


  • QA your experiment to make sure it works as expected

Before you go live with your test, make sure to QA and see if everything is working as you had envisioned. The last thing you want is to start your test only to realize there’s an error that may screw your results.


  • Test both versions simultaneously

In order to achieve concrete results, it is important to always split traffic between the two versions so that they are being tested simultaneously. Do not wait to test one version and then the other, or you might have external factors cloud impact your results.


  • Test one element at a time

While it is possible to test multiple changes at once, it is best to test one change at a time. The reason is this way you can clearly identify the elements responsible for the test results. In a multivariable test you cannot point out which of the changes directly influenced the webpage’s performance.


  • Add up test results and apply your learnings to other pages of your website

The key to achieving a huge boost in conversion rates is to A/B test multiple aspects of your website, combine the winning variations and apply these positive changes wherever possible. The results from one test can be combined and carried over to similar pages.

Remember that whatever the outcome of your test is, A/B testing should be an on-going initiative to ensure your website experience is always optimal and you are making the most out of your existing web traffic to increase customer loyalty and repeat purchases.

For more specific examples on how A/B testing can increase conversions and boost sales, watch our on-demand webinar: click here.

Total Views: 1800 ,

Leave a Reply

Your email address will not be published. Required fields are marked *