A/B Test Case Study: Homepage
This post is contributed by Janis Lanka (@janislanka, who manages front-end development for Elastic Path Software.
This post is a continuation of a series of posts related to conversion optimization for the Official Vancouver 2010 Olympic Store. Following checkout process and product details page optimization, in collaboration with Wider Funnel, we looked at the store’s homepage. Following list of hypotheses were made:
- Too many banner spaces create high clutter
- Secondary (left side) navigation doesn’t stand out and is difficult to navigate
- Product photos are too small with no indication on available alternative colors
Control
(Click to enlarge, will open new page)
Variation A
(Click to enlarge, will open new page)
Variation B
(Click to enlarge, will open new page)
As a result, we produced two alternative variations with following changes:
- Reduced banner amount and increased size to improve prominence of each banner
- Increased prominence and clarity of secondary navigation
- Provided color thumbnails to products that have alternative colors
- Increased size of photos and reduced amount of products shown under New Arrivals, Featured Products, and Most Popular tabs
What We Learned
This was a very tough test where even 2053 transactions and 21 days did not provide a statistically significant winner. However, decision had to be made and Variation A was chosen based on following data:
- Variation A converted (GWO) 3.14% better than control variation
- Visits with Variation A resulted in 12.54% less Bounce Rate
- Overall site Conversion Rate was increased by 0.59%
- Average Order Value was increased by 5.16%
As a bonus point in our findings (to put it in perspective), if hypothetically we would be using the winning AOV and Conversion Rate, revenue would be increased by 5.78%.
Finally, a thing we learned which might be already obvious for some: use banner space very strategically. Based on your overall strategy – be that to increase AOV or Conversion Rate – you will need to choose carefully what to advertise and where to send users.
You may also like these similar posts:
Original post by Janis Lanka
<-- ads -->