Questions? (877) 643-7244
FutureNow Post
Friday, Nov. 5, 2010 at 9:11 am

Web Site Change Worth the Effort? First Test, Then Invest.

By Whitney Wilding
November 5th, 2010

For a few of our clients, implementing some of our recommendations can be a bit of a challenge due to the limited resources they have. So, how can you decide if implementing a certain change to your site is worth the the time, effort and money? This quandary highlights the importance of being able to attribute a particular value to a change. Not only does the value of an change impact the decision to implement it across a web site, but it extends to the priority you give that particular change too. While you may have good reasons for that nagging hunch you have, there is nothing like a good ol’ test to prove it right! Having witnessed the portability of Brendan Regan’s most recent, rhyming tip, I will put my sage advice in poetic form too: first test, then invest!

A Client’s Predicament

Product Page-Shopping CartAll of our recommendations are born of a client’s analytic data. As an example, let’s consider a product page from our client, Curious Country Creations, who sells all variety of decorative dried plants (in case you feel the sudden urge to buy pine cones). The conversion funnel report in Google Analytics, left (click to enlarge), indicated a 91% drop off rate from the product pages to the cart. While a large drop in visitors at this step is common for e-commerce sites, it leaves a lot of room for improvement. A brief heuristic analysis of the page revealed an absence of customer ratings and reviews for the products. We have known for some time how valuable reviews are for e-commerce sites. Still, we understand that implementing customer reviews and ratings on product pages may require a large portion of a client’s resources; validating this addition improves the click-through rate, quantifying the potential gain, and comparing it to the cost of implementation are important .

An Analyst’s Solution

Product Page with ReviewsWhat better way to understand the value of the recommendation than to test it? Rather than blindly forging ahead with a site-wide implementation of customer reviews, we asked the client to test the change by taking these steps:

  1. select a highly-trafficked product page, right (click to enlarge),
  2. gather customer reviews of the product
  3. set up an A/B split test on the page, comparing the original (no reviews) vs. the variation (with customer reviews)
  4. if the new variation wins, determine the conversion rate to sale (ie – not just click-through rate on the two pages in question, but all the way through the shopping cart too) for both the group that saw the original page, and the group that saw the new version
  5. multiply the average amount of traffic to hit that product page each month by the conversion rate to sale for the old page to get the total number of sales it would generate in a month; do the same calculation for the new page variation
  6. subtract estimated monthly sales generated by the old page from estimated monthly sales generated by the new page to calculate the additional monthly sales generated by the new variation
  7. using your average order value (AOV) and your profit margin (if possible), calculate the estimated revenue increase associated with these additional sales
  8. compare the cost of implementing the change to the revenue generated by the change

Our initial indicator of success was a direct comparison of the percentage of prospects who moved forward from the product page to the shopping cart. Our hypothesis withstood the test: having customer product reviews on this page significantly increased the number of visitors that entered the shopping cart, and thus the number of sales. Additionally, our calculations showed that implementing customer reviews on this page alone generates an estimated additional $3,276.09 a month, or $39,313.08 a year, for this client. Furthermore, the change demonstrates excellent ROI, making site-wide implementation a clear win. Just imagine the potential revenue increase after featuring reviews site-wide!

The Take-Away

Now think big picture. The steps above apply not only to the example we used, but to any change you are considering on your site. Step #2 will differ depending on the change you are considering, and the particular page you select at step #1 may be different, but the idea remains the same: identify a high-traffic page representative of the group of pages you are thinking about changing; gather the materials you need to make the change. Don’t overlook these two other factors at step #4 either:

  • It is possible that some changes you make to a page will affect not only the click-through rate on that page, but will impact the click-through rate of future pages in the click-stream of a visitor as well.
  • The further away from the shopping cart (earlier in the buying process) you get, the more complex (less linear) scenarios can be.

Of course, we have a methodology for dealing with that complexity, so if you find yourself getting overwhelmed, don’t be too shy to ask how we can help.

If you are not sure a change to your site is worth the trouble, first test, then invest. Not only will you be able to quantify the value of making the change, but you will also be able to prioritize that change relative to the other items on your implementation list.


Add Your Comments

Comments (26)

  1. There are some great points brought up in this post. Blindly making wholesale changes without testing first is certainly asking for trouble, especially when budgets are thin.

  2. There are some great points brought up in this post. Blindly making wholesale changes without testing first is certainly asking for trouble, especially when budgets are thin.

  3. I completely agree with this idea, but one thing I don’t get is how you it will take less work to implement this change on one page in comparison to all of them, since I assume this is a database driven site? My guess would be that to implement it on one page and on the entire site would take more or less the same amount of time??

    That being said, I do see value in testing in case the new variant would perform poorly in comparison, but I don’t see the time issue you’re referring to (unless of cause you have a static html site where you’ll need to edit every single page manually).

  4. Our company has been weighing whether or not to overhaul our website. This information is very helpful and I look to introduce some of the points that it contains. Thanks.

  5. Great case study. I find it particularly interesting because that site is running what appears to be a stock standard version of Zen Cart or perhaps osCommerce, something which I had the unfortunate experience with…!

  6. Testing is a must. I have made changes to some of my websites in the past without testing first and lost major profits. Test then invest is certainly the way to do it

  7. I´m surprised that no customer insights are used to test reviews on the landingpage. I would think that it would be better to get your ideas of what to test from feedback from customers (by analyzing surveys, emails, call logs etc). Looking forward to your opinion on this.

  8. “Not only does the value of an change impact the decision to implement it across a web site, but it extends to the priority you give that particular change too.”

    Reminds me of the “Heisenberg uncertainty principle”.

    The A/B split testing is an excellent idea, but what are the steps to implementing, say in WordPress?

  9. You make great points and although many people think their website requires certain changes, it very well may not be the case. Your point of “first test, then invest” holds true for nearly all websites.

  10. Already try this method. Its rather difficult and waste the time at first. However, once you did it successfully, you’ll be enjoy and found you’re facing the worth effort

  11. I agree…I am on a shoestring budget and have to implement strategies with the biggest bang. I am a very big follower of the test it first, then put it into action–rather than just putting the chips “all in” and crossing my fingers.

  12. Another relevant question is “Will your Unique Selling Proposition Shine with this proposed change?

    Some marketers are even drastically changing their blogs or leaving their blogs to microblog or post more videos to significantly improve their online presence.

    On the other hand, there is Seth Godin who refuses to change the simplicity of his blog format and it works for him! Can’t imagine a seth godin blog cluttered with photos, videos, and etc.

    So think first if the proposed changed will advance your USP. Not worth it to gain momentary traffic but lose your USP in the long-run.

  13. @Michael Thomas – we have found that a program of continuous optimization often produces better results than a complete website overhaul done as a one-off project. This is because a total overhaul typically doesn’t involve any change in strategic planning, and throws away the good with the bad to build a site without any more insight than the last one. Continuous CRO focuses on identifying what is working well (keep it) and what isn’t (improve it), and has a strategy that validates every change made to the website with a quantifiable win.

  14. But, it takes more time to implement a change on the site. right?

  15. You are absolutely right: split tests are a must but… first test small changes then bigger ones. Anyway what if things go bad? You wasted time. There is no solution to that, I guess.

  16. I am trying it two times but i did not get any fruitful result.It’s only time waste.

  17. On my website, I have added custoemr testimonials to try and provide a comfort factor to our web visitors. Conversions are still slow despite this. We tried video testimonials but our custoerms would not agree to do it.

  18. @House Alarm Expert – I am sorry to hear you are struggling to find ways to improve your conversion rate. What works for one company’s website, may not always work for another. That is one of the reasons we recommend testing. There is a lot of support for the impact customer reviews can have on conversion, but they certainly are not the only factor on a site that impacts conversion. Perhaps there are other mitigating issues that are preventing you from realizing your full potential.

  19. It’s always a good idea to update a website if you feel the design is outdated. People don’t seem to put enough importance into the impact a bad design can have on a visitor’s first impression. Key is to find a quality designer and developer to drive it home

  20. I agree with Herbalife. If you feel your website is outdated, change it. Apply plug ins and applications which you think are useful and could make your website more attractive. People tend to have the first impression on the website itself rather than the content. And most of all, use good keywords to optimize web traffic.

  21. There where a full support for the impact customer reviews can have on conversion, but they certainly are not the only factor on a site that impacts conversion.

  22. some excellent points here. Driving customers where you want them has been difficult since the birth of ecommerce. Many many hours studying webtrends :(

  23. You can only test so much before you really have to just jump in with both feet and maybe take a few hard knocks, but the key is to minimize those hard knocks through sound strategic preparation.

  24. One can spend so much time on testing and still not get the higher conversions they’re looking for, when the real problem is content. I’ve seen some pretty sloppy sites with great content that out sells the “pretty” site. I really think it’s all about the content.

  25. @Easy Violin Lessons – this very well may be true. Content changes to a website are changes nonetheless, and can be tested like any other element of your site. Also, bear in mind that even negative test results, or no movement on the needle, give you some kind of learning. Negative results tell you you’ve hit on an element of your site that has the ability to affect conversion. If you are not seeing results from the tests you are running, there may be a problem with your testing strategy. Ask yourself some questions about how you have determined what to test: Was your decision about what to test and your hypothesis based on informed (rooted in data from your web site) observations? Are you testing high impact areas of your site?… Or are you just testing willy nilly; testing everything? Are you just testing things that worked for someone else; are on a list of best practices?

  26. useful tips, but not a total solution , there is always a chance of profit or loss. we have to reduce the loss chance.

Add Your Comments


Print this Article

More articles from Whitney Wilding

Marketing Optimization Blog
FREE Newsletter Sign-Up
send it once every: