Don’t Dismiss the Base Hits

September 3rd, 2008

Readers of this blog, and especially those involved in testing, know that conversion rate optimization is the goal we’re after. It’s a great feeling to know that a test you worked on increased conversion, especially when it’s a “Home Run.”

We define Home Runs as triple-digit increases in conversion rate. When we help our clients hit home runs, we ring the bell and celebrate.

But one of the dangers of early testing efforts is the problem that some baseball players have: “swinging for the fences.” (Apologies for the baseball references, but it’s getting to be that time of year.) What if your test, or series of tests, doesn’t appear to raise conversion rate? Do you dismiss the tests as failures because they’re not home runs?

Of course not!

Worst case scenario is that you’ve learned something about executing meaningful tests, and about what does or doesn’t resonate with your customers. But more often than not, you are affecting your website in more subtle ways. Remember that conversion rate is often a blended, averaged, blunt instrument. Especially when it’s averaged across large volumes of organic search traffic, SEM traffic, email house list traffic, different product lines, etc.

Here are some things you can monitor when your tests aren’t having huge impacts on your overall conversion rate:

Micro-conversion rates - If you’re testing product detail page layouts and “Add to Cart” buttons, check if those test variables are having an effect on the micro-conversion rate of adding products to the cart.
Funnel conversion rates - If you’re testing lots of minor copy changes to your shopping cart, check for changes in your funnel conversion rate.
Bounce rates - If you’re testing images, copy, or other changes designed to build up the credibility of your site, watch for changes in bounce rates.

These types of incremental improvements are tests results to get excited about! If your micro-conversion rate increases, and your funnel conversion rate stays the same, that’s still more money in your bank account. If you reduce the bounce rate, you’ve gained the chance to convert that customer later, instead of your competitor.

So don’t dismiss the base hits because you’re disappointed about not hitting a home run (this time.) Take it from a patient analyst who’s favorite baseball player was famous for lots of base hits and not all that many home runs

In A/B Testing, Analytics, Bounce Rate, Conversion Rates, Improving Conversion, Key Performance Indicators

Add Your Comments

Comments (5)

Ben
September 3rd, 2008
11:09 am

I was with you until the last sentence. Ted Williams hit 521 career home runs. That’s not too shabby…
JoshBaker
September 3rd, 2008
11:21 am

Great points! You’re a glass is half-full type of guy!

One of the things I like to do in my after test summary documents are not only to report on how the test performed against my test question/goal, but document what additional information was learned from the test that some may consider secondary or third to the desired test. In many instances, I have been able to take commonalities that I see in the 3 areas you mentioned (both successes and failures) and compare to past tests on the same or similar pages and see if there is any pattern, sometimes this helps provide some input for designing our next round of testing - maybe just maybe we were testing the wrong thing initially (by wrong I mean maybe the opportunity was not at were we initially thought our data pointed us to, but in a secondary “position”), and may can help us uncover where the true “home run” opportunity may really be at.
Brendan
September 4th, 2008
11:04 am

Ben: Can’t control my favorites, but maybe I should’ve used Ty Cobb?

JoshBaker: Great comment, we too reserve a part of reports for learnings, particularly “ideas for follow-up tests.”
Paul Willard
September 8th, 2008
1:04 pm

I would add segmentation to the list of things to measure. Just because your overall conversion rate did not change does not mean that conversion in some segments did not go way up or down. If you use segments that your web server is aware of, you can show different behavior on your site to different visitors and optimize conversion for each little segment. Examples of things your web server can be aware of include:
Referal source - the url email link vs search vs display vs the red banner vs the green banner, etc
behavior on your site - visitors that looked at digital cameras, people that registered for your newsletter, folks that have previously visited, previously purchased, etc
Rummikub
November 4th, 2008
10:40 am

Great addition Paul. I usually monitor Bounce rates for each page but I really need to A/B test al lot more on each page to increase conversions.

Comments (5)

Add Your Comments

About the author

Related Posts:

Around the Net: Around the Netâ�¦