The ABCs of A/B Testing

My friend Cindy has taken up hiking, and, being a shoe kinda gal, she has taken an intense interest in hiking boots. She's the first one who will tell you, "They may be a dream on the shelf, but you really have no idea how they're going to work for you until you start trail testing them."

You could say the same thing about ebusiness. When you start doing business online, you enter the realm of "trail testing." And here is where you discover that the practical work of managing any online enterprise has only just begun. Because no matter how thoroughly you think you've covered your bases in the development phases of your Web site or your email campaigns, you are about to discover countless opportunities for improvement.

How do you take that first step in monitoring your performance based on all the little tweaks and fidgets you think will help? Ah, it's as simple as A, B ... and no C!

Maximizing your conversion rate is not simply a matter of making changes, it's about making

a) the right changes,

b) at the right time,

c) in the right sequence, and then

d) evaluating the results before continuing the process.

If you are not methodical in your approach to change, much of your effort will be wasted. So take your time, and keep these guidelines in mind as you test:

A/B testing - sometimes it's called an A/B split - is the simplest and easiest method of testing elements in your emails or on your Web site. You divide your audience into two groups. You expose one group to the original version of whatever you are testing. You expose the other group to an alternative version, in which only one element has been changed. Then you track the results.

For example, suppose you want to figure out the best subject line for your promotional email. Prepare two separate emails, identical except for the subject line. The email with the first subject line goes to half your list, while the email with the second subject line goes to the other half.

To gauge the effectiveness of your subject line, compare the open rates between the two groups. Once you find your winner, it becomes your control or benchmark. Test it against another subject line, and so on, until you are convinced you've found the best possible subject line for your messages. And the results may surprise you. In one of our testing scenarios, one subject line generated an open rate 300% higher than its closest competitor. A difference like that can have a tremendous impact on your bottom line!

What should you be testing? It's easier to make a list of what you don't need to worry about, because that would be a blank sheet of paper. But here are some possibilities to get you started:

Emails: bonus gifts, coupons, P.S. messages, guarantees, opening sentence image, closing sentence image, from-field, calls to action, opening greetings, type styles, layout elements, graphic images, etc.

Web Sites: landing pages, language of copy (headings, body, calls to action, assurances), colors, location of elements, look/feel, hyperlinks, etc.

A/B testing is far from rocket science (and there are other more complicated and robust ways to test), but it has a sweet advantage: it isn't complicated. More importantly, it means you won't have to make potentially expensive decisions based on your gut reaction. You'll know, because you can say, "Here, look at the numbers!"

click here for a printable version of this entire article

Latest News: AOL is The #1 Converting Search Engine?

Our recently published study reflects more than 29 million search engine visits to dozens of e-commerce sites using WebSideStoryís outsourced HitBox services ( The e-commerce sites used in the study cover a broad range of products and services, everything from gifts and electronics to shoes and office equipment.

Why does AOL beat out the likes of MSN, Yahoo and Google? Our analysis answer why different search engines have such different conversion rates, the factors that may be impacting those conversion rates and what marketers can do to improve their results. Find our more about our exclusive study Converting Search Engine Traffic: A Q4 2003 Conversion Rate Marketing Report for Online Retailers.

Final Notice on Our Copywriting Contest!

Youíve heard me say it often enough, ďSpeak to the dog in the language of the dog about what matters to the heart of the dog.Ē Meat matters to the dog. Thatís what gets the dog so excited he canít wait to get his teeth into that meat.

For this contest, weíre asking you to write sales copy to promote that splendid book Persuasive Online Copywriting (yours truly on the cover, no less). But donít write just any old copy. Write copy that will do the same thing that meat does for the dog. Identify the critical message, in language folks will find appealing, that gets them excited about sinking their teeth into this book.

You may submit as many entries as you like up until the deadline, November 17, 2003. Send them to Weíll fly the winner from anywhere in the 48 states to New York City, expenses paid, for a day of in-depth copywriting training at the offices of Future Now.

Have you checked out the other places to meet us on our latest event schedule?


Keep ĎEm in the Cart 

If youíve got a shopping cart Ė or bag or basket or whatever Ė youíre probably aware of a little problem.† Not everyone who enters your checkout process manages to complete it.† Thatís why, when I talk about the buying process, I make a distinction between deciding to buy and actually completing the purchase.

Between those two acts, your visitor must navigate the obstacle course of your shopping cart process, yet another contained conversion system (typically a third-party software solution) that requires multiple clicks and the willingness to part with some amount of personal information.

But, like I said.† You are probably intimately aware of your shopping cart abandonment rates, because you monitor this sort of useful information.† Right?† Of course you do.

So today Iím going to ask you this question:† Should you be turning your attention to refurbishing your shopping cart?†

A Matter of Priorities

Itís an important question:†before you do anything, decide if the next issue you need to address is shopping cart abandonment.† Yes, you are losing folks there.† Everyone does.† There is no 100% conversion rate because this is not a perfect world.† Even when youíve covered every base imaginable, humans will still leave for reasons you canít do anything about.

But if your Web site is typical, you donít lose the majority of folks from your shopping cart.† You lose them much earlier in the process Ė at the home page, or just a page or two in.† If this is you, Iíd say your efforts are better spent, for now, encouraging more folks to stay with your buying/selling process.†

Because fewer folks bail out of your shopping cart compared to the number who bail out before they even get to checkout, improving your abandonment rates might take you from, letís say, 3% to 3.3%.† Sure, the difference is not to sneeze at.† But keeping more visitors in your conversion process from the get go helps you increase your conversion rate by multiples.

The Metrics to Help You Decide†

If you are on top of your Web analytics, youíll know where your biggest problems are.† These are the key metrics that will help you decide where you should be directing your energies.

You really want to consider these pages before anything else.† What is wrong and how can you remedy it?† I mean, weíre talking folks who werenít even inspired to look a single page further into your site!

If they donít get to the shopping cart, you certainly donít need to worry about them abandoning it.† But letís give them the chance.† Look at the pages they reject and evaluate how to improve things.

How Much Moneyís Left on the Table?

We get lots of questions about how to solve the problem of shopping cart abandonment.† But sometimes you donít necessarily need an answer to this question Ė you need a better question.†

Much better results, relatively speaking, can be yours through providing relevant and persuasive content throughout your Website:†content that is based on an understanding of your visitorsí intentions and needs, content that instills confidence, content that reassures, content that makes an emotional connection.† Visitors who have been thoroughly persuaded demonstrate uncanny motivation to navigate even the worst checkout process!

But, letís face it.†Money on the table is money that isnít in your pocket.† So, if youíve got a tidy little cybershop and want to improve your shopping cart abandonment rates, then Iíve got any number of ideas for you.† Next time!

click here for a printable version of this entire article


P.S. If you enjoyed this issue, why not share it with your colleagues and friends?
They'll appreciate it. Forward This Issue To A Friend!