The ABCs of A/B Testing

My friend Cindy has taken up hiking, and, being a shoe kinda gal, she has taken an intense interest in hiking boots. She's the first one who will tell you, "They may be a dream on the shelf, but you really have no idea how they're going to work for you until you start trail testing them."

You could say the same thing about ebusiness. When you start doing business online, you enter the realm of "trail testing." And here is where you discover that the practical work of managing any online enterprise has only just begun. Because no matter how thoroughly you think you've covered your bases in the development phases of your Web site or your email campaigns, you are about to discover countless opportunities for improvement.

How do you take that first step in monitoring your performance based on all the little tweaks and fidgets you think will help? Ah, it's as simple as A, B ... and no C!

Maximizing your conversion rate is not simply a matter of making changes, it's about making

a) the right changes,

b) at the right time,

c) in the right sequence, and then

d) evaluating the results before continuing the process.

If you are not methodical in your approach to change, much of your effort will be wasted. So take your time, and keep these guidelines in mind as you test:

A/B testing - sometimes it's called an A/B split - is the simplest and easiest method of testing elements in your emails or on your Web site. You divide your audience into two groups. You expose one group to the original version of whatever you are testing. You expose the other group to an alternative version, in which only one element has been changed. Then you track the results.

For example, suppose you want to figure out the best subject line for your promotional email. Prepare two separate emails, identical except for the subject line. The email with the first subject line goes to half your list, while the email with the second subject line goes to the other half.

To gauge the effectiveness of your subject line, compare the open rates between the two groups. Once you find your winner, it becomes your control or benchmark. Test it against another subject line, and so on, until you are convinced you've found the best possible subject line for your messages. And the results may surprise you. In one of our testing scenarios, one subject line generated an open rate 300% higher than its closest competitor. A difference like that can have a tremendous impact on your bottom line!

What should you be testing? It's easier to make a list of what you don't need to worry about, because that would be a blank sheet of paper. But here are some possibilities to get you started:

Emails: bonus gifts, coupons, P.S. messages, guarantees, opening sentence image, closing sentence image, from-field, calls to action, opening greetings, type styles, layout elements, graphic images, etc.

Web Sites: landing pages, language of copy (headings, body, calls to action, assurances), colors, location of elements, look/feel, hyperlinks, etc.

A/B testing is far from rocket science (and there are other more complicated and robust ways to test), but it has a sweet advantage: it isn't complicated. More importantly, it means you won't have to make potentially expensive decisions based on your gut reaction. You'll know, because you can say, "Here, look at the numbers!"

How do you prefer to be addressed:

Your email address



We Value Your Privacy!


Forward this issue of the Grok to a friend!

  Your Privacy is Guaranteed

1999-2002 Future Now - Publishers of GrokDotCom. Your Privacy is Guaranteed. We will never give, lease or sell your personal information. Period!