Our recommendations aren’t pulled out of a list of best practices.  Our analysis and recommendations follow a process, linking everything back to a piece of data.  This is why data from your analytics is so important in optimization.  If you can’t draw a direct line between why you’re changing something on your site, and a piece of data, make sure you recognize this as a risky move, and track it very carefully.  Learning how to properly track information in an analytic program like Google Analytics, and crunching numbers to isolate potential problems are both blogged about often.  Searching for these types of articles (hint: His last name is Kaushik) will help more experienced DIYers find some of the more sophisticated tips that they’re looking for.  It won’t be as easy as “make your button stand out” or “make sure your security assurances are click-able,” but it’s useful information that’s out there to be consumed.

What I mean is that once you get beyond the basics, optimization is no longer a one-size-fits-all solution, just like there is no average conversion rate that you should measure yourself against.  So, when you look at examples of problems and solutions that are more complicated, the direction you get from those endeavors is much more likely to be about how to approach a particular kind of problem than it is to be a specific “change X to Y” kind of learning.  Why is that?  Well, let’s revisit my lead-in statement: I can’t give you recommendations that’ll revolutionize your conversion rate without looking at your data. The operative words in that statement are “you” and “your.”

Contrary to popular belief, optimization is not over after you’ve eliminated basic problems that every site needs to account for (e.g. load time, font size and legibility, visible calls to action, security assurances, etc).  But, beyond that point is where the real fun begins, and where the deeper opportunities lie.  That’s when conversion optimization transitions from sheer mimicry into a complex dance where the particulars of a situation drive how you apply overarching principles and practices.  We can talk to you about what those principles and practices are; show you some examples of those principles and practices being applied.  But you can’t just do what we do and expect it to work for you, because you’re already beyond the point where you’re just like everybody else.

That’s right: you ARE special. You’ve been waiting for someone to say it, and now it’s finally out there.  Now we can commence with our love affair.  In fact, it’s because your business is unique that optimization sometimes can be so difficult: what worked for Sally’s website might not work for yours, because you are not Sally. That doesn’t mean that you can’t learn something from Sally’s struggles.  You can learn the techniques that Sally used to resolve them.  You can read about how Sally struggled, and the progress she made all by herself.  And you can also investigate the experts that Sally used to help her learn how to look at her site and data to break through the barriers that held her back.

If you want to use Sally’s deeper learning to try to shed light on your own optimization efforts, you have to use your instinct.  You have to ask yourself, “Am I just like Sally?” or, “Am I similar enough to Sally?”, and, “How might I be different from Sally and what does that mean for how I attack this problem?”

Instinct can be a very powerful thing for those who have some confidence in optimization, but make sure you have a back-up plan as well.  A great security measure for instinct is testing.  Testing can save you from making a wrong move when trying to optimize your site. However, it’s also one of the easiest things to mess up if you don’t know how to do it properly.  Thankfully, there is a ton of information out there on how to run tests!  Creating a scientifically valid test is the key to creating a ’successful’ test.  (Successful in this case = results that can be trusted, not necessarily results that generate you money.)  Numerous blog posts give pointers about testing calculators that determine how long a test must to run to achieve significance, creating tests that will give you the clarity you seek, and testing when you have lower traffic.

So, to you experienced DIY optimizers: don’t lose faith when you read yet another article on best practices; just understand that the “beyond-the-basics” direction you’re looking for is hard to come by because it has to be tailored to you.  It doesn’t mean you can’t try those kinds of things on your own.  You certainly can.  But please recognize that if you find yourself wishing this blog post went into more detail about someone “just like” you, with a problem “just like” yours, then that’s probably a signal that you need some expert guidance. There is no shame in admitting that.  And believe me: you’re worth it!

We often get asked:

How much traffic do I need before I start optimization?

Do I get enough traffic to my site to even consider optimization?

Will my tests take forever to run if I don’t have a lot of traffic?

How fast can I expect to see the results of optimization if my traffic is low?

These are natural questions from folks who haven’t dipped their toes into the optimization waters, but these questions indicate a few myths that I’ll attempt to dispel.

Myth #1: Traffic is the ‘magic metric’ when it comes to optimization

The fact is that the number of conversions is at least as important as traffic levels.  We’ve always said that a minimum threshold to do formal online testing is 5-10 conversions per week, and that this amount of conversions (regardless of traffic) can at least get you results in a matter of weeks instead of months.  A simple tool that illustrates the relationships between site traffic, conversion rate, and test durations is the Google Website Optimizer Duration Calculator.  This free tool lets you play around with data points and estimate test durations before you start your experiment.  I can personally say it has saved me a bunch of grief by warning me away from certain client experiments due to the duration estimates being way too long!  Try changing the page views metric, and see the impact on duration.  Now, try changing the conversion rate metric, and you’ll see it, too, can have a drastic impact on the duration.

Myth #2: Low-traffic sites won’t see results from optimization

The fact is that low-traffic sites can still see improvements from optimization.  Remember that “optimization” doesn’t just mean formal testing using a tool like those provided by Omniture or Google.  Optimization is about getting your company on a program of continuous improvement through hypothesis, change (via formal testing or not), and monitoring of the results of those changes.  Once you have “results,” you feed those learnings right back into the program and keep going.  Lower traffic sites may or may not be good candidates for formal testing methods like split testing or full factorial multivariate testing, but they may be great candidates for optimization!  We often try formal testing on low-traffic sites, and if it’s not productive, we switch our focus to “serial testing,” which means benchmarking the performance of something (an ad, a landing page, etc.), making a change, and then monitoring the impact of that change. Either way, we tend to get results.

Myth #3: Testing and optimization doesn’t really work for low-traffic sites

The fact is that optimization can definitely work, and even formal testing can work if you do it right.  One way to do formal testing on low-traffic sites is to focus on higher-trafficked pages.  Another is to test micro-conversions (e.g. reading product reviews as opposed to buying the product).  Still another is to make a B2C “view cart” page the conversion point instead of the purchase confirmation page.  There are plenty of methods to shortening the duration of a test, but we’ll keep a few of those to ourselves for now

Myth #4: It’s better to defer optimization until site traffic grows

The fact is that the time to optimize is always “now.”  There’s an opportunity cost associated with not doing anything, when you could at least be doing something less-than-optimal.  Your traffic may be less than optimal, but you can still optimize, and when your traffic becomes optimal, you’ll reap the rewards of earlier optimization work.  We’ve always used the metaphor of the “leaky bucket”:  If your conversion funnel is a bucket with a bunch of holes in it, why would you spend marketing money to pour more water (a.k.a. traffic) into that bucket?  Wouldn’t it make more sense to fix the holes, then start pouring more water?

We’re definitely invested in dispelling any other myths about testing and optimization, so feel free to ask questions or contact us if you think you have a unique case and aren’t sure if optimization is right for you.

Instead, we’d like to present our brief “Best Of” List so you can get some quick highlights from our (somewhat biased) perspective:

Here we go…

• Best Presentation: “How to Turn Your Web Analytics into a Money Making Machine.” Bryan Eisenberg, Avinash Kaushik, Jim Sterne, and moderated by Mike Grehan. I thought this presentation was great because it brought together a few of the industry’s uber-minds, and the subject matter was absolutely crucial: What’s the point of investing in search if you aren’t making money? How can you figure out how to make money off of search marketing if you aren’t properly leveraging data to drive decisions, experiments, and investments?  Great job, Bryan, Avinash, and Jim! The presentations were wonderful, but it was the Q&A that gave the session a life of its own.
• Best Booth: A 2-way tie between iProspect’s “gold mine” and Facebook’s sexy white couches and shag carpeting.
• Best Giveaway: OrangeSoda gave away a sweet orange cruiser bike. We even gave the winner some unsolicited advice on how to get it back home via car!
• Best Schwag/Tchotchkes: NO ONE.  We thought the schwag was pretty weak in general.  If you saw some great schwag that we missed, add it to the comments. One notable mention was PeoplePond’s fill in the blanks t-shirt (check out Jim Sterne and Bryan Eisenberg getting their t-shirts).
• Best Booth Outfits: SuperPages.com’s gold capes cracked us up and helped them market their new “superguarantee” concept.
• Best Announcement: Sandra Cheng’s announcement of the YouTube mega-test using Google Website Optimizer: 1,024 variations in a multivariate test on YouTube’s homepage! Not sure if they’ll be blogging about it, but stay tuned.
• Best Marketing Mantra: Another 2-way tie between People must be at the center of your search strategy–not keywords. by Charlene Li and Turn opinions into hypotheses. by Avinash Kaushik.  Both great phrases to market by; we couldn’t agree more

And to those folks we met at SES, it was nice meeting you, and keep in touch!

And Tom isn’t alone in considering Michael Lewis’s sports writing to be a hidden treasure; just look at this marketing-based analysis of his book,  Money Ball.  But Tom Peters has been alone in recognizing the business applications of Michael Lewis’s astonishing article on the surprising innovation and success of Texas Tech Football, written no less than three years ago.

Click here to view the embedded video.

Action & Tempo:

“…[Coach Leach] had been harping on tempo all week: he thinks the team that wins is the team that moves fastest, and the team that moves fastest is the team that wants to. He believes that both failure and success slow players down, unless they will themselves not to slow down. ‘When they fail, they become frustrated,’ he says. ‘When they have success, they want to become the thinking-man’s football team. They start having these quilting bees, these little bridge parties at the line of scrimmage.’”

You have to learn by implementing, and it’s incredibly advantageous if you work hard to keep your testing tempo as fast and continuous as possible.  Theory and intuition are great at helping you figure out what to test, what to look for in your analytics, and how to interpret your data, but untested assumptions can kill you.  If you think that customers would respond well to X, figure out an easy-to-implement test to confirm or disprove that.  The last thing you want to do is let your website sit static for months while you plan a major change based off of faulty assumptions about the market and/or customer motivations.

Plus, even if you have a brilliant plan to improve your website, it won’t help you until you’ve actually implemented the changes.  So a fast cycle of smaller tests and changes not only keeps you safer by verifying assumptions and improving learning, but successful tests implemented early can pay off during the time you would have wasted staging a larger “batch” of changes.

In a similar manner, Texas Tech is well aware of the “opportunity costs” involved in not keeping their offensive op-tempo as high as possible:

“An idea about the use of football time was being challenged. The typical football offense seeks to eat up as much of it as it can. The Texas Tech offense, which at that point in the season had passed for more touchdowns than any team in the country, uses just a shade over two minutes on each drive. But speeding everything up has a curious effect on game time. A typical college football team runs 65 to 75 offensive plays a game. Texas Tech tries to run 90 – and sometimes does. A college team with a robust passing game might throw the football 35 times a game; at this point, 8 games into an 11-game regular season, the Red Raiders were averaging 53 passes a game.”

Preferring batch implementation of changes and tests is kind of luck holding onto the football to control the clock – you’re wasting opportunities to move the ball down the field and score.  Yet most companies, like most traditional football offensive teams, don’t have Texas Tech’s sense of urgency; they don’t understand the often substantial opportunity costs involved.  Here’s a real life example:

I presented a client with a lead generation website for a considered purchase with a Persuasive Scenario Analysis towards the end of August.   As part of that report, I also presented a prioritized list of “most likely to generate dramatic improvement” changes/tests.  And among those suggested tests, I predicted that the easiest to implement change that was also most likely to produce immediate results was to prominently display the company’s phone number within their banner.

About 1.5 weeks ago they finally made that change (along with several others) and went from getting 0 calls from their website each week to 20 calls in the first full week they had stats for the revised website.  One of those 20 calls converted into a sale.  Most sales average in at $20,000 to $30,000.

Now, I’m hesitant to put too much weight on only one week’s worth of results, but even conservatively downgrading those figures still results in a significant opportunity cost for NOT implementing that change right away.

Orientation

“Leach made his way to the sideline and from his back pocket pulled a crumpled piece of paper with the notations for dozens of plays typed on it, along with a red pen. When a play doesn’t work, he puts an X next to it. When a play works well, he draws a circle beside it – “to remind myself to run it again.” But at the start of a game, he’s unsure what’s going to work…

The Red Raiders trotted off the field at halftime with a lead, but not a large one: 14-10. A disappointing half, yet with hidden value. For 40 plays Leach’s offense had groped – digressing, probing to learn something new – and it had been useful to see how the empty spaces on the field shifted. Coach and quarterback now knew what they wanted to know about the A.&M. defense.  They had paid for the knowledge with time, but time means less to them than it does to any other offense in the land. A half to the Texas Tech offense is as good as a full game to most. The game within the game was about to begin…

In the five full years Leach has coached Texas Tech, four or five times each season the team has flopped around ineffectually for the first third or so of a game before racing off to score touchdowns at a rate unheard of in organized tackle football. It’s as if his opponent’s defense has some deep dark secret that takes time for his offense to extract.”

Action isn’t good enough if you’re just throwing stuff against the wall and not learning from it by reinforcing your successes and killing your failures.  Coach Leach doesn’t just know that this play worked and this play didn’t, he also seeks to understand why, so that he and his quarterback can adjust their overall strategy accordingly.  Once the Raiders have correctly sized up their opponents, that’s when the real scoring opportunities begin to appear.  Here’s what that looks like in the game Michael Lewis was describing:

“Leach had just a few minutes with Hodges, but he told him what he had noticed. First, the A.&M. cornerbacks were disguising their intentions. They were lining up as if to cover the fade routes – that is, before the play began, they stood between the receiver and the sidelines – but then, just as the ball was snapped, they were scampering back into the middle of the field. To Hodges it looked as if fade routes would be covered, so he had been sending his receivers on slants into the middle of the field. ‘Throw the fade,’ Leach said. ‘It doesn’t look like it’s there, but it is.’

The other glaring opportunity, to Leach’s mind, was A.&M.’s response to Tech’s formations. On the few occasions when Texas Tech lined up in a formation that suggested a running play, with two running backs, the Aggies ’put their ears back and stop the run.’ But when Tech was, as it preferred, in its passing formation, A.&M.’s fear of the pass caused them to leave huge empty spaces to run in. In the second half, the Tech running backs would be charging into pass coverage, and the Tech receivers would be running toward the sidelines.

There was one other thing Leach had noticed – and Hodges had noticed it, too. The A.&M. front line appeared tired. ‘The minute you see the defensive line bent over and their hands on their hips,’ Hodges told me, ‘that’s when you know you have them.’ The A.&M. linemen were a lot bigger than the Texas Tech linemen. They may or may not have been fatter – Leach insists they were – but their bodies were clearly designed for a different sort of football game than this frenetic one. ‘That’s the risk of playing 330-pound guys,’ Leach said later. ‘You get good push, but if you got to run around a lot, you get tired.’”

The problem with most companies is that even when they do run A/B and multivariate tests, they’re often just testing random variables or best practices, which means they have no basis for interpreting the results in terms of a larger ‘game strategy.’ If you only know that headline “A” outperformed headline “B” without understanding WHY headline “A” worked best, it would be like Coach Leach only knowing that play X worked and play Y didn’t without seeing the larger patterns or flaws in his opponents defense and without being able to exploit that during the second half.

Here’s a practical web example of this principle taken from Microsoft’s Experimentation Platform blog.  The post in question features three separate A/B tests and the second test of two different site search bars is a perfect example of how the WHY is so crucial.  But first, here are the two search bar designs:

Which one worked better?  Neither: the results were statistically negligible.  Now, if that’s all that you took away from that test, you’d have lost out.  But if you started the tests with some hypothesis about why one design might work better, you could follow up with goal scoring, revised search bar.

For instance, most people would find the search area of Option A much more inviting because it’s more spacious.  Plus, the “Popular Searches” is labeled as such in Option A whereas it’s something of a disconcerting mystery in Option B.

On the other hand, Option B does one very important thing right, that Option A doesn’t: it labels with strong verbs!  Rather than guessing that the magnifying glass means “search,” I can look at the big green button and instantly know that clicking on it will start my search.  That one is kind of a no-brainer, actually, especially since Steve Krug has rather famously taught that search buttons should either say “Search” or “Go.”

If you started with those assumptions, you might have actually created an Option C that combined the best elements of both features.  Something like this:

And then I’d be willing to bet rather heavily that you’d come up with a very clear winner. But if you simply threw Options A and B up in a simple split test and accepted the results without thinking about them, you’d never get to an improved search bar.

So how can you more consistently move past a “best practices” or a “let’s test everything” approach to Website optimization?  What kind of methodology will let you advance beyond page-level optimization to Website-wide conversion improvement?

Well, while that subject definitely builds on what we’ve just discussed, it’s also worthy of a post in itself, so make sure to subscribe to get Part II as soon as it comes out.

What: “Always Be Testing” Webinar: Landing Page Optimization: Layout and Design Elements

When: Monday, September 29, 2008 | 2:00pm EST

Where: Online, register here to receive your invitation

How much: It’s free, but space is limited so sign-up today!

About this session:

July kicked off the series with a webinar on testing trust and confidence building elements on landing pages (view archive ). Our August session covered improving landing pages focusing on testing calls to action and headlines. This month session will cover layout and design elements. Register today! Space is limited.

“Always Be Testing” Webinar Series is a monthly webinar that covers specific testing ideas, how to structure tests, and how to use Google Website Optimizer. The best part is that Google has decided to participate in it first-hand, offering the lastest tips and insights straight from Tom Leung, Google’s Business Product Manager.

“Landing Page Optimization: Layout and Design Elements” will be on September 29th, at 2:00pm EST and last about 30 minutes.

For more details and to sign-up to attend, visit futurenowinc.com/abtwebinar.htm

We hope you’ll attend and share this with anyone you know who is looking to begin to test their marketing or to increase their testing effectiveness.

About the Series:
Whether your business has just started testing, is planning to test, or has been testing for years identifying the areas and elements that have impact is often challenging. Each month, Bryan Eisenberg starts off by taking attendees on a dive deep on a specific subject area to test in your marketing and give you ideas on variation you could test. Along with these ideas each month Tom Leung will bring you useful insider tips and tricks about using Google Website Optimizer to easily test your marketing campaigns and website. Lastly, the two will team up to answer a popular question spotted on the Google Website Optimizer forums or sent in by our readers and listeners.

Register for the event today (limited space available).

Bryan will be presenting the session “Pay Per Conversation” together with Brett Crosby, Group Manager of Google Analytics and Google Website Optimizer, on the 18th at 11:15am and immediately after, Bryan will be available to sign copies of his book being provided as a courtesy by Google.

Get there early since Google will only be giving away a few hundred books. If you can’t make it there you can always sign up for our next Always Be Testing webinar for your chance to win 1 of 3 signed copies (by both of us).

Want to know more about the book? Watch the interview below or read one of the online reviews:

Click here to view the embedded video.

Click Z (Mike Grehan):
http://www.clickz.com/showPage.html?page=3630480

100 Dollar SEO (Carlos Del Rio):
http://www.100dollarseo.com/always-be-testing-a-cookbook-for-web-optimizer/landing-page-optimization

DamnIWish (Andy Sernovitz):
http://www.damniwish.com/2008/08/use-reviews-to.html

Click here to view the embedded video.

Better yet, why don’t you buy a copy for yourself and send us your own review to link to.

What: “Always Be Testing” Webinar: Landing Pages: Headlines and Calls to Action

When: Wednesday, August 27, 2008 | 12:00pm EST

Where: Online, register here to receive your invitation

How much: It’s free, but space is limited so sign-up today!

About this session:

Last month kicked off the series with a webinar on testing trust and confidence building elements on landing pages (view archive ). Our August session will continue to explore landing pages focusing on testing calls to action and headlines. register today! Space is limited.

“Always Be Testing” Webinar Series is a monthly webinar that covers specific testing ideas, how to structure tests, and how to use Google Website Optimizer. The best part is that Google has decided to participate in it first-hand, offering the lastest tips and insights straight from Tom Leung, Google’s Business Product Manager.

“Landing Pages: Headlines and Calls to Action Webinar” will be on August 27th, at 12:00pm EST and last 30 minutes.

For more details and to sign-up to attend, visit futurenowinc.com/abtwebinar.htm

We hope you’ll attend and share this with anyone you know who is looking to begin to test their marketing or to increase their testing effectiveness.

About the Series:
Whether your business has just started testing, is planning to test, or has been testing for years identifying the areas and elements that have impact is often challenging. Each month, Bryan Eisenberg starts off by taking attendees on a dive deep on a specific subject area to test in your marketing and give you ideas on variation you could test. Along with these ideas each month Tom Leung will bring you useful insider tips and tricks about using Google Website Optimizer to easily test your marketing campaigns and website. Lastly, the two will team up to answer a popular question spotted on the Google Website Optimizer forums or sent in by our readers and listeners.

No doubt, these are exciting times if you love Web tools. For the many folks who are dizzy trying to sort out conversion optimization tool choices, it might be a little frustrating.

In “The Interactive Marketer 2.0,” I made the case for improved optimization in interactive marketing and to think outside the campaign. I listed several steps to get started, including the first step: Get good at free tools, then pay for them. Tools aren’t the indicator of success, but having a process and the people in place to take action are.

The good news in this barrage of 2.0 goodies is that many believe we’ve finally reached tool parity in the Web analytics space. JupiterResearch states the following:

“Despite some small skirmishes over capabilities like video and audio measurement, the Web analytics feature race is largely over,” explained John Lovett, Senior Analyst and lead author of the report for JupiterResearch. “Leading vendors will forge ahead by making data accessible and actionable while expanding offerings into adjacent marketing technologies.”

I agree.

Several people have accused me and my firm of having a Google bias. (Full disclosure: FutureNow is an authorized Google Optimizer consultant.) This simply isn’t true. A sizable percentage of our clients use other tools like Omniture, WebTrends, and Coremetrics. Our policy has always been to work with the analytics/tool vendors of the client’s choice. For many who are just getting started or are experiencing a marketing budget squeeze, the free and robust Google offerings simply make sense. Others have found a need for features available in other tools, and we’re happy to help them use those tools better.

A tool is just that, a tool.

A tool doesn’t persuade your visitors to take action, nor is it exclusively responsible for a company’s success in optimization. So when a client approaches me requesting a tool suggestion I always answer the same. If you have a tool in place now, use it better. If you don’t, start with something free and get good at using it.

A free tool may be all you need. While certain analytics vendors offer what are considered enterprise-level tools, the free and lower priced solutions are typically labeled for use by small to medium-sized businesses (SMBs). Still, you’d be surprised at the number of large clients who are satisfied with free or cheaper tools. And, we also have several SMB clients that have more sophisticated needs.

Bottom line: don’t get hung up selecting a tool. Any business, no matter the size, that isn’t optimizing today can extract great value from any tool on the market today. The important thing is to get started optimizing and measuring more effectively. There are no more excuses, not even a lower optimizing budget.

Which brings me to my next point. How does one use a tool effectively? You must operationalize it. Your process must lead your team to take an action, e.g., make a change that you can measure. Lastly, you must be able to gain insight about customer behavior from the data. And, you must do this over and over again. Without those three things in place, no tool will usher in the success you seek.

You must always do the work of optimization. A better treadmill won’t, all by itself, trim your love handles. Likewise, a cheap camera in the hands of a skilled photographer will always take better pictures than one used by a clumsy newbie.

A tool is a tool is a tool. Pick one, learn how to use it effectively, and you’ll see optimization success. Then we can talk about what other tools you might need.

Then we can all afford drool worthy gas grills in our backyards.

*Cross-posted on ClickZ.

. .

Editor’s Note: If you’re buried in data and looking for a better process to keep your campaigns customer-focused, accountable and metrics-driven, contact us today for a confidential and free consultation.

While attending a Google Website Optimizer presentation, our good friend Sean D’Souza crafted some cartoons that do a great job of “getting the point across.”

It’s amazing when you actually take the time to do testing, how all that accumulated learning leads to some pretty impressive results.  It may not happen all at once, but, in the long run, you’re going to win.

Check out the rest of the cartoons inspired by the presentation.

P.S. – If you haven’t read Scott McCloud’s Understanding Comics, you’re missing out.

Want to see more cartoons by Sean D’Souza?

Click here to view the embedded video.

Bryan and I have co-authored a new book all about testing and helping you figure out what to test. It’s called Always Be Testing: The Complete Guide to Google Webiste Optimizer (published by Sysbex/Wiley) and we’re expecting it out next month; you can pre-order it now on Amazon.

Granted, a rose by any other name would smell as sweet, but online, your customers don’t quite have the luxury of taste, touch, or smell. So one thing we can learn from ProFlowers.com’s impressive conversion rate last month is that images matter. A lot.

But what works for one website may not work for yours. Oftentimes, the product (or service) itself dictates which strategy is most effective. For instance, if you’re selling jackets, you may want to give visitors multiple views and zoom features.

It all depends on what you’re selling and how much the customer needs to see in order to feel confident to buy.

At last month’s eMetrics Summit in San Francisco, Bryan had a chance to sit down with WilsonWeb.com’s Ralph Wilson to discuss the importance of product images and how they affect conversion. Here’s the video…

That’s why A/B split-testing is essential; it’s how you know the images are making a difference.

For more ideas on how you can test your way to a better conversion rate, we cordially invite you to join us on Wednesday, July 9th for our free “Always Be Testing” webinar, co-hosted by FutureNow and Google Website Optimizer.

. .

Editor’s Note: Don’t keep this all to yourself. Help spread the word on Facebook…

What: A new monthly webinar series brought to you by FutureNow and Google.

FutureNow friends and fans,

We are so excited about our new free webinar series we have just kicked off called “Always Be Testing”. It’s a monthly webinar that covers specific testing ideas, how to structure tests, and how to use Google Website Optimizer. The best part is that Google has decided to participate in it first-hand, offering the lastest tips and insights straight from Tom Leung, Google’s Business Product Manager.

Our first webinar will be on July 9th, at 12:00pm EST and last 30 minutes. The subject we will be covering will be confidence and trust-building elements on landing pages.

For more details and to sign-up to attend, visit futurenowinc.com/abtwebinar.htm

We hope you’ll attend and share this with anyone you know who is looking to begin to test their marketing or to increase their testing effectiveness.

Best Regards,
-Brian Bond
VP Marketing, FutureNow, Inc.

When: Wednesday, July 9, 2008 | 12pm EST

Where: Your computer

Why: Whether your business has just started testing, is planning to test, or has been testing for years identifying the areas and elements that have impact is often challenging. Each month, Bryan Eisenberg starts off by taking attendees on a dive deep on a specific subject area to test in your marketing and give you ideas on variation you could test. Along with these ideas each month Tom Leung will bring you useful insider tips and tricks about using Google Website Optimizer to easily test your marketing campaigns and website. Lastly, the two will team up to answer a popular question spotted on the Google Website Optimizer forums or sent in by our readers and listeners.

How much: It’s free, but space is limited so sign-up today!

We all take the Google homepage for granted.

It’s so remarkably simple that (unless you’re, say, a blogger for a website optimization firm) it’s unremarkable; just a text box, two call to action buttons, and a few links that most humans don’t even look at, let alone use. That’s all.

But why? Why is it that the Google homepage has barely changed in the past decade? Are they obsessed with minimalism, or is there more to the story?

From its humble origins as a research project by a couple of Stanford graduate students, Sergei Brin and Larry Page always knew that testing matters. They realized that it’s not just important to build a good online experience, but that they would need to know why it worked in order to make it better.

Take a look at Google’s 1998 homepage and see if you can guess its most innovative feature:

Got your answer? Feeling lucky (hint: that’s not it)? Great. Now hold that thought.

Speaking at the Google I/O conference last week, Marissa Mayer, vice president of search products and user experience at Google, explained how the company developed a culture of testing, insisting that, “On the Web in general, [creating sites] is much more a design than an art. You can find small differences and mathematically learn which is right.”

The CNet article (also linked to above) details some of Google’s many innovations from their years of A/B split testing. Their results illustrate one of our mantras here at FutureNow: “Believe what they do, not what they say they will do.” For instance, users claimed they wanted to see more search results per page, but testing proved otherwise. You can read the article for details on that one, but I digress. The homepage example remains the most telling. Here’s how Mayer tells it:

[Our beta testers] would sit in front of the Google screen for 15 seconds, 30 seconds, 45 seconds, a minute…Google was perplexed.

So Mayer would eventually intervene and ask what was holding up the searchers. “I’m waiting for the rest of it,” they’d say. Clearly they expected more of the flashy ads and busy text of other search pages of the 1990s.

“The very first home page was that misunderstood. People didn’t resonate with it,” Mayer said. One woman even thought the Web site was a fake construction that was part of a psychology experiment. As a result, the company put a copyright notice at the bottom of the page. “It’s not there for legal reasons,” Mayer said. “It’s there as punctuation. That’s it. (It tells the searcher) ‘Nothing else is coming; please start searching now.’”

So here it is, the big innovation they came up with back in 1998:

Amazing. Just showing a copyright assurance is what gives us the confidence to proceed.

Mayer said a lot of insightful things in her presentation, but this quote struck me:

“The urgent can drown out the important.”

So true. With website optimization, what seems to be of urgent and of vital importance from your company’s perspective may not at all be what’s urgent and vital to your visitors. In fact, your visitors may not even know what they’d fix about your website if they could.

Who among us would have told Google back in 1998 that they should try putting a copyright symbol on the homepage? [*Hears crickets chirping in background.*] Exactly. And that’s why developing hypotheses and testing from the visitor’s perspective is a must.

. .

Editor’s Note: FutureNow is an “authorized consultant” for Google Website Optimizer. (We’re writing the book on it.) To learn about which of our optimization services will best help you boost conversion rates and other key performance metrics, please contact us for a free consultation.

Formerly available only to AdWords users, Google Website Optimizer can now be accessed by anyone with a basic Google account. Even if you only have so much as a Gmail account, you can start testing your website — for free — regardless of whether you’re running a paid search campaign. (No worries, AdWords users, Website Optimizer still works seamlessly with the rest of the Google product suite.)

Companies of all sizes are getting results with Website Optimizer. And since it runs independently of your analytics program — or in sync with it, if you have Google Analytics — there’s little room for argument between departments as to whether or not your company should be testing. You should.

To find out more about Google Website Optimizer, visit their new…

• Website – Unglued from the AdWords site, you can visit Google.com/websiteoptimizer directly.
• Blog – Subscribe to websiteoptimizer.blogspot.com for product updates, case studies and ideas worth testing.

Why is FutureNow so excited about Google Website Optimizer? A few reasons:

1. Authorized Consultant – FutureNow was one of the original firms to be invited to test GWO with clients, and the case studies have been remarkable so far. Is yours next?
2. Writing the book on GWO – Bryan Eisenberg & John Quarto-vonTivadar, inventors of FutureNow’s patented Persuasion Architecture® methodology, have teamed up to write Always Be Testing: The Complete Guide to Google Website Optimizer, in book stores this August.
3. It works! – The results speak for themselves.

Of course, knowing what to test — design elements, copy, etc. — is everything. If you’d like to start testing and you’re not sure if you need to hire a firm, here are several free resources to get started with Google Website Optimizer.

Got questions about Website Optimizer or testing in general? We’d love to hear your comments.

The presentation is 63 minutes, and the sound on the YouTube version isn’t the best. If you prefer, you can watch it full-screen and with better sound via WebEx.

The webinar starts with an introduction to Website Optimizer by Google’s Tom Leung. Then I share some of what we have learned at FutureNow over the the past decade of optimizing websites, in order to show some of the most important things you should test. We then go into Q & A, but since there were additional questions that weren’t answered on the call, we will gladly answer them for you in the comments below.

As always, if you need help figuring out the specifics of what to test on your site, we can help.

We get this question a lot.

Marketers — particularly small business owners and do-it-yourself-ers — want to know if optimization is worth it. They’re short on time and they’re dealing with limited resources. They can’t wait six months to fix something that’s broken now. They don’t have the luxury.

If you’ve realized optimization can’t wait, and you don’t have the budget to hire a firm, consider these…

Tips for Optimizing Low-Traffic Websites

1.) Get a testing platform — Any testing platform will do, but if your budget is tight, we recommend using Google Website Optimizer. It’s free to use and FutureNow has developed several free resources to help you get started.

2.) Stick to A/B split testing — For a low-traffic site, you’ll want to stay away from multivariate tests and stick to simpler A/B split testing. Multivariate testing involves optimizing more than one page element at a time, often with more than one variation per element on a given page. For example, you might be testing four different headlines, three different pictures, and two variations of your body copy on a given landing page. That means you’ve just created 24 (4×3x2) different page combinations for your test. Getting enough traffic to come up with a statistically valid results could take a low-traffic site an exceedingly long time to do that. Assuming you had 50 visits per day and a brilliantly high current conversion rate of 10%, that still means it would still take more than two thousand days (about 6 years!) to get any data worth looking at. Meanwhile, A/B testing only a few combinations can give you statistically valid data within a month or two. Again, low-traffic sites should stick to A/B testing. (This white paper can help you determine whether it’s too little or too early to A/B test.)

3.) Don’t make hasty conclusions — Be patient. Wait for the tests to fully complete before jumping to conclusions. Once they do complete, take a deep breath. On any given test page, the “Chance to Beat Original” and “Chance to Beat All” percentages are crucial — and potentially misleading if you’re not up on your statistics. Basically, anything less than 90% is simply a trend that might be reversed from one week to the next. We’ve actually seen these kind of reversals happen, where a positive change (with 70% chance to beat original) flipped negative from one week to the next. Think of it this way: If you randomly flip a coin, you could get 3-4 heads in a row over 4 flips and conclude that heads was the clear “winner” over tails. Not smart. Only after many, many flips is it safe to assume you have a clear winner (or a very weird coin).

4.) Know what you’re looking for — Make sure you know how to get a hypothesis worth testing. In other words, you should know ahead of time how to interpret the results. Don’t randomly test this image or that headline. Do so because you have reason to believe the headline “should” better appeal to buyers with a given buying motivation, or because the picture “should” resolve a particular concern. That way, you have a basis for interpreting the results. That doesn’t mean the results will be absolutely conclusive (it’s possible that people really do have your hypothesized motivation but your headline was merely a bad execution of the concept), but you’ll have a way to interpret the results and do further analysis if needed. Intelligent testing essential, especially when you don’t have much traffic.

5.) Test one click at a time* — Shorten the distance between the Experiment Page (where you’re running the test) and the Goal Page (where you count conversions). This will yield conclusive results in less time. A quick e-commerce example: Use the shopping cart as a Goal Page for a test being run on a Product Page (as opposed to using the Order Confirmation Page as the Goal Page).

6.) Ensure success with Pay-Per-Click* — Purchasing traffic to validate changes to your site is like buying insurance on the effectiveness of your web design. If your PPC ads are well targeted and attract more (and more qualified) visitors, your test results will be more accurate. With enough visitors, testing is like letting visitors design your site for you.

7.) Prioritize your optimization efforts — Optimizing for usability and conversion is usually easier than optimizing for persuasion. Before a site can persuade, its basic elements must work. Go for the low-hanging fruit, then work your way up the Hierarchy of Optimization.

Got questions on how to optimize your site? Feel free to contact us or leave a comment below.

. .

*Indicates a tip that has been added to the list.

[Editor's Note: Today is your last chance to register for the Persuasive Online Copywriting seminar, happening tomorrow, Friday the 28th, in San Francisco.We're keeping class size small and there are only two or three tickets left, so hop to it!]

In the discussion thread, a respondent worried that he may not be able to use GWO because his company’s website has a database-driven content management system. He described himself as a “perfectionist” and it didn’t settle well that content was somehow taken “out” of his site and hosted on Google. Further, one of his company’s consultants commented to him that GWO just “isn’t useful” for a complex database-driven site.

First off, we can tell you from experience* that his consultant is mistaken. (See explanation here.)

Secondly, everyone thinks their own site is complex. Everyone. (Just like everyone thinks their kid is cute enough to be a model for Gap Kids.) But ecommerce sites are pretty similar — and simple. It goes something like this:

• Get customer to site
• Display product to customer
• Help customer decide to buy
• Accept her money with a thank you
• Ship out the goods
• Repeat

Customers don’t care if what we have behind-the-scenes is simple or complex. All the customer cares about is how simple and enjoyable — or not — the experience is for them.

Now, back to the issue of perfectionism. This fear of taking an incremental step lest it turn out wrong, even if the step is toward improvement, seems to evoke fear, dread and a certain “deer in the headlights” mentality.

Ever hear the adage, “Anything worth doing is worth doing wrong”? It’s a great way to think about testing and improvement of any kind, because it deals with the fact that the first step toward improvement always “feels” the hardest. It speaks to the moment when you’re most susceptible to false objections like “It’s too complex!” or “That’s inefficient!”

Let’s get those first steps out of the way. Let’s embrace being wrong, because we will almost surely learn some way to improve. The fact that the improvement won’t be immediate or perfect just isn’t a viable reason not to try. Asking for it to be perfect first and always is a perfect recipe for “never”.

If your company does, say, $5m/yr online and you can raise the conversion rate from, say, 4% to 5% (a 20% lift) because of your testing with GWO — or any testing tool for that matter — you just added $1 million ($5m x 20%) to the bottom line. If I were a CEO and found that so-called perfection was costing me $1m/yr in lost revenues, plus employee salary, I’m pretty sure I could find less expensive, less perfect employees.

I wonder, just how many companies out there are paying millions of dollars a year for perfectionism? And how many imperfect employees, freed from this apotheosis, consistently deliver better results for their companies and their customers?

Could this be why three quarters of online retailers don’t test even though it’s free?

. .

*FutureNow is an Authorized Consultant for Google Website Optimizer.

You’re likely familiar with Maslow’s Hierarchy of Needs, which states that human beings must first prioritize basic needs, such as food and shelter, before they’re able to seek higher needs, like social interaction and self-actualization needs. What good is owning a Harley-Davidson or finding the perfect outfit for a trip to a club if you’re starving to death?

Looking at your site in a similar fashion is extremely helpful. Since I first introduced our concept of the hierarchy of optimization last year, I’ve wanted to dig into it a little deeper:

Taking a step back and examining the entire pyramid will help you better assess where to start or assist you in knowing exactly what you’re optimizing now. The hierarchy also gives insight into optimization’s potential impact.

Let’s start at the bottom. Remember, the higher you go on the pyramid, the bigger the impact you’ll make on optimization. Also remember that the pyramid doesn’t indicate the level of effort needed to optimize, because this is as different from site to site as we are different from each other.

The Hierarchy of Optimization

Function is almost below the basics. Does your site have long periods of downtime? Do you deliver hundreds or thousands of 404s? Does your shopping cart constantly freeze up on visitors? Can users log in? Do images load? Is your site heavy on customer-facing errors? As a first order of business, work to make your site as reliable as the sunrise.

Another aspect of function is making sure that back-end functions are also in place. We’ve worked with companies that were spending ample on marketing and great site widgets, but the back-end shipping process was broken, causing an embarrassing amount of orders to go unfulfilled. This isn’t sexy marketing; it’s Business 101. Why go through all the hard work to market and sell a $1,000 dress only to have the customer walk up to a dirty checkout lane with a broken cash register circa 1950?

Having solid, clean user data for analytics also falls in the function level, otherwise anything higher up on the pyramid can’t be optimized with any accuracy or confidence.

How accessible is your site? Remember the recent lawsuit brought against Target.com for not having alt tags on its images? Font size, language issues, and pages and sections that don’t load correctly are other accessibility issues. Browser-specific issues fall in this level as well. Check your access logs to determine if you’re under-serving or ignoring a visitor segment. Optimize for people with disabilities, allow fonts to be resizable for users who need larger print, and solve browser-specific issues. If you remember, 38 percent of the retailers had difficult-to-read fonts in our 2007 Customer Experience Study. Optimize for dial-up users (there are still plenty of them out there). Access for mobile devices should also be considered.

Are your buttons easy to find and see? Is the search dialog where users expect it? Do you use drop-downs when you could use a radio button? Usability is about moving site elements around and using size, color, and contrast to improve the ease of use of your site. Thousands of great articles have been written about usability. Jared Spool’s are my favorites.

Call-to-action button optimization is a popular optimization item for marketers. For most, the effort is low, and it can have significant impact. Still, it’s only one aspect of the usability equation.

While similar to and often confused with usability, the intuitive layer is about improving the flow of the visitor’s site experience and optimizing aspects that keep the visitors from buying. Point-of-action assurances, product detail pop-ups, customer reviews, upfront shipping costs, and current in-stock messaging all reduce friction in the buying process, anticipate customer questions, and offer answers at the point the customer asks.

On a lead generation site, optimize form questions, try to shorten the time needed to fill out the form, and introduce ways for the visitor to take more control of when and how they’re contacted.

At the top of the pyramid are site elements that move a customer toward making a decision to buy your specific product. Persuasion issues are almost always high impact.

Improving persuasion on your site is mostly done by improving copy or product images. Product descriptions, feature tours, demos, and product comparisons (even with competitors) are considered persuasive issues. On a lead gen or B2B (define) site, it’s your service description, case studies, testimonials, and white papers. Make sure your copy addresses each of your personas.

Brand image and a site’s overall look and feel are often persuasion issues, especially if there’s a disconnect between the brand promise and site design. But have no doubt that a strong familiar branded product will forgive a multitude of site errors, as many of us have endured horrible sites and process to buy products and services we really wanted.

Assuming the bottom three levels are sound on your site, persuasion scenario planning will assist in planning and measuring the intuitive and persuasion challenges you face.

Conclusion

Start at the pyramid’s bottom and list each of the optimization tests or changes you need to consider. For each item, rank the effort it will take your team to make the change or test possible. Start with low-effort items, even if they’re low on the pyramid. Then work your way up.

Best of luck with your optimization efforts this year. If you need help planning and prioritizing your tests, we’d be happy to oblige.

This originally appeared in my ClickZ column from 2/29/08.

“Lead generation went up 5,000%!” (Tumbleweed rolls.)

“They doubled their conversion rate!” (Crickets chirp in disbelief.)*

Don’t think you can do it? Bryan Eisenberg and Google’s Tom Leung disagree. Join them for a free webinar on March 11th.

*Please Note: Case studies not boring when about your site.

Who

What: Want to improve your website’s performance? Interested in learning how to get the most out of Google’s free A/B and multivariate testing tool, Website Optimizer? This one’s for you.

In previous webinars (”Introduction to Website Optimizer” and “Creating and Launching Experiments“), the Google Analytics and Google AdWords teams have shown how to use the tool. But this time around, they’ll be delving into a more fundamental question: “What should I test?”

In recent articles, we’ve shown how testing can transform your site over time. Yet it doesn’t do much good just to know you should test. Without a clear hypothesis, it’s easy to lose track of your own guesswork. Knowing how to test a hypothesis is critical. So, by popular demand after many Website Optimizer users asked for more insight on what exactly they should test, Google has invited us — and all of you — to join them for this free tutorial.

During this online seminar, Tom and Bryan will:

• provide a brief introduction to Website Optimizer for newer users
• talk about testing best practices
• discuss some of the top elements to test on any webpage, and
• review top mistakes people make when developing new content to test.

Where: Your computer

When: Tuesday, March 11th | 9 – 10 am PST | 12 pm EST | 4 pm GMT

Why: Because you’ve reviewed Future Now’s 7 free resources to get you started with Website Optimizer, and, sure enough, you’re ready to get started — almost.

Is this something you can do on your own, or should you hire a website optimization firm? That all depends on the time and resources you’re willing to put into it. But before you dive in with both feet, it’s smart to know what’s worth testing.

Register for Free! Upon registration, Google invites you to submit specific topics you’d like Tom and Bryan to cover.

That may or may not be true. At the moment, this statement is merely a guess, an assumption — but it’s testable. It’s a hypothesis.

People love to insist that your website is made of magical ones and zeros. “It’s HTML,” they’ll say. “It lives on triple-redundant co-located servers,” they’ll argue. Yet the truth is much simpler, and scarier, than that.

Your website is a tower of assumptions. Everyone’s is. Perhaps yours was built according to a specific blueprint. Maybe it was built from a template. Either way, if it’s not properly maintained, the structure will collapse. But before you demolish the current structure and start over from the ground up, you should test the existing site.

If you want to improve your website, testing provides the scaffolding to fix it. And just as you wouldn’t hire a renovation crew that uses scaffolding made of toothpicks, your optimization tests require strong hypotheses. Of course, you can always test a bunch of random variables and see which configuration works best with your visitors, but that generally takes too long, adds noise to the data, and makes it difficult to gain any real insight.

The better thing to do is to start with a hypothesis.

Dropping Science

In my last post, I showed how testing allows you to optimize by letting visitors design your site for you. By giving them new versions of navigation and content elements and closely monitoring to see which ones work best, your visitors can vote with their clicks, and you can more easily adjust your site to fit their needs.

Be careful, though. If you don’t have a solid hypothesis, improvements can take longer — and be more incremental — than they should be. Recycling random variations of a page just to see what works often yields a much smaller return on investment than hiring a website optimization firm.

It’s the most common problem we see among companies that don’t outsource their testing: They don’t really know what to test.

Regardless of who tests your website, the scientific method [define] must drive the process. Your venture into testing must begin with curiosity. Curiosity is fundamental to humanity, and the basis for our achievements. To have success online, you must be curious as to why things happen and what is influencing them.

• Observation: “Why do so few people add an item to their cart from the product page?”

• Observation: “Why do my blog posts with short titles seem to get more comments?”

Curiosity is the initial spark to start a learning experience, but ideas and explanations must be conjured to satisfy that curiosity. This is where the hypothesis comes from.

Don’t Believe the Hypothesis

Again, a hypothesis is just an assumption. The ideas and explanations you base this assumption on can come from real world examples or basic intuition. To write a hypothesis, simply take the action you’re considering and state the result — a benefit, we hope — that you expect it to have.

• Hypothesis: “Making the ‘add to cart’ button larger will increase our conversion rate.”

• Hypothesis: “Using blog post titles with six words or less will increase the amount of comments.”

The one and only purpose of running a website optimization test is to prove (or disprove) your hypothesis by exposing it to real world conditions. As such, you’ll need to create variations of the elements you wish to test in a way that properly reflects your hypothesis, so you can test them against the original version to see which one works best.

Let’s start with “Making the add to cart buttons larger will increase our conversion rate.” To test this hypothesis, you’ll need to create a version of the page with a larger add to cart button. To be sure, you may also want to test more than one size. If a large button isn’t ideal, maybe a medium-sized one is.

Wash, Rinse, Repeat… TEST

Lets say the test proves our hypothesis to be valid and you decide to make the “add to cart” button larger. Wonderful, but you might want to hold off on the champagne.

Now it’s time to create another hypothesis about the best color for the “add to cart” button. For instance, “A green ‘add to cart’ button will yield a higher conversion rate than similar red or blue buttons.”

The point is to learn something — anything — about what is and isn’t working on your site. Approach testing in a systematic way and record what you learn to guide you through future tests. You may also want to revisit certain tests to see if they still hold true, especially if you’ve changed other elements on the page.

It’s very important not to get discouraged. Even if your hypothesis is disproved, you’ve learned something valuable; that what you have is working well enough for you to focus on another area of your site that needs attention.

On the other hand, if your hypothesis is strong — and the test results prove it — you’ve begun remodeling your “tower built on assumptions” into a high-rise casino, where the odds are stacked neatly in your favor.

. . .

[Editor's Note: Blinded by science? Need a renovation? Future Now can help you test it.]

Over 800 people showed up to hear the speakers and meet with consultants (like me) to evaluate their current websites — and some even discussed mock-ups and prototypes of new versions of their sites. My back-of-the-napkin calculation is that Internet Retailer gave away around $150,000 worth of consultations, but I’ll ask you the one common question I asked several of the companies I met with just for the price of spending a few minutes reading.

What makes you think the new design you’re working on is going to work any better than the one you have today?

I recall sitting with one of the most recognized brands on the Web and him pulling out his mock-ups. They felt that they had issues around how products were presented and how their navigation worked. They offered a complex menu with way too many options in their current navigation, and were hoping to improve it by using a top level menu with a javascript rollover.

“What research do you have that indicates that new navigation will work better than the current one?” I asked. He was honest and said, “None.”

So why don’t you test it?

Internet Retailer did a pre-conference survey and asked the top 500 retailers if they’re doing multivariate or A/B testing, and included the results in their Website Design & Usability Guide.

Amazingly, 76.7% of retailers surveyed don’t test!

Huh? Now that you can get A/B and Multivariate testing tools for free from Google, why aren’t you testing?

We’ve also found that it’s safer to roll out a redesign in stages, in order to avoid the initial drop in conversion that often results from a redesign. Why not roll this out as a series of tests?

If you need help, call us at 877-643-7244 (ext. 3316). We’d be happy to help you make more money before and after your redesign.

We chatted back and forth about the conversion issues involved with placing it there — and in fact, one of our Conversion Analysts, Peter, commented on this very topic in his latest post — but soon our conversation turned to something much more interesting than left-sided calls to action: the testing of left-sided calls to action.

“Do you think they tested it?” Bryan asked.

“Hmm, the Joker in me wants to say Yes, but I’m guessing the money bet is No,” I replied.

Now, that’s not because Crutchfield doesn’t test. In fact, I’ve no idea at all what sort of testing culture Crutchfield nurtures; I’m just saying that in our experience, only rarely does this sort of innovation ever come about from testing. Instead, it’s sadly de rigeur for it to arise from a designer wanting to try something “different”, or an IT staff that doesn’t perceive one shopping cart as different from another, or maybe Matilda the Intern just forget an HTML tag. Anyway, the point is to go with the simplest explanation — which, in 2008, is that most companies still don’t test.

“I think you’re right,” Bryan continued, “cuz if they did test it, it probably wouldn’t do well.”

“Maybe some Clown in IT or Marketing just wanted to be ‘kewl’.”

Here’s what we’re talking about, as shown on Crutchfield.com:

Intuitively, I hope you’ll agree with us that right-sided feels like a better than even-money bet (though that in itself is a reason to do a test) — but what’s the point of leveraging your intuition to be “directionally correct” unless you eventually try to back it up with some evidence that you’re actually correct?

That started me down the road thinking about how to actually test this hypothesis.

(I can be wordy, so if you’ve lost the trail of thought, the question is, “Which converts better? Right- orLeft-sided Add-To-Carts?” and the hypothesis would be, “Right-sided Add-To-Carts convert better than Left-sided Add-To-Carts.”)

Here’s where it gets interesting: The supposition is that most Web surfers are so used to right-sided Add-To-Carts (and right-sided Calls-to-action, generally) that a left-sided one is bound to produce some cognitive dissonance. It might not be consciously noticed — less so on “narrower” sites and more so on wider ones — but the placement on the left will “feel” odd.

With that in mind, just how do you go about running a test you already know has a skew to it? How would you really determine whether the Clowns or the Jokers win The Great Add-To-Cart Positioning Debate of Aught-Eight?

Here’s what I would do: First off, start with the most obvious test, because we have to get a quick benchmark of just how far Clown is from Joker. Throw some percentage of traffic at the left-sided Add-To-Cart — enough for some statistical significance — and see just how well Right does vis-á-vis Left. (The fascinating thing about intuition is that a fair percentage of the time it’s fabulously, gloriously, achingly, wrong — and if this is one of those times, better to find out early and move on to the next good idea.)

Assuming we’ve shown some evidence of the skew in favor of right-sided shopping carts — otherwise, why continue reading this post? — how do we go about removing the skew that comes about from people being “trained” that right-sided is “normal” to answer the real question: If folks weren’t biased by convention, which side converts better?

To do that, what you’d really want is to look among your customers who’ve already successfully converted using one particular side and to present them with similarly-sided add-to-carts in the future (hmm, might have to set a cookie!), so you can gauge what the conversion rate is for people who’ve shown at least some indication that they can successfully convert.** The idea here is that, all else being equal — something the pre-existing bias hurts — the true question should be, “Do people actually have a preference for sidedness at all”?

By picking only from those who’ve successfully converted previously, you’re making a first attempt to say, “Hey, at least these folks don’t seem to be impeded by a systemic bias”; therefore, those who buy consistently using left-sided calls to action might then be expected to convert at approximately the same rate as those who buy consistently using right-sided calls to action.

“And surely,” you might argue, “those who show a preference for left-sided add-to-carts should convert better when consistently presented with left-sided add-to-carts than Right-Siders who are suddenly presented with a left-sided add-to-cart.”

See, you’ve switched the tables.

Get it? In short, you try to come up with series of tests — a Testing Campaign, if you will — which attempt to disprove the way your original hypothesis was leaning (we figured Right would do better, so let’s design tests that indicate when Right does poorer), and let us challenge any underlying bias (i.e., that Add-To-Cartss typically appear on the Right) that gives unfair advantage.

Well, those are my thoughts on the subject. What I hope you got out of that is that a “culture of testing” means thinking as deeply about the design of experiments as it does their performance.

I’d love to hear more about you. Are you a “Clown” or a “Joker”? Or are you just “Stuck in the Middle”? Would your brand loyalty or the customer’s familiarity with your site’s User Interface simply override any preference you have for being a Clown or a Joker?

- – - – - – -

**A few readers will feel reassured to know that, in actuality, you’d still send at least a few visitors who preferred one Side to see an opposite-Side call-to-action once in a while just to keep things honest; enough to get insight from the data, but not enough to cost the company too much from the loss from the expected conversion differential. I figured I’d say that as a footnote before some Sharp Tack out there writes in to scold me.

[Author's Note: What's with all the Clown and Joker references, you ask? From the song "Stuck In The Middle With You" by Stealers Wheel (c.1973), comes the lyric "Clowns to the Left of me/Jokers to the Right/Here I am/Stuck in the Middle with You." I was bound and determined to get that song into a post sometime this month, just to stop humming it in my head. There. Now it's your problem. ]

[Editor's Note: Want more profitable ideas on how to beat assumptions with better testing? Take a look at our free website testing resources, including John's A/B testing white paper.]

Here’s the “Cliffs Notes” version, in case you couldn’t make it:

A/B Test — Allows you to test different versions of the same page. Let’s say you want to test the homepage. To indicate which version of the homepage is performing better, a Goal Page is selected, e.g., an order confirmation “thank you” page. So, once a visitor comes to the test page — in this case, the homepage — they’re presented with one version (’A') that you created to test against another version (’B') of your homepage. Google Website Optimizer will then record the visit as a success or failure by crediting either version ‘A’ or version ‘B’ — whichever one ultimately led them to the Goal Page. Once sufficient data is collected, a winning page variation will be selected.

Multivariate Test — Very similar to the A/B test in that it allows you test different variations of the same page. The main difference is that in an A/B test, you’re creating and uploading different files and they’re presented as different URLs (”www.homepage.com/A; www.homepage.com/B”), so Google Website Optimizer splits the traffic to different version and tracks the performance of each. Mulitvariate tests allow you to select different sections on your page and create different versions for each section or content element. So, if you want to test different headlines while testing different “add to cart” buttons, a multivariate test can save time when coding. Google Website Optimizer will create different versions of the page, using every possible combination and variation for each page and content element you’re testing.

If we’re testing three headlines and two “add to cart” buttons, GWO will create six unique pages by combing every possible variation. The winning variation is then selected.

Split-Path Test — This test will split your traffic among different linear paths containing multiple pages for each path. This is different in that you’re testing the performance of grouped pages against other grouped pages. For example, you could test a checkout process by splitting it into two variations; one with four steps (or pages), and another with only three steps. Each variation of grouped pages will have the same Goal Page (e.g., order confirmation page). Once the data is collected, the winning checkout process will be the one that converted a higher percentage of visitors.

Multi-Path Multivariate Test — This will test different sections on multiple pages at the same time, all within one experiment. This test is best used after a winning combination from a split-path test is selected. For example, you could test images, testimonials, and contact information on the pages throughout a checkout to find out which combination of the elements across the group of pages is working the best.

“Do Anything” Test — This is a useful testing functionality that allows a test to have multiple Goal Pages, as opposed to only having one Goal Page. For example, let’s say you’re testing the headline and name of a category page. Now, to determine whether the headline is a success, simply test how many click-throughs you’re getting from the category page to any of product pages that it links out to.

Clock — This will register an experiment as a success after the visitor has spent a specific amount of time on the page. For example, let’s say you wanted to test variations on the title of a blog post. You can consider a visit from someone who stays on the page, reading the post for at least 30 seconds, a success. So, if you’re still reading this, hopefully that means we’re successful Google Website Optimizer will collect data on which headline is engaging the most readers to stay on the page for at least 30 seconds and show us a winning title.

Click — This functionality allows you to set the goal for an experiment as not just a page on a site, but a specific event, or click, on a page. An example would be to test which copy is getting the most people to click on your link to an external checkout source, like Google Checkout or PayPal. Since a click on any link brings the visitor to an outside page, you cannot send them to a Goal Page, but you can measure success by defining a specific click.

If you missed the webinar, don’t worry. There’s a wealth of information on the GWO Help, Forum, Case Study, and Demo pages. Google also said they will soon be posting abbreviated versions of the webinars. Besides, we have more free Google Website Optimizer resources to get you going. And if you decide you’d like expert guidance with your tests, we can help.

People often ask us, “What’s the average Conversion Rate in my industry?” Fair enough, but does it matter? If you’re converting less than 1/10th of your traffic, you should be concerned. If you’re converting more than that, you can do better. We all can. It’s even possible to double your conversion rate, if you ask bigger questions.

When JigsawHealth.com contacted us, they were converting three times better than the average. But they the didn’t just ask bigger questions, they put them to the test. Now, after testing some new verbiage and some design tweaks, they convert 1/5th of browsers into buyers.

Inc. Magazine explains…

If you’re ready to put big questions to the test, or if your business already demands big answers — and quickly — don’t assume the prognosis can’t improve just because you’ve “already tried everything” else.

Want to boost online conversions on landing pages and beyond? FutureNow can help.