Questions? (877) 643-7244
FutureNow Article
Friday, Jan. 25, 2008

Testing Add-to-Cart Buttons: Stuck in the Middle With You

By John Quarto-vonTivadar
January 25th, 2008

super sounds of the 70'sBryan walked into my office the other day to point out an interesting item found while surfing: a left-sided add-to-cart button on a product detail page.

We chatted back and forth about the conversion issues involved with placing it there — and in fact, one of our Conversion Analysts, Peter, commented on this very topic in his latest post — but soon our conversation turned to something much more interesting than left-sided calls to action: the testing of left-sided calls to action.

Do you think they tested it?” Bryan asked.

Hmm, the Joker in me wants to say Yes, but I’m guessing the money bet is No,” I replied.

Now, that’s not because Crutchfield doesn’t test. In fact, I’ve no idea at all what sort of testing culture Crutchfield nurtures; I’m just saying that in our experience, only rarely does this sort of innovation ever come about from testing. Instead, it’s sadly de rigeur for it to arise from a designer wanting to try something “different”, or an IT staff that doesn’t perceive one shopping cart as different from another, or maybe Matilda the Intern just forget an HTML tag. Anyway, the point is to go with the simplest explanation — which, in 2008, is that most companies still don’t test.

I think you’re right,” Bryan continued, “cuz if they did test it, it probably wouldn’t do well.”

Maybe some Clown in IT or Marketing just wanted to be ‘kewl’.”

Here’s what we’re talking about, as shown on Crutchfield.com:

crutchfield sells the ipod touch to leftys

Intuitively, I hope you’ll agree with us that right-sided feels like a better than even-money bet (though that in itself is a reason to do a test) — but what’s the point of leveraging your intuition to be “directionally correct” unless you eventually try to back it up with some evidence that you’re actually correct?

That started me down the road thinking about how to actually test this hypothesis.

(I can be wordy, so if you’ve lost the trail of thought, the question is, “Which converts better? Right- orLeft-sided Add-To-Carts?” and the hypothesis would be, “Right-sided Add-To-Carts convert better than Left-sided Add-To-Carts.”)

Here’s where it gets interesting: The supposition is that most Web surfers are so used to right-sided Add-To-Carts (and right-sided Calls-to-action, generally) that a left-sided one is bound to produce some cognitive dissonance. It might not be consciously noticed — less so on “narrower” sites and more so on wider ones — but the placement on the left will “feel” odd.

clowns and jokers uniteWith that in mind, just how do you go about running a test you already know has a skew to it? How would you really determine whether the Clowns or the Jokers win The Great Add-To-Cart Positioning Debate of Aught-Eight?

Here’s what I would do: First off, start with the most obvious test, because we have to get a quick benchmark of just how far Clown is from Joker. Throw some percentage of traffic at the left-sided Add-To-Cart — enough for some statistical significance — and see just how well Right does vis-á-vis Left. (The fascinating thing about intuition is that a fair percentage of the time it’s fabulously, gloriously, achingly, wrong — and if this is one of those times, better to find out early and move on to the next good idea.)

Assuming we’ve shown some evidence of the skew in favor of right-sided shopping carts — otherwise, why continue reading this post? — how do we go about removing the skew that comes about from people being “trained” that right-sided is “normal” to answer the real question: If folks weren’t biased by convention, which side converts better?

To do that, what you’d really want is to look among your customers who’ve already successfully converted using one particular side and to present them with similarly-sided add-to-carts in the future (hmm, might have to set a cookie!), so you can gauge what the conversion rate is for people who’ve shown at least some indication that they can successfully convert.** The idea here is that, all else being equal — something the pre-existing bias hurts — the true question should be, “Do people actually have a preference for sidedness at all”?

By picking only from those who’ve successfully converted previously, you’re making a first attempt to say, “Hey, at least these folks don’t seem to be impeded by a systemic bias”; therefore, those who buy consistently using left-sided calls to action might then be expected to convert at approximately the same rate as those who buy consistently using right-sided calls to action.

“And surely,” you might argue, “those who show a preference for left-sided add-to-carts should convert better when consistently presented with left-sided add-to-carts than Right-Siders who are suddenly presented with a left-sided add-to-cart.”

See, you’ve switched the tables.

Get it? In short, you try to come up with series of tests — a Testing Campaign, if you will — which attempt to disprove the way your original hypothesis was leaning (we figured Right would do better, so let’s design tests that indicate when Right does poorer), and let us challenge any underlying bias (i.e., that Add-To-Cartss typically appear on the Right) that gives unfair advantage.

Well, those are my thoughts on the subject. What I hope you got out of that is that a “culture of testing” means thinking as deeply about the design of experiments as it does their performance.

I’d love to hear more about you. Are you a “Clown” or a “Joker”? Or are you just “Stuck in the Middle”? Would your brand loyalty or the customer’s familiarity with your site’s User Interface simply override any preference you have for being a Clown or a Joker?

- – - – - – -

**A few readers will feel reassured to know that, in actuality, you’d still send at least a few visitors who preferred one Side to see an opposite-Side call-to-action once in a while just to keep things honest; enough to get insight from the data, but not enough to cost the company too much from the loss from the expected conversion differential. I figured I’d say that as a footnote before some Sharp Tack out there writes in to scold me. ;)

[Author's Note: What's with all the Clown and Joker references, you ask? From the song "Stuck In The Middle With You" by Stealers Wheel (c.1973), comes the lyric "Clowns to the Left of me/Jokers to the Right/Here I am/Stuck in the Middle with You." I was bound and determined to get that song into a post sometime this month, just to stop humming it in my head. There. Now it's your problem. :) ]

[Editor's Note: Want more profitable ideas on how to beat assumptions with better testing? Take a look at our free website testing resources, including John's A/B testing white paper.]

Add Your Comments

Comments (29)

  1. I am going to make a guess here, the call to action button has been on the right side for so long, and I bet, few people will change it to the left to test it, that it will remain on the right side forever…..

    I think we place things in certain locations because the customer expects it to be there. The call to action button is one of those elements.

    Interesting idea, testing may tell thou……

  2. John,

    newspapers, magazines etc have for decades charged a premium for ads (calls to action) that appear on the top right hand side and right facing pages. (Place one on the left side of a left page and you get a huge discount.)

    They do this ‘cos they know that that is where people look.

    Not sure when the research was done, probably middle of last century, or maybe they just figured it out over centuries of publishing. Be interesting to research that one and find out.

    Point is, this whole right positioning thing has been around for centuries, a lot longer than web design.

  3. Mark makes a great point.

    I was thinking you may need to test this with people who have never used the web before – and have not been conditioned to add to cart on the right. Now, that would make for an *interesting* and probably frustrating test!!

  4. You know how easy it would be for the e-tailer I work for to test add-to-cart button placement?

    As easy as it would be for me to walk over to 34th St and topple the Empire State Building with my bare hands.

    Dream on, guys.

  5. Mark, Linda, Audio: thanks so much for your comments. Absolutely it is conventional for that AddToCart to be on the right. I suspect it goes hand in hand with reading from left to right (easy to find out how in right to left languages (Hebrew, Arabic) where do those language-specific sites tend to put AddToCart? And what about Japanese when read top to bottom? You’d think in all cases you’d put the AddToCart at the place in the prose where the persuasive thought comes to mind for the customer to say “I want”.

  6. Hi Ricardo,

    It’s a sad state we’re in, isn’t it? It’s not that the left-side versus right-side AddToCart is not determinable — we could actually design experiments to measure it and a host of other important varaibles….but there’s no collective corporate Will to do so.

  7. Ricardo,
    It’s very easy for a 3rd party vendor to help you run such product page tests. My company runs countless such tests…You just need to be willing to look outside your current IT team!

    Hope not so see you trying to push over buildings with your bare hands but please send pictures if/when you do.

    Thanks.

  8. I think the point Ricardo may be making was that to perform a test you need a reliable high volume of conversions, which excludes most etailers, plus the technical issues of product-based websites which may be doing all sorts of wizardry in the background.

    Once you get into the “You previously bought that, so you may like this” sort of stuff, plus tracking such data, auto-generated offers, seasonal stuff and all that jazz – moving the button from one side of the screen to the other is not always as simple as it sounds!

    Sometimes there’s a neat and elegant way of doing it, and sometimes it would be easiest to run the entire system twice rather than mess around trying to be smart. Most off the shelf CMS shopping systems were designed before such testing became popular so it’s not a no-brainer by any means.

    It strikes me as one of those things some bright young spark comes up with, the IT claim they can do it rather than look silly and then it all goes horribly wrong when some customers get billed twice, some lose their orders and by the way, here’s an email saying our logo is half off the screen and it looks crap in Firefox.

    Of course that was the bad old day, WE are so damn smart we never have ANY bugs or screwups and everything will work perfectly what with CSS and being net 2.0 an’ all. It’s just that old-school management have heard all that before and know how quickly “There’s a problem…” features in conversations.

  9. hmm, I *hope* that’s not the point Ricardo is making, because you don’t in fact need a huge volume of conversions to do this sort of test. You’ll do your first test (as originally described) simply to distinguish whether left or right does better generically and at that point it doesn’t take a large sample to distinguish a non-random distribution of results (i.e., something other than 50/50) due to something other than random chance. You don’t need high conversion rates per se, you simply need sufficient traffic to each element in the test for the results to be statistically significant, regardless of what they do once they see the AddToCart.

    Then, following part 2 of the test, you’d only be looking at on-going conversion of those who did convert earlier, i.e., you’re repeat customers. Although that, presumably, is where the left-side AddToCart customers will need to build up some traffic for significance, you’re also drawing that traffic from a sub-group of your customers who’ve by definition have already shown a propensity to buy your products (repeat customer typically have a higher average conversion rate than first-time customers — atleast for businesses who have legitimately good products that satisfy customer needs). So you’re starting from a higher conversion rate population anyway, which will help tremendously with the return traffic you’ll need for significance

    In other words, if your site’s conversion rate is so low that you can’t get enough data to make a start at this sort of experiment then this isn’t the sort of experiment you should be considering anyway — you should be doing much simpler optimizations that increase overall conversion (the so-called “low hanging fruit”) before tackling what may in fact be a persuasional challenge for your company’s business topology.

  10. [...] the add-to-cart button on the left column. Rather than showing a screen here, you can see one on Grokdotcom’s blog post or at [...]

  11. [...] what Future Now has to say about this unorthodox e-commerce [...]

  12. [...] Brian Eisenberg’s company wrote an article about Crutchfield’s new left hand side call to action.  They’re not big fans … and they’re guessing it came to pass because of a lack of testing. addthis_url = ‘http%3A%2F%2Fthecartblog.com%2F2008%2F01%2F29%2Fchanging-your-carts-look-be-sure-to-test%2F’; addthis_title = ‘Changing+your+cart%27s+look%3F++Be+sure+to+test’; addthis_pub = ”; [...]

  13. Hmm, Cart Blog, please do not put words in my mouth. I did not say I wasn’t a fan of Crutchfield, nor that Crutchfield doesn’t test — in fact, I bent over backward to mention that we had no special insight into Crutchfield’s approach to testing one way or the other.

    What I did assert outright was that testing is infrequently done at companies, even in 2008. And so, in the absence of any evidence to the contrary, unorthodox UI use should best be assumed as non-tested.

    Nevertheless, the point of the article — perhaps missed in your rush to link to us — is not what Crutchfield is doing in the specific, but how companies in general can begin to think about testing techniques for unorthodox elements when they find themselves in a testing culture where such experiments can occur.

  14. I do find it interesting that the left nav is replaced by the buy option on the product page – a position where people are used to looking for further navigation options … Has anyone ever tested what happens at that point in the buying cycle when someone is presented with something new to choose from?

  15. [...] you want to read some great tips on testing your site for conversions, including things like the placement of the add to cart button, head over to Grokdotcom. Some of it will be very difficult for beginning webmasters, but [...]

  16. Web usability experts (Jakob Nielsen) have done studies that show web viewing patterns following an “F” pattern where people tend to look across the top of a site, and down the left hand side. Based on that logic, the Crutchfield layout is actually right in line with such eye-pattern studies. It would be nice to test that theory – but as many have already pointed out, that seems very difficult for the average site.

  17. Michelle,

    but there’s also plenty of other research which proves completely the opposite. What to do?

    As the crew at Future Now keep pointing out, conversion rates for most sites are till stuck at 2%.

    We need to be careful about accepting anything as definitive when it comes to web design. With conversion rates that low, everything is up for reinvention.

    The F pattern merely shows that that’s how most web designers layout their pages, which forces people to look in that pattern.

    There’s another great post on this site. (the bunnyfoot sunsilk ad test) that shows that just by changing the direction of a model’s eyes, you can drastically change where people look.

    Now, that’s got me thinking. The effectiveness of Calls to Action are obviously also dependent on what’s around them. So how do we go about getting some definitive tests?

    As I said in my earlier reply, publishing and advertising in the west already charge a premium for right placed content.

    It would be great to find out their research. Anybody know anyone in advertising who can get hold of this research. I know Ogilvy did a whole lot in the 60′s and 70′s but I lent my book of his to somebody and I never saw it again.

    PS John, this is obviously a great post and hot topic. Maybe theirs another article or chapter in an upcoming book on this?

  18. Hi Mark, you’ve brought up a great point. I think it’s worth repeating the other side of the same coin — the “F” pattern reported by Neilsen et al, is simply a reported observation of how visitors eye-scanned a page. This does not guarantee it’s a preference for how they gather such information and if designers know about the “F” pattern and repeat it, they run the real risk of a “lemmings effect” wherein they are all doing it — and visitors are all scanning that way — for no reason other than everyone else also doing it. Hardly a way to stand out in a crowd, but nevertheless safe. Which came first, the designer or the visitor?

    It makes you wonder, are visitors just trained that way now for web surfing? And if so, how strong is the habit? I’ve got a pretty strong habit of good penmanship (Thank You, Sister Mary Agnes! Your wooden ruler inflicted quite a bit of unintended “cursive”, if you’ll pardon the pun). That penmanship would be hard to break now, for sure. But I’ve also got a more conventional habit of putting my fork on the left when sitting to dinner — a habit I could easily by-pass with just a little effort. So is the “F” effect a deep or a shallow scanning habit? Does it come from the actual preferences of the visitors, or does the common elements of design during the web’s first decade simply imbue a certain level of “F” proclivity?

    And you’re correct, Bryan and I are in fact in the midst of a new book, precisely what we were working on when the Left-Side Add-To-Cart discussion first occured. I’ll leave details for a later post once we have a better sense from the major publisher as to release date and the like.

  19. Some interesting aspect and points to this discussion but at the same time it seem we try to remove the discussion of left and right from the actual design of the page.

    There might be some universal truths about left and right, but at the end of the day the main issue is connected to the individual design – and the way the design drives our eyes.

    When looking at the page live on the site (http://www.crutchfield.com/S-fahcfnLTVOg/App/Product/Item/Main.aspx?I=472TOUCH16&cc=01&wm=cl), the lack of content to the right of the product is affecting your behavior.

    It forces your attention in one of two other directions – either to the left to the price and the call to action or down to… absolutely nothing.

    The most prominent problem with this design might well be the lack of a secondary call to action at the bottom of the page to collect the “lost” users. Having just one “call to action” on this page seems something of a gambit.

    When all is said and done I do agree with everyone that challenging a convention is a challenge you want to measure the result of, not gamble you business on.

  20. [...] very smart people have contributed to this conversation, so it’s worth checking out. Testing Add-to-Cart Buttons: Stuck in the Middle With You We haven’t come up with any definitive answers yet, but the discussion is lively and has [...]

  21. Hmmm…very interesting. Has anyone tried to contact their web team to see if they actually did test this? Anyone willing to test it themselves?

  22. [...] received emails and comments from people who want to know the secret. They've read a few posts on website optimization testing, but they're disillusioned. Their tests haven't been [...]

  23. Not to be controversial, but I’ve tried the exact design from crutchfield before. My boss practically made me clone their website, and we didn’t see any improvement over our previous design. We did however see an improvement when we copied adirondackchairs.com That’s a site that’s part of the netshops network. It’s really usable, but it’s not that exciting. It tested really well against other designs we tried, and we tried to test out things that we didn’t like, but they always lost against the adirondack design.

  24. Thanks, Florida. Certainly nothing controversial there — I think you’re echoing the theme of this blog post — that the left side add-to-cart doesn’t seem to improve conversion at all, almost as expected.

    I wouldn’t mind hearing more details about how you did the Crutchfield test — are you able to share them?

  25. very interesting

  26. I agree with idea.

  27. Great article!

    I just add this article in my bookmark.

  28. There are indeed quite a number of IM metrics groups that will run this entire ‘add to cart’ button research for you. I actually believe that in each case, the positioning for the button may be different. In other words, it may perform better at the right for your site, but on someone else’s site, the middle area may be better. It all boils down to doing your own testing.

  29. As it is at the moment the majority of the buttons are on the right hand side and until such time as they become usual to be in different positions it will stay on the right.

Add Your Comments

 

Print this Article
Share

John is the co-author of the best-selling Always Be Testing and 3 other books. You can friend him on Facebook, though beware his wacky swing dancer friends, or contact him directly at john@johnquarto.com

More articles from John Quarto-vonTivadar

Marketing Optimization Blog
FREE Newsletter Sign-Up
send it once every: