Questions? (877) 643-7244
FutureNow Post
Thursday, Oct. 14, 2010 at 9:05 am

Don’t Confuse Usability Testing for Conversion Rate Optimization

By Marijayne Bushey
October 14th, 2010

Before working with you, usability consultants likely can give you a checklist of items they will evaluate and/or fix. That’s because usability is based on best practices, or items that apply to the majority of the people, the majority of the time. It might look something like this:

  • Navigation
  • Homepage
  • Category Pages
  • Product Pages
  • Search
  • Checkout

Typically, this checklist of focus areas will be the same for all eCommerce or Lead Generation websites, regardless of a business’s particular goals, limitations and revenue model. Certainly, the recommendations that come out of this kind of evaluation often can be very helpful. There’s nothing wrong with looking at some key areas of your website through this kind of lens. But don’t confuse this for conversion optimization… at least not intelligent (ie. informed and accountable) optimization.

These cautionary words don’t come out of nowhere. We recently had a client tell us about a company that wrote a blog post detailing exactly such an evaluation of his site, and he wondered why FutureNow’s “roadmap” for analysis is so much less concrete.  Here’s the response from Melissa Burdon, one of our lead analysts, that inspired me to share this hot tip with all of you, dear readers:

I actually found this report interesting. It brought me back to my first year with FutureNow (almost 5 years ago). Sometimes best practices are good to look at because they really focus on those stumbling blocks that affect anyone and everyone. At FutureNow, we call these conversion stumbling blocks, because they are blocks in the linear path from homepage to checkout.

There are a few issues with only looking at conversion stumbling blocks the way this company has:

Problem #1- It isn’t data driven analysis. Whether we’re looking at persuasive problems or conversion problems, we should always let our analysis be driven by the data. The data tells us where potential problems may be occurring with high bounce rates or exit rates, etc.

Problem #2- Related to the first problem of course, is the fact that this analysis isn’t focusing on high priority, highest impact things first. The data tells us where the highest impact areas of a site are. The top landing pages and top exit pages as well as problems within checkout, etc., are all shown to us via data and are the areas of the site we should focus on first to get the best bang for your buck.

Problem #3- By only focusing on conversion stumbling blocks, they’ve bucketed everyone into a single grouping of people. You should never look at your visitor from a mass market view point. You need to treat visitors as people, as individuals with specific interests, motivations and needs as well as problems and hesitations. At FutureNow, in the higher levels of service, we create personas to represent our client’s customer types. We use the Myers Briggs Personality types as the foundation for these personas, as the four personalities really approach their buying process very differently. This means visitors interact with the content in different ways; they look at different parts of the page, and respond to bullet points or text links differently; different language will attract the various types differently. We map the demographics to the psychographics to create “real” personas to help us, as well as you, really empathize with your customers, which will help you treat them as individuals. All that is to say, usability studies completely ignore treating people as unique individuals.

The bottom line: good optimization goes beyond usability to question whether or not best practices even apply to your site, and under your circumstances, and to shine a light on how your customers will interact with your website. It has a strategy to address the highest impact area of your marketing efforts first, and to prioritize the highest impact changes to those efforts. It does this by looking at your data and using that to drive your optimization efforts. This means your data drives the overall focus of the analysis, not just the specific changes to be made to your efforts.

There are some touch points that stand out as good places to begin analysis:

Beyond that, the scenarios that lead from those sources, and the data that pertain to these scenarios, are what drive the next steps of the optimization process. So, analyzing where to go at each step of the process, and being open to re-charting your direction, become important components of success. Changing business goals and KPIs (key performance indicators) are also a factor, but more on that in another post. In this sense, a pre-determined list is counterproductive because it locks you into looking at certain items first, and won’t allow you to adjust when high impact opportunities arise.

To those of you who like to know where you are headed with something, and how to know if you are “on schedule,” this ad hoc approach might make you very uncomfortable. But, when you start to hyperventilate about what your analyst will be focusing on next, step back, take a deep breath, and remind yourself that the most successful optimization processes are customized to YOUR business, your efforts, and your data.

Add Your Comments

Comments (37)

  1. I think usability is the basis of any site, regardless of monetization or motive in creation. It is at the very basic of web creation much like IA. Sadly, many people focus 100% on SEO’ing their sites and not making them usable. You can’t convert what people can’t find and don’t like using, period.

  2. You made a good point. However, usability does impact conversion rate. A website that is not usable will not have loyal visitors or will have a high bounce rate and you cannot truly optimize your conversion until you fix usability. However, I do agree that both are quite separate tasks though they do have some indirect relationship.

  3. ‘Typically, this checklist of focus areas will be the same for all eCommerce or Lead Generation websites, regardless of a business’s particular goals, limitations and revenue model.’

    It depends what your conversion goal is! Just finding what the user is looking for can also count as an conversion…

  4. Good points all around, Marijayne. Thank you also for reminding us that context determines everything, and it is the context that should dictate the route of the most suitable action.

    Looking forward to your post on “changing business goals and KPIs” now.

  5. Great post. Especially the part about the different types of visitors. Its a great idea to see visitors as different types of people who will react with the site differently rather than lumping them into one category and then wondering why your predictions about them might be wrong.

  6. @hissing kitty: So true! First, visitors must be able to take action on a website. Still, I would hasten to say that there is a difference between being able to do something, and desiring to do it. Good SEO should account for conversion principles and vice versa, and ideally they would be practiced in tandem. Check out the post written by Brendan Regan about overlaps between SEO and CRO. Or read what our friends over at SEOmoz have to say about a marriage between the two.

  7. @Roy Trent: Absolutely! Being able to take action (and subsequently fixing things that stand in the way of your visitors doing just that) does impact conversion rate. And only after we’ve ensured that an action can be taken are we ready to tackle the challenge of making a particular action more compelling to take.

  8. @Chanti: right you are! When it comes to CRO, we address this by asking three questions about any one point in a click stream:
    1) who is your customer?
    2) what is the action you want them to take?
    3) what is the information they need to feel comfortable taking that action?
    These questions naturally encompass basic usability principles: the ability to take an action is implied in question #3. Beyond that, goals can vary depending on who your customer is and where they are in their buying process. Steps along the way are called micro-conversions, while the big goal at the end of the journey is a macro-conversion.

  9. While I’d agree that CRO is broader than usability, usability is NOT just best practices and heuristic evaluation. Testing is also a huge part of usability – all of those heuristics came out of sitting people in front of sites and watching them go to work. That could be one-on-one lab testing, split testing on live sites, and even eye tracking and heat mapping. All of those tactics come out of usability, and they’re fundamental to good CRO.

  10. I really doubt that some usability consultant is the right person to ask. First of all he has to be in the same niche. When he tells me how users that are looking for real estate are behaving it could be totally different to for example marketing agency clients. So either they have a reaaaally broad experience or some of their suggestions is just useless. I worked with two already and it seemed that they read a couple of books and tried to make any standard adjustment. All their usability tests have shown big improvements and our tests concluded that it’s slightly worse. The difference was that we picked real clients that use our websites and not some random guys. The usability for a 16 year old Facebook junkie is totally different than for a 54 year old. Just my two cents. Conversion rate optimization is a totally other game though.

  11. @Dr. Pete – Agree. Usability does involve quite a bit of testing. And usability principals are certainly fundamental to CRO. We favor live testing on a case-by-case basis, even of best practices… I am sure you will agree that results of lab testing don’t always convey to the real world. As we like to say, believe what people do, not what they say they will do (or what they do in constrained or hypothetical situations). We are all for using data from real users to find problems on the site and validate possible solutions, AND we also advocate letting the data drive where you conduct your analysis in the first place.

  12. Great post. Usability testing and CRO aren’t necessarily the same thing. The types of traffic, whether its direct, search, from links can influence how a page should be designed for conversions. This type of analysis may not be uncovered in a usability test.

  13. @Marijayne – No argument there, and apologies if my initial reaction was a bit on the crabby side. Sometimes I bristle a bit at artificial distinctions, and I think good usability work and good CRO have a lot in common, including common roots. Likewise, there are plenty of bad practitioners of both.

    Usability is one tool, and I think it’s an important one, but there’s always a danger of thinking too narrowly. Anyone who thinks any website problem can be solved by JUST eye tracking or JUST heuristic evaluation or JUST “good” design is fooling themselves for the sake of feeling good about their own specialty. We all have to keep learning.

  14. @Dr. Pete: No apology necessary. You have raised a number of very good points. In much the same way that overspecialization in medicine can contribute to lack of continuity in a patient’s medical care, overspecialization in the digital marketing world can limit the health of your website. More collaborative, integrative approaches to website optimization are needed. We witness the evidence of a lack of integrative strategies every day with our clients, as well as the people who call looking for help. Good news though: I think I’ve noticed an uptick in posts from a variety of digital marketing areas calling out a similar need. :-) It’s about time!

  15. The context determines everything, and it is that should dictate the route of the best actions.

  16. #

    Usability is one tool, and I think it’s an important one, but there’s always a danger of thinking too narrowly. Anyone who thinks any website problem can be solved by JUST eye tracking or JUST heuristic evaluation or JUST “good” design is fooling themselves for the sake of feeling good about their own specialty. We all have to keep learning.

  17. Usability is the most basic criteria for any website. Without easy of usability it doesn’t matter if your site has a high page rank or not.

  18. Before optimizing a site, there should be a lot of contents related to your site. There are some people who prioritize optimizing their sites, with less content relevant to there sites.

  19. Another source of input to guide your optimization efforts before you start is customer survey feedback.

    If you have some kind of on-site survey data available, I find it useful to review this looking for pain points specific to your visitors on your site.

    Step 2 is to take this data and try to check it out within your clickstream analytics to see if you can confirm the problem and get an idea of the impact. At this point the process which you describe so well above takes over.

    It may sound contradictory but I have sometimes found that this approach will reveal things within the clickstream data which were not obvious on the surface but which do turn out to have significant impact.

    The principle remains the same as the one you’re outlining: start by looking for real evidence, specific to your site, and then move on to optimisation and user testing.

  20. @audio books : It would be best to have lot of unique content and to optimize it and build backlinks regulary..

  21. I think that both usability as well as conversion are important and they go hand-in-hand. If you spend all of your waking hours trying to attract visitors and the first impression of those visitors is not never return to your site, what have you gained?

  22. Thanks for a very usefull article. I tried to perform conversion optimization on my own shop 6 months ago, by simply changing from a 3 page checkout to a 1 page, and sales went up more then 20%. So its definitely worth doing

  23. [...] trotzdem ist es beides doch etwas vollständig anderes.Marijayne Bushney bringt es in ihrem Beitrag auf Grokdotcom schnell auf den Punkt: Es ist natürlich nichts falsch daran, seine Website auf Basis von Regeln, [...]

  24. The context determines everything, and it is that should dictate the route of the best actions. it’s true

  25. Usability is the most important criteria.

  26. Usability testing offers a rare opportunity to receive feedback from the very people the website is aimed at – before it’s too late to do anything about it. Best information.

  27. One issue this will come up against is it needs critical mass to work. Without enough people using it, it’ll just die off like so many other similar services.

    They’re gonna have to work incredibly hard to establish the same level of usage as hash tags.

  28. An experienced usability consultant does not look at a site based on best practices only. A usability review or usability test takes into account: the context, the environment, the different user types, the tasks and scenarios that are important to the user AND the tasks, scenarios, conversions that are important to the client. I think there is either a lot of misunderstanding of what usability is, or else you all have been hiring usability consultants that don’t know how to provide value!

  29. I am agree with both, as they are quite separate tasks though they have some indirect relationship.

  30. I agree with everything that was said. Good optimization has to go beyond usability. Your site has to have a certain degree of flexibility so that changes can be made smoothly and successfully when high impact conversion opportunities develop. Looking forward to the post on KPI’s and Business Goals

  31. @Susan Weinschenk, we would agree that a good usability assessment should take into account all of those things, as should any reputable marketing consultant regardless of marketing area (SEO, PPC, design, copywriting, etc).

  32. The difference was that we picked real clients that use our websites and not some random guys. The usability for a 16 year old Facebook junkie is totally different than for a 54 year old. Just my two cents. Conversion rate optimization is a totally other game though.

  33. I have only really started to get into this topic. I have a number of websites and (imho) they all score well in terms of SEO and usabilty.

    My conversion rate is not great though, and I have not really gone through my web stats in any great detail.

    Its so hard to find the time when you are up against it dealing with customers all the time. But I guess CRO is my next piece of homework.

  34. Good optimization don’t just rely on the design. Your sites has to have good contents and optimized on a proper keyword.

  35. Understanding customer behavior has a value. And I lot of that can be captured from the web analytics.

    And then, understanding how customer thinks is also priceless. I try A/B, multivariate testing for that.

    I agree that usability analysis by itself doesn’t give all that we need.

    Nice post, thanks for writing it.

  36. As a conversion rate optimiser first and foremost, I agree with this article: usability testing isn’t conversion rate optimization. Having said that, most usability consultants I know wouldn’t simply wade in and start preaching ‘best practise’ without doing some solid user research based on specific customer segments with specific user journeys in mind. I wonder if the role of a ‘Usability consultant’ has become over shadowed since 2010 with the ever more versatile User Experience consultant?

Add Your Comments


Print this Article

Mj has been proselytizing the merits of customer-centric, data-driven, continuous Conversion Rate Optimization (CRO) for FutureNow since 2007.

More articles from Marijayne Bushey

Marketing Optimization Blog
FREE Newsletter Sign-Up
send it once every: