Questions? (877) 643-7244
FutureNow Article
Wednesday, Nov. 14, 2007

Measuring Visitor Engagement: Tools + Tips

By Ronald Patiro
November 14th, 2007

The other kind of engagementEngagement” in the web analytics world is about as emotionally-charged a word as it might be with someone you’ve been dating for a week. At best, it’s a conversation-killer. At worst, it’s a nuclear warhead. Marketing and analytics experts have a hard enough time agreeing on what exactly engagement is, let alone finding the metric(s) to illustrate it.

But this confusion among smart people makes sense when you think about it. When was the last time you had a face-to-face conversation with someone, only to realize they weren’t listening? How can we expect to measure engagement with metrics, when we often can’t tell if the person right in front of us is truly engaged? In fact, the only people who can reliably tell when you’re tuning out are your friends, family, and significant others. There’s a reason for that. They’ve seen your behavior before, analyzed it, and suddenly, in their minds, you’re easier to predict than Paris Hilton.

Likewise, engagement means different things to different websites. Since each site has its own unique characteristics and purpose, engagement must be defined by your site’s goals — not by Amazon’s, eBay’s, or Ms. Hilton’s.

The first step is to define how an engaged visitor behaves in terms of your site’s goals.

  • What is the ultimate purpose of your site?
    • Content site example: Get people to read my cooking blog.
    • Commerce site example: Get people to buy hats from me.
  • What actions do visitors exhibit when they’re interacting with the site and moving toward its ultimate purpose?
    • Content site examples: Reading articles, signing up for newsletter, subscribing to RSS.
    • Commerce site examples: Viewing products, reading reviews, viewing about us page, adding items to cart.

So, what exactly does an “engaged” visitor do on your site? What are some of the clues that engaged visitors leave behind in your analytics?

  • Do they stay long?
  • Do they click a lot?
  • Do they visit the site many times?
  • Are their repeat visits days apart? Weeks apart?
  • Do they penetrate deep into the site or bounce off of it?
  • Do they view lots of pages?
  • Do they take a given action like sign-up for a newsletter, refer a friend, or download a file?
  • Do they leave comments on your blog?
  • Do they link, Digg, Stumble, or otherwise find you del.icio.us? ;)
  • Do they purchase?
  • Do they purchase repeatedly?

Some sites will have an even harder time than others at capturing the elusive engagement in their analytics and may instead need to combine the quantitative data with qualitative analysis, like surveys. (Here are three great survey questions.) But proceed with caution. While many sites could benefit from using surveys on their quest to find missing pieces of the engagement puzzle, it’s easy to be mislead by what customers tell you in a survey. Ever take an online survey where the questions were fundamentally flawed? Do you prefer the taste of New Coke to CocaCola Classic? (The folks who were surveyed did.)

What’s even more dangerous is that only certain personality types bother to participate in surveys in the first place. (And good luck getting a Spontaneous customer to fill out a survey unless they’re either angry or bribed.)

A common approach to getting an initial handle on engagement is to take certain metrics that relate directly to your visitor’s main goals: those that measure if visitors are taking the actions you want them to. Monitor them closely, and see how these metrics play off each other when certain changes happen — e.g., changes in season, updates to a checkout process, special promotions, inactivity on a blog, industry trends — affect the site.

When Metrics Lie

When selecting which metrics to use, keep in mind that it’s easy to be deceived by your own numbers. Proceed with caution by giving an in-depth look into the stories these metrics can tell you before placing your trust in them. In order to be sure that your metrics are an accurate reflection of engagement, you shouldn’t take one-off metrics at face value.

“Page Views” are a great example of a metric not worth trusting on its own. In this case, it may very well be that a visitor isn’t finding what they’re looking for. Perhaps they’re “pogo-sticking” from page-to-page in search of what they need. Now you’re keeping them on the site longer, thus increasing “Time Spent,” which, again, can be deceiving by itself. Although wasting the customer’s time — so long as they don’t leave the site — will increase the page views and time spent, it may not mean you’re actually engaging visitors. (Not in the way we’d hope, anyway.)

Engagement Metrics + Toolkit

With your site’s goals in mind, and a rough understanding of how an engaged visitor behaves, here’s a sample of some metrics that may be useful relative to your site’s purpose:

  • Visitor Engagement Index = (Visits) / (Visitors)
  • Take Rate = (# of Visits Taking Part in Desired Activity) / (Visits)
  • Repeat Visitor Share = (Repeat Visitors) / (Visitors)
  • Heavy User Share = (# of Visits with X or More Pages Viewed) / (Visits)
  • Committed Visitor Share = (# of Visits Lasting Longer Than X Minutes) / (Visits)
  • Committed Visitor Index = (# of Page Views in Visits Lasting Longer Than X Minutes) / (# of Visits Lasting Longer Than X Minutes)
  • Committed Visitor Volume = (# of Page Views in Visits Lasting Longer Than X Minutes) / (Page Views)
  • Bounce Rate = (# of One Page Visits) / (Visits)
  • Scanning Visitor Share = (# of One Minute Visits) / (Visits)
  • Scanning Visitor Index = (# of Page Views in One Minute Visits) / (# of One Minute Visits)
  • Scanning Visitor Volume = (# of Page Views in One Minute Visits) / (Page Views)
  • Average Order Amount = (Total Sales) / (Total Orders)
  • Sales Per Visit = (Total Sales) /(Visits)
  • Repeat Order Rate = (# of Orders From Existing Customers) / (Total Orders)
  • Order Acquisition Ratio = (Marketing Expense/Number of Orders) / (Marketing Expense/Visits)
  • Conversion Rate = (Number of Sales) / (Visitors)
  • Page Views per Visitor = (# of Page Views) / (Visitors)
  • Average Time on Site

(Eric Peterson even offers his own complex engagement calculation, and discusses the web analytics community’s challenges to it.)

Once a set of metrics is selected that directly relates to potential engagement on your site, constructing a weighted average of the set might help. This needn’t be some painfully complicated multivariate regression model, needing someone with rocket science experience like our buddy John to make sense of it; just some metrics that can serve as a collective vital sign to measure how well your site is engaging people while carrying out its core mission.

Jim Novo makes a potent case for using visitor recency to measure engagement and how to leverage it. If you can collect information relative to the history of each specific user, and the recency of their visits, his approach can send your ROI skyrocketing.

Novo’s approach shows how recency can explain a visitor’s potential value, given their propensity to return to your site frequently, as represented by the horizontal axis below. The vertical axis, meanwhile, shows how often the visitor has taken the action being measured.

Although fuzzy and directionally correct at best, engagement is vitally important to measure because it’s a predictive metric. If your current visitors are exhibiting behaviors indicating that they’re engaged, they’re likely to return soon — and often. If you see signs that visitors are becoming less engaged with the site, it’s safe to suspect that recent changes to your site or the flow of its traffic may be working against you. Either that or your competition’s finally outdone you. Regardless, it’s always good to know when to hang it up and try something new.

Engagement can also be a useful measure of the effectiveness of your branding. If visitors are showing signs that they’re engaged with your site, they’re generally showing affinity for your brand.

While engagement has become a heated buzzword, and arguably an excuse, it’s important not to be mislead. Since it’s a state of mind for your visitors, and therefore not easily quantifiable, there’s no simple way to measure engagement. But attempting to measure will help you to keep your site from proposing on the first date.

Do you have any unique approaches for measuring engagement? Let us know. We’d love to get a conversation going in the comments.

Add Your Comments

Comments (21)

  1. You had me at hello.

    :)

    Nice work Ron, very well thought out.

    -Avinash.

  2. Thank you!

  3. [...] Measuring Visitor Engagement: Tools + Tips  Tags:web analytics web metrics linksIf you enjoyed this post, make sure you subscribe to my RSS feed and feel free to rate/review one of 150 web analytics solutions This entry was posted on Saturday, November 17th, 2007 at 2:28 pm and is filed under Links. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site. [...]

  4. Hey Ronald,

    These are definitely well thought out and practical. I am sure it will help everyone.

    Thanks,
    Mrutyunjay

  5. Pretty interesting. Very right. Using some sort of compounded index is probably the best approach, depending on the type of business. Monitoring Sentiment Analysis is probably another way for larger brands to look at Engagement as a metric. In a nutshell, Sentiment Analysis draws on computational linguistic and semantic parsing to transform unstructured online dialog into marketing insights.

    -arnaud

  6. [...] I get this comment by Ron Patiro asking: Besides simply not being actionable, what are some of the common pitfalls and tangles of [...]

  7. Hi, Ronald –

    This is very valuable stuff. Thanks for sharing!

    I’d like to humbly suggest another measure of the “engagement power” of a site. It’s called Content Interest Index, and here’s where you can download the whitepaper (free, and with no registration required):

    http://www.digitalsolid.com/?p=219

    Also, I explain the utility of this new metric in a quirky little Pecha Kucha (I’ll let you see what THAT is when you arrive). Here’s the link to that YouTube video:

    http://www.youtube.com/watch?v=C_5l6hXwzUo

    This isn’t comment spam, BTW. I’m merely trying to spread the word on an open source “meme” that can be extremely valuable to content manages sites that get a lot of traffic and place an emphasis on conversions.

    I hope you and your readers find it intriguing. I’d love to see any comments for readers on my blog or yours!

    Jeff

  8. Thanks for sharing Ronald! Great stuff.

    I often advise folks to use what is applicable to their online model and leave the rest. When we give too much math to some groups they tend to glaze over and start thinking about where they may have a beer after work. I like straight forward – actionable KPI that everyone can get behind and, more importantly, understand. Sure we can complicate measurement to where only a few folks “get it” but if they have no pull (E.g. aren’t the HIPPO – thanks for that one Avinash) nothing gets done. Just like out marketing – our KPI must be delivered to the right person with the right message (data) and at the right time.

    Knowing is half the battle – right?

    Cheers,
    AJ

  9. [...] for content managers to gauge success. The best they had were more global, site-wide metrics. NOTE: This Tools + Tips post on GrokDotCom provides an excellent run-down of some existing engagement metr… for overall site [...]

  10. Thank you for the comments and additional insight!

    Aaron: Jim Novo’s post on analysis and insight definitely helps explain why the eyes of management or a client may glaze over in front of overly complicated models.

    Jeff: I checked out your white paper on “Content Interest Index” and do see some value in it, but want to reiterate that each sites unique purpose will define how important any specific metric is. CII could be summed up as Take Rate which offers a good measure for micro-conversions and could also be related to engagement. Once again both are contingent on the site’s definition of an engaged visitor and their unique set of goals.

  11. Seems to me engagement is just a variation of the plain ol’ scoring concept used in direct marketing since forever. What is engagement, after all, but a “qualified” customer measure. As such, a customer’s engagement index has value in its ability to predict conversion (however broadly you want to define conversion). The ability to predict conversion, in turn, becomes the validation test of you engagement model. IE. you can say people with engagement x have a value of $y to the business. If not that, then what’s the point of engagement?

  12. Hi, Ron –

    Thanks for the quick and thoughtful feedback.

    I’m not sure I agree with you, however, on Take Rate being like CII.

    CII measures at the page level, not the site level. Consider it a “micro-Take-Rate,” perhaps, because it uses two proxies for reader interest as the “mini-conversions” of that page, and can be used as a coaching tool to help content managers understand the impact of their content changes over time.

    Its primary value is as a coaching tool. What the CII “coaches” is reader engagement, as measured by their readers’ willingness to share the content or send it to their printer (for filing, annotation or physical pass-alongs).

    Your greater point is quite correct, of course. Everything relies on the objectives of your site. There is not one-size-fits-all engagement metric.

  13. Robert: Measuring engagement certainly is related to the bottom line as you stated, but it covers area that extends beyond the reach of direct marketing. Tracking engagement can be applied across the entire site to see if all of their traffic is tending to “engage” with them more or less over time. Actionable insight can be taken from any changes in a site’s measure of engagement, and can monitor the impact of any changes made to a site without waiting for the effect to make it all the way to the bottom line.

    Jeff: I like that Content Interest Index does apply to page level as it can give some indication of how well visitors are interacting with your content. In comparing Take Rate to CII, I do not want to say that you should abandon CII. I still say that CII is your unique formula that is very useful for your purposes and I’m sure many other sites would agree.

    To show you that it shouldn’t be applied as a universal metric we can apply it to blogs. Blogs tend to measure links and comments more heavily than friend referrals and printed pages. Applying more flexibility to a CII could be an approach for any site to create one or several Take Rate actions combined together to give their own Content Interest Index, Engagement, etc… The point is lets not get hung up on names and any rigid structures when we are dealing with metrics that must measure this very non-rigid medium of the Web. We must do what works best for the specific needs and goals of each individual site being analyzed.

  14. Measuring and knowing how your visitors behave after entering into your site will give you a brief idea whether you should start doing something better. If you have 100 visits and 112 pageviews, start doing something.

  15. It is always powerful if you could know exactly what web pages your visitors visit and how long they stay there. If you are doing PPC, you could also evaluate how well the landing page appears and converts.

  16. [...] Consumer engagement has always been required for the success of any media outlet. [...]

  17. Measuring as well as learning how your website visitors behave after entering into your blog provides you with a short notion whether you should start doing something better. Should you have a hundred visits and 112 pageviews, begin doing something in order to improve your marketing process.

  18. Thanks for sharing Ronald. I am with Avinash on this one. A lot of information to in but if you dont measure what you do or need to do you will be as profitable as you could be.

  19. is there any kind of a good benchmark/standard for average time on site? I know it would vary by industry/type of site but a basis for comparison would be helpful.

    thanks!

  20. @marie: try the Fireclick index at http://index.fireclick.com/. Or, if you use Google Analytics, you can opt in to sharing your data for benchmarking purposes in your account settings. Then, under Visitors » Benchmarking, you can see time on site benchmarked.

  21. Well google analytics kind of does this and you can get a rough idea what’s up with your site.

    But then again there’s no 100% accurate statistic so keep that in mind

Add Your Comments

 

Print this Article
Share

Two legs, two arms, and 10 typing fingers!

More articles from Ronald Patiro

Marketing Optimization Blog
FREE Newsletter Sign-Up
send it once every: