Questions? (877) 643-7244
FutureNow Post
Tuesday, Jul. 10, 2007 at 8:55 am

Nielsen’s “Time Spent”: Worst Ranking Metric Ever?

By Robert Gorell
July 10th, 2007

surreal metrics: forest or trees?Here’s something (not at all) shocking: Nielsen/NetRatings has officially replaced “page views” with “time spent” as its default ranking metric. There’s just one problem. The page view’s been dead for some time now. So, not only is this announcement one of the worst-kept secrets since Sir Elton John’s orientation–we laughed when The Wall Street Journal reported the move to “time spent” back in April–it’s like firing a mummy and replacing him with a zombie to track your website’s performance.

Is an “undead” metric any better than a dead one?

What’s funny is that Bryan Eisenberg and I were just chatting about this over IM last night, wondering what ever happened to Nielsen’s “June announcement” about this dumb sea change in metrics. For those who don’t know, Bryan eulogized the Web Page last October in his ClickZ column, and did so in person at Emetrics Summit. Still, I thought he said it best over IM (edited for grammar):

None of these metrics measure anything telling about the user experience in a Web world. They’re proxies for the old “reach” and “frequency” metrics Nielsen’s been pawning for years. When old media wonks wanted some “scientific” measurement system to “accurately” buy and sell media. Yup, those panel surveys were accurate and scientific, alright. Because of the inconsistencies from one experience to the next, this medium cannot–and by it’s nature will not–allow for comparative metrics. In this medium, the visitor’s in control and, as we can see from comScore’s cookie-based research we’ve seen, they’ll continually make it harder to measure them.

Google’s succeeded in selling lots of ads because they didn’t buy into the old school paradigm. Everyone else just doesn’t want to admit they’re waiting for their cat to bark, and for the world to go their way again. Let me know how that works out… ;)

Speaking of IM, it turns out that “time spent” now gives AOL ranking credit not just for people at AOL.com, but for people using AIM–to which we can only say, “LOL!” Google’s rank drops from #3 to #5 because they give people quick, relevant results, while AOL jumps to #1 from #6 because people are using a one-off application. Hmm…

Of course, the blogs are already going nuts. Micro Persuasion‘s Steve Rubel who, months after Bryan, predicted the death of the page view in Not-stradamus-like fashion is reveling in the news. Back in February, Rubel asked “What Willl Replace the Almighty Page View?” (whether he read my response about the futility of “time spent” is anyone’s guess). Meanwhile, some are more hesitant to rejoice:

The news immediately made Slashdot, where worries were expressed that this emphasis on viewing time will cause designers (and their bosses) to try anything they can think of to slow down the user.

OK, now that I’ve flooded you with links, here are the real questions:

  • What are the rankers at Nielsen smoking?
  • Are they aware of this thing called “tabbed browsing”?

[UPDATE: Bryan Eisenberg discusses "What Advertisers Should Be Measuring" and why "time spent" could prove disastrous.]

Add Your Comments

Comments (21)

  1. On the topic of “time spent”, I’ve decrying it’s use as a synonym of “attention” (check Compete.com twisted attention metric and other posts on the topic of Attention Economy), which are really two different metrics. Your example about Google and Yahoo is right to the point: time is an indication of success only in some cases, most of the time, it’s not.

    S.Hamel
    Web analytics practitionner and blogger
    immeria.net

  2. Reports of my death … All web metrics have substantial noise and need a lot of interpretation. But noise and interpretation is inherent to measurement.

  3. All very well to knock todays imperfect metrics, but what is the alternative solution?

  4. The Valuation of Online Ads – up in the air again……….

    Neilson is going to de-emphasize the Web 1.0 staple, the page view and look at other – woollier in our view – measures of Ad effectiveness going foward.

    “Based on everything that’s going on with the influx of Ajax and streaming, we feel total minu…

  5. [...] hour and 15 minutes, eh? How's that for "time spent"? Technorati Tags: blog buzz, [...]

  6. I understand and respect your frustration with me for not offering alternatives, but there’s good reason. There just aren’t any one-size-fits-all metrics just because an organization happens to be on the Web.

    We should question what’s being assumed here. Should Wal-Mart’s metrics for success be the same as Exxon’s? Would it make sense for Morgan Stanley’s website to measured in the same way as MySpace? (In fact, for social networking, “time spent”–although fuzzy–is probably more often a good thing.)

    Honestly, it’s not as though Google’s CEO Eric Schmidt woke up today and said, “Wow, 5th place? We’re really losing market share.” No way, because they’re not.

    One website’s “rank” vis-a-vis another is broadly meaningless. So, unless and until your metrics are tied to your own actual business goals, you might as well use page views, time spent, or time wasted. The results will only be directionally correct, at best.

    BUT if you plan the experience ahead of time, and tie it to business goals, knowing what to measure isn’t a mystery. “Conversion” and “engagement” are the preferred metrics around here, but those mean something different for each organization. That’s what every organization needs to worry about–not what Nielsen thinks will appease advertisers in a world with Ajax.

    Thanks for the feedback everyone. Hope this further clarified my opinion.

  7. Active time (scrolling, clicking, typing etc.) could be interesting, but a poorly designed website could cause unnecessary interactions.

    Passive time is pointless. We don’t know if the visitor is busy reading, surfing in tabs or went out for a walk!

    As an advertiser I dislike CPM since it doesn’t say anything about quality, only about quantity.

  8. This has got to be a typo:

    What are the “rankers” at Nielsen smoking?

  9. Sune,

    No, that’s not a typo. It just happens to not be an accepted word; not in this context, at least. We call it Seussing.

    Hope you liked the piece, anyway. If you haven’t had a chance, you might also be interested in Bryan Eisenberg’s follow-up article, “What Advertisers Should Be Measuring.”

  10. Thanks for the opinion – do you have a better idea than time spent? I agree, it’s not the “best” – but it’s certainly “better,” isn’t it? I’m one of the bloggers who rejoiced at Nielsen’s announcement, because even though they don’t have a great solution yet, their shift demonstrates that they know there’s a problem with web metrics and that they’re trying to come up with something better! One reason I like “time spent” metrics on my blog, even though they’re not perfect: if people are spending more time, then by and large they are actually interested in what I’m writing, which makes my posts (and, by extension, me) more influential than those that are nothing more than attention-grabbing headlines with no substance (and no time spent). A good comparison is the “influence” of People Magazine vs. The Economist. I’ll gawk at a People cover for a few seconds far more often than I will at an Economist, but once I “visit” the pages of The Economist, I will spend more time with it than with People, and will be influenced to a vastly greater extent than I am by People.

  11. Worth,

    You may be right that “time spent” makes a certain amount of sense for your blog, although I’m not exactly sure what it means to you. Perhaps it could mean that, unlike some other bloggers, you write longer posts which, in turn, take longer to read. But even if time spent does make sense for you, should the same metric necessarily make sense for People Magazine, the Economist, Google — which rightly boasts how many fractions of a second it took to give you relevant search results — or for RadioShack.com, BestBuy.com, or any number of smaller e-commerce shops? No.

    More time spent by the visitor does not NECESSARILY mean more value to the visitor, and it definitely may not mean anything to the bottom line.

    You should really read Bryan’s response to see what I mean.

    Comparative metrics are the unicorn of the Web world, as they will be for any experience-driven medium. The problem here isn’t that “time spent” is the wrong metric, it’s just not for everybody and thus useless in terms of comparison.

  12. Robert, we are in agreement with regards to there being no universal metric. Just as Mr. Nielsen himself devised various measures (points, share, etc.) and invented things like “Sweeps” (which I never did see the all-powerful relevance of, but I refuse to digress) to try to allow ad professionals to get their heads around tv and radio audience measurement, so too will the Net Ratings opportunity need to be developed and exploited.

    I think I’ll now take your advice and click the link to Bryan’s response, as those Eisen Brothers are the reasons I subscribe to this thing in the 1st place (thanks, Monday Morning Memo)!

  13. Worth,

    Thanks for the compliment. I’m hoping your blog roll will reflect that with some link love ;)

    The older Eisen Brother

  14. Done – you’re now one of a select few on the roll, since I can feel good about sending people here and, almost as important, you were bold enough to ask (no one’s ever come right out and point blank asked before – you really ARE a sales/marketing genius!) Expect to see a sharp spike in traffic over the coming months to the tune of an extra hit or two. And after making the painfully effortful act of clicking on the link to your response post, here are my impressions:
    1-site metrics should only be useful to the people who own the site; to anyone else, they would only be useful for giving them an idea of what kind of exposure someone else’s site has (since the actual site owners would have all of their own metrics and wouldn’t be relying on outside data compilers such as Nielsen), which would only be useful to people wanting to engage in advertising based on exposure (CPM) rather than results (PPC), which is absolutely ridiculous for websites since, unlike tv and radio, the web offers advertisers the ability to pay only for action taken rather than paying for “exposures” like in the bad old days – violent agreement with you on this – why would anyone in their right mind pay for a million “exposures” with no idea of whether or not any action has been taken?
    2-there’s a mention of Google working to improve the relevance of their ads, which they do to some extent; unfortunately, they appear to be working just as hard if not harder on going after the “bad old days” ad $ as they roll out their print advertising “opportunity” for the unsuspecting CPM-focused suckers out there

  15. Uh, the page view isn’t the only facile, incomplete measure. The “click” or CPA is as lame when used as the full measure of value delivered. Aren’t we really looking for a more accurate way to capture the ratio of cost to value? Is there no CPM price at which it makes sense to buy? Is there no CPA price at which there are more profitable activities elsewhere?

    Why should the publisher not be paid for delivering an identifiable audience to a marketer with poor creative, insufficient frequency, a poor understanding of his customer, or just a useless, inappropriate, product?
    To “optimize” my inventory, do i have to become an expert in every possible industry and rigorously analyze every business that comes to the door, spending my time rewriting copy, re-doing campaigns, making product changes, and deciding how smart the guy placing the ads is? Sounds a lot like what marketers and agencies get paid the big bucks to do.

    But if they’re no longer putting skin in the game, i’ll invest in these capabilities and take my return out of the marketer/agency piece of the pie as well.
    I’d much prefer delivering the audience and demographics and collaborating with the many ad buyers who know a good CPM deal and with whom i’m happy to share what we do know to make them more effective.
    It’s not as if CPM advertisers are forced to come back if they don’t get a good response.

    And that is where i wish Nielson and the assembled company would spend their time…understanding more fully the value created by advertising online. The page AND the click are inaccurate if taken alone.
    Consumers need to see a message many times before a purchase. Many products don’t lend themselves to clicks (though we’ve all seen the twisted knots marketers will tie to get those clicks…even if it mean sacrificing reach to customers. Why should i ever click on an ad for a TV show.? )

  16. Mike, in your last paragraph, you state that “consumers need to see a message many times before a purchase.” I agree. What I don’t agree with, however, is paying for an “impression” that is one of 5 ads at the bottom of a page that people need to be interested enough in for them to scroll all the way down to the bottom of the page to even get the impression. Yet, as you say and as I agree with, advertisers are free to experiment and then stick with what produces verifiable action on the part of the potential customer. The reality is, however, that in my case, and in the case of most, we’re not working with large “experimental ad” budgets that allow us to try many different hit and miss campaigns and then go with what ends up eventually producing results (I’ve never tried paying an invoice with an experimental check or an experimental credit card – I need real dollars); in our situations, we need to only pay for what produces action and not for anything that doesn’t produce action, which is exactly what PPC allows.

  17. [...] OPA stats use the controversial Nielsen’s time spent metric, and measures sites that account for over 90% of web users and 55% of total online [...]

  18. [...] one thinks about time spent as a true metric of the quality of the web user experience, the move away from [...]

  19. [...] change in script reflects the fact that "page views" are dead (although some have replaced them with zombie metrics). Additionally, this round of GA updates makes it easier to [...]

  20. [...] Time on site is the length of visit on your website. A high time on site may indicate your visitors may be interacting extensively with your site. However, high time on site can be misleading: [...]

  21. Yeah, you know the issue is that “time spent” can mean a lot of different things. The metric has to be used in conjunction with other metrics to try and make sense of that data. For all we know, a greater “time spent” metric might just be telling you that your visitors are confused and are spending more time trying to get where they need to go.

Add Your Comments

 

Print this Article
Share

More articles from Robert Gorell

Marketing Optimization Blog
FREE Newsletter Sign-Up
send it once every: