Page view metrics? Bah, humbug!

(A personal, non-work mini-rant about page view metrics.)

I want to come to Yahoo’s defense about something. A recent spate of reports says that Yahoo has been surpassed by various companies in terms of page views. Why is that relatively bogus? Because of Yahoo’s switch to AJAX for its mail. According to Alexa data, 49% of Yahoo visitors go to mail.yahoo.com. Everyone knows that I take Alexa data with a grain of salt, and that 49% fraction may be high, but Yahoo definitely gets a lot of traffic from Yahoo Mail. Yahoo’s new mail system uses AJAX. And how do the metrics companies handle AJAX? Typically, not well.

At SES San Jose recently, I asked a metrics company about how they count AJAX and the metrics person got a deer-in-the-headlights look on their face. What does your traffic look like if 30-50% of your page views are suddenly converted to AJAX where a page never really reloads? Your traffic doesn’t change and you may have happier and more users, but your metrics will plummet. By the way, that’s probably why you saw this post recently on the Yahoo! Anecdotal blog talking about page views.

If you think about it, you’ll see why AJAX breaks everything. Page views are “easy.” Take all your GETs, POSTs, HEADs, or whatever and add them up. Maybe do something smart about images/CSS/JS/framesets and everyone’s metrics will roughly agree. Now, what’s the relative value of a Maps AJAX request vs. a Gmail AJAX request? Do you see why your head will hurt if you try to come up with good metrics for an AJAX site that isn’t yours? By the way, Evan Williams pointed out months ago that Google Analytics can help you track your own AJAX applications. But that’s different: if you’re writing your own AJAX site, you know what events matter. But metrics companies won’t know which AJAX events should be counted unless they read your code carefully. And that’s assuming that the ISP they’re buying data from gives them AJAX requests in the first place.

What are the takeaways from this?
1. Remember that post that said Gmail had a 2.5% market share? Shortly afterwards, you started to see a “Google succeeded in search, but hasn’t done as well in other areas” meme. I wonder if we should reconsider the origin of that notion.
2. Everybody ask your favorite metrics company how they handle AJAX. Start with “exactly what level of AJAX data access do ISPs sell to you?”.
3. If you’re doing a start-up and want impressive page view metrics, stay the hell away from AJAX.
4. If you would even *for one second* consider staying away from AJAX for the sake of impressive metrics, you’re running your start-up ass-backwards.

Okay, I’m out of steam. If I had more steam, I’d rip into the idea of an hours spent searching each month as a good metric for search engines. πŸ™‚

51 Responses to Page view metrics? Bah, humbug! (Leave a comment)

  1. ☞ An Idea:

    If in fact AJAX is depriving Webmasters of their pageView count, Length of time on page/site should be given as an alternate measure – and prioritized.

    This way there could be an assumption for AJAX sites, that excessive lengthy visits were actually because of the AJAX tech.

  2. A good mini-rant Matt. Hope you feel better now. It must be frustrating reading so much rubbish and not having the time to put the facts right. I’m glad you mentioned the gmail 2.5%. I was suspicious it wasn’t accurate.

  3. Regarding the metrics comment, do you have a suggestion for what a start up reporting back to a VC should do about this issue? Just because you say it’s ass-backwards does not mean a VC/Angel Investor/etc will suddenly cease to want it! Unless clear and agreed upon new metrics yardsticks are communicated.

    I’d like to politely offer a counter opinion to your statement, “you may have happier and more users” (using AJAX) Why does everyone in certain regions of the country always think that everyone prefers and is happier with an application converted to AJAX? I know several Yahoo! Mail users who strongly prefer the old layout and template to the new one (i.e. they tried it and switched back within a few days – you can count me among that group for now as well).

    I might be atypical in that I have a Gmail, Live.com/Hotmail and Yahoo! Mail acouont to stay on top of the changes and best practices (as well as a few other reasons not important here πŸ™‚ )

    What do I care about most as a user of web based email? SPAM. Specifically never getting a false positive while getting rid of the garbage, the ROI for the spammer goes down if effort is put into this area causing a disincentive for the spammer to continue activity as the ROI falls. At the present time, the big 3 seem more focused on how the interface looks than the actual functionality of it.

    In regards to that, Gmail recently took a step backwards with that clumsy new selector bar in the top right hand corner to take action on a message. Anything that requires two clicks where previously one click worked is not an improvement!

    In regards to Gmail I ask Santa for:
    1) Removal of above mentioned selector switch and return to old interface
    2) Automatic spell check as I type, preferably utilizing a 21st Century dictionary of terms (applies to all 3 providers)
    3) I want my email signature to always show up on top of the text I’m replying to – why would my reader want to scroll down all the way to the bottom? That is right, they don’t want to!(applies to all 3 providers)
    4) I’d love a way to sift out and remove large unwanted attachments in my Gmail Inbox.
    5) Ability to unbundle a conversation when desired
    6) An end to the Beta phase with reliable uptime

    I’ve got cookies and milk waiting for the provider who provides that experience first. πŸ™‚

  4. Perhaps the answer is for some enterprising analytics company to create a (web-SOAP-based) API developers can call each time *YOUR-AJAX-THINGY-HERE* performs an operation. In fact, it might provide more accurate analytics and better insights than the current analytics offerings.

  5. plus Gmail can be forwarded to another account so you don’t even have to go online to check it. How are they going to track that? πŸ˜‰

  6. But Bob Tabor, people who are self-reporting thei AJAX metrics will be inclined to inflate those metrics. Developers can’t be trusted to be lily-white about their sites’ usage either. πŸ™

    S.E.W., length of time on a site might be fine for something like YouTube or CNN. But for a search engine, where your main goal is to get people away from the search engine and to their destination as fast as possible, it doesn’t make as much sense. It would make even less sense to combine properties where you do want a long stay time (e.g. Google Video) with a short stay time (e.g. Google Search) into one confounded number.

    David Dalka, interesting question. Don’t assume that I think AJAX-ifying everything is automatically good though. πŸ™‚ The traditional answer might be something like growth in users or user engagement (repeat visitors, number of users that take a certain action). But you may be asking the wrong question. The right question may be: in these days when you can begin a start-up for so little money, should you be chasing after VC money in the first place?

  7. Great sensible post – points #3/4 were hilarious … but I thought you were on “verrrrrrry light blogging” with family in town – already two posts?!? πŸ˜‰

  8. I know, blashpemy for an AJAX centric company like google to admit, but chalk lack of metrics as another con for AJAX. Boy that con column sure looks a lot like Flash/actionscript con column.

    It’s funny to hear Mr. Clean Website lamenting the lack of good metrics with AJAX could possibly affecting his companys metrics. In another universe (if google didn’t use it so much) Mr. Clean would be saying to stay away from AJAX for these very same reasons.

    Certainly ironic…

  9. Having recently re-looked at Yahoo I can say I hate the new interface. Instead of just serving me the page it keeps trying to do js in the background just making it work harder and taking longer. BTW I dislike all the js nonsense in gmail too there’s just more of it in to dislike in Yahoo.

  10. It’s not difficult to implement technologies that work with Ajax to continue to record page interactions. It was a couple lines of JavaScript for me to do this on http://www.payraisecalculator.com… now my ‘page interactions’ are accurately depicted in my analytics.

    Granted, this is not an ‘externally’ facing analysis that advertisers are interested in, but I don’t know that good Advertisers care. My page interactions on Payraise Calculator have 3 times the normal CPM on the ads on that page because my visitors remain longer and can see the ads longer! Any advertiser would see this in their analysis and jump on the chance.

    It’s time for Page Views to be expanded to ‘Page Interactions’.

    Regards,
    Doug

  11. We are finally seeing industry leaders downgrading the effectiveness of page view – about time. Performance (I mean conversion oriented) metrics are the key for the web 2.0 – insiders all know this and marketers are getting there soon. Pageview metric will soon RIP

    Two things stick out that hit me as the founder and ceo of a startup in the media business (note: I think about the future of metrics 24/7)…

    Matt nails it when he says ….”If you’re doing a start-up and want impressive page view metrics, stay the hell away from AJAX. If you would even *for one second* consider staying away from AJAX for the sake of impressive metrics, you’re running your start-up ass-backwards.”

  12. Boy that con column sure looks a lot like Flash/actionscript con column.

    Yeah, but Flash will always have one con that none of the others will have: “It was invented by Macromediadobe.” πŸ˜‰

    There’s one thing in this equation that I don’t really understand, having not worked with AJAX yet. My understanding of AJAX (and I may be wrong) is that the browser sends an HTTP request to the server, which then sends back the appropriate response via Javascript, thus allowing a page to be changed without reloading.

    If that’s the case, could a stats package/toolbar not pick up the HTTP request to the server (either from the browser in the case of a toolbar or from raw logs in the case of stats) and factor it in? Or am I missing something completely here (I probably am)?

    By the way, Matt, you’re SUPPOSED to be relaxing. If you don’t, and you keep posting, the rest of us can’t and we’re gonna get slapped by our girlfriends, wives, significant others, husbands, boyfriends, or if any of us are really freaky all 5. πŸ™‚

  13. Great observations Matt.

    Wait… you call this light posting? Go hug your mom!

  14. Page views and user counts and hours searching are all fun, but they’re somewhat related to the curse of Web 1.0 – finding new metrics to explain success without looking at dollars. Companies who lack revenue and business plans want to show how well they have done, without being forced into becoming a real capitalist venture.

    AJAX or DHTML or CSS or WYSIWYG or whatever you use still generates content which may or may not become paid content. Every startup should look how to convert non-business metrics into real dollars, and then the argument is moot.

  15. How does google deal with indexing ajax heavy pages =P

  16. I don’t know of any other way to get this up for Matt to see.
    Will someone email him? πŸ™‚
    Merry Christmas to all!

    http://www.elfyourself.com/?userid=9db84e04acf11664c084938G06122310

    Jan

  17. Doug Karr, I do agree that annotating your AJAX to track page interactions a great idea. It’s also not that hard to think about building in the ability to use the back button in an AJAX application such as Gmail. There are ways to do it.

    Jan, that is just disturbing in so many ways. Joe Duck, I’ll try to go hang out with the family now. πŸ™‚

  18. Matt,

    Thanks for the suggestions for alternatives for metrics, some of those are good ones.

    I wasn’t implying VC is the be all end all (just like you apparently weren’t for AJAX – apologies if you felt I implied that) I forget the source but 98% percent of all start ups never receive a dime of funding from a VC. Yet, many startups are too significant to be started without some form of angel with wings funding it – got your checkbook ready Matt? πŸ™‚

  19. If you think Ajax tracking is bad, what about those poor schmucks that have a complete FLASH site?

    There are a lot of 100% Flash sites in the artist community and they’re totally screwed when it comes to knowing what people were really viewing.

  20. When MatchLogic arguably deployed the first 3rd person ad server and then arguably deployed the first transparent pixel and we began counting all of General Motors onlines ads in 1996 we augurably defined a cornerstone of online metrics and measurement.

    Perhaps again, the breakthru that we’re looking for will be something like the simple placement of a transparent pixel or a new count snipette that will define the “non page” world we are entering.

    Nielsen knows all about Time Spent, and Click/Operations Activity has been a part of our IAB world for a while now. Perhaps what is needed are counts and a summary score that provides a blended rating and share. OMG not TV ratings again. Maybe something new from comSCORE. Something that accounts for loyalty and rich media engagement and gives media buyers comfort that they can model effective media packaging.

  21. Doesn’t Alexa attempt to calculate the proportion of Internet viewers who visit each site in a day. This is presumably largely independent of the page view count, since it will be dependent mostly in the number of unique IP addresses.

    Surely venture capitalists have learnt to use metrics closer to eyeballs, than page hits?

    Our webmaster stuck a script to rotate some images on the front page. The other day two users clearly left their browsers opened on the front page. The extra 10,000 images freshness checks crept into the bottom of our object by hits statistics.

  22. It’s a huge issue for the analytics companies and for all analytics users. I posted my complete thoughts when this Yahoo vs MySpace page-count-issue broke.

    The Slightly Exaggerated Death of the Web Page

  23. Merry Christmas Matt and have a great 2007!

  24. comScore and NNR use locally installed HTTP proxies that keep an eye on the traffic. Historically they’ve only accepted content-type text/html as page views (PVs). Now they’re struggling to extend this to JSON, XML, etc.

    Note that the time spent is pretty much
    (time of last PV) – (time of first PV) + delta
    where last PV is based on a timeout. So you need that last PV! If all your traffic is XML after launching (pretty much the case in Yahoo! Mail beta) then the tracking companies can’t accurately track time spent either. You’ll have to start counting all traffic which will explode the page view number.

    Additional reading:

    “In fact, a higher number of PVs indicates low usability.”
    http://ontokyotime.com/2006/12/02/page-views-are-already-dead/

    “I’m convinced that advanced Internet users are underrepresented in this panel.”
    http://ontokyotime.com/2006/07/25/comscores-google-numbers-are-too-low/

  25. Merry Christmas Matt !

    Thanks for the article.

  26. “There are a lot of 100% Flash sites in the artist community and they’re totally screwed when it comes to knowing what people were really viewing.”

    More people use SWF that use only SWF, but the problem of dynamic refreshes is not new with last year’s sudden recognition of Microsoft’s proprietary XmlHttpRequest and its clones.

    Look into DART Motif for how people have been tracking advanced interactions the last few years.

    It’s true that “page views” as a metric fails when you move beyond “pages”, though. Not as bizarre as measuring automobiles by “horse power”, but close…. πŸ˜‰

    jd/adobe

  27. >>Jan, that is just disturbing in so many ways.

    Aw, Matt…I thought it was adorable and perfect! πŸ™‚
    I hope you have a great holiday. You can send that PR8 to Myrtle Beach…:-)

    Jan

  28. Merry Christmas Matt to you, your Family and to all the Google Emplyees from Merano, Italy.

    martin

  29. Matt

    Where is this year Happy ho-ho-holidays! post πŸ™

  30. Jan, I did enjoy it. But I had the sounds off on my computer (sleeping relatives have descended on the Cutts household), so I was just staring at my silent disembodied elf self boogie-ing. It was surreal. πŸ™‚

    Matt

  31. I agree with you that the number of page views are a silly way of evaluating sites, especially with ever changing technologies (ajax is this year’s example … what will next year bring?).

    All that is true, except that Yahoo, Google, MSN, and a whole host of other Media companies rely on page views as a measure of revenue potential. I mean, advertisers buy page views (impressions), so when a particular property has a noticeable decline in inventory people sit up and take notice. Agreed, it’s an explainable difference (re: ajax and tech changes), but it need to be discussed and examined.

    Food for thought.

  32. It’s even more surreal with the Alvin and the Chipmunks voice.

  33. A good mini-rant Matt. Hope you feel better now. It must be frustrating reading so much rubbish and not having the time to put the facts right. I’m glad you mentioned the gmail 2.5%. I was suspicious it wasn’t accurate.

    Thank you.

  34. Haha, Matt. I’m not sure you can call what those elves do “boogie-ing”.
    I thought it was impressive that there is no southern accent with a chipmonk voice, though. And “your” laugh was perfect…:-)
    To see your casual macho-ness on that sissy little body was so funny.
    CNN even put that website on their show all day today too. I’d hate to see their bandwidth charges this month..
    Have a great one, Matt.
    Jan

  35. Matt, you scared me with your point #3, I considered *for a second* only because you were saying it… The thought of doing ANYTHING just for show when there is a more efficient way of doing things just makes me angry.. Reminds me too much of middle management. πŸ™‚

  36. Is Google bad at dealing with heavy AJAX sites?

  37. Yahoo has always had more unique users than google. and always will. google is a big box waiting for you to search so google can get paid. yahoo is very old company and they don’t care if they make more than blue cross. on the other hand google thinks they will make as much as blue cross, but it’s out of the question. never will it happen.

  38. There’s other kinds of stuff that can’t be measured with analytics-type tools. But crazyegg.com et al have come to the rescue with mouse click heatmap tools. Google Analytics site overlay is can find the same data but not as eloquently.

    free ones:
    http://sourceforge.net/projects/clickmaps/
    http://www.glennjones.net/Post/805/AjaxLinkTracker.htm

    pay ones:
    http://www.clickdensity.com/
    http://www.clicktale.com/
    http://www.crazyegg.com/

    not sure:
    http://www.adgreed.com/

    relevant blog posts:
    http://www.37signals.com/svn/archives2/crazyegg_crazy_good.php
    http://www.seomoz.org/blogdetail.php?ID=1552

  39. Matt,

    You said, “4. If you would even *for one second* consider staying away from AJAX for the sake of impressive metrics, you’re running your start-up ass-backwards.”

    While I haven’t considered staying away from AJAX because of pageview metrics, I have been concerned about SEO of AJAX pages.

    Let’s say I designed a search engine that was AJAX. Snap comes to mind, as does searchmash. (Let’s ignore /robots.txt for this question, and assume I want Google to crawl my site, and index it). What can one do, without cloaking? I think this is a major reason many people are scared to AJAX-ify their web sites.

    Maybe you can dedicate a blog posting to this sometime (or maybe you already have and I missed it!?). Thanks

    -Chris

  40. In his own way, Bill Gates has lent support to Yahoo as well when he announced the death of the pageview.

    Web design is definitely trending away from the pageview, and instead is focusing on user experience.

    My question – of course – is how this will affect tracking tools ranging from Google Analytics to basic log analysis programs.

    It seems that the tracker programs will have to evolve substantially?

  41. GMail Chat could have been implemented using AJAX polling. Instead it’s implemented using Comet. If a metrics company compares two online IM apps where one uses AJAX and the other Comet, it may come to a wrong conclusion…

  42. Matt,

    Great to see you coming to Yahoo’s defense – eve if for a moment. πŸ˜‰

    At the end of the day, I believe that the AJAX change on Yahoo Mail goes to providing a better product – much better. As such, who then really cares about page views? This is a bit subjective, but not totally – it’s just better and people are using it because it’s a better product.

    And specifically with mail – Yahoo will know how they’re doing b/c the have the data on mail usage – and new sign-ups.

    Thanks for another great post.

  43. I am very curios about this “problem” with metrics. The largest problem I see in this area is that Ajax developers are not willing to bend to standards. Dont take offense just yet… Imagine if you will that you have an AJax application in dir /home/web/index.wtver now the Ajax requests presumably have to go somewhere on your server since Ajax is a domain restricted protocol. Therefor, creating a driectory called /home/web/ajax_docs/ should allow the “spiders” of the search engines to see your pages from your sitemap.xml file and thus get you good results, as well as accurately measuring your metrics to individual ‘locations’ in your site. I know I know… everyone is screaming right now “yea, but what about when a user clicks that link from the search result?” ok fine, I’ll admit, that can be as awkward as walking into a mall wearing only a towel. So at the top of all of you Ajax content, you put a javascript redirect to your “Home Page” plus args to redirect the user to the correct place, via Ajax in your site. The spiders cant read JS, and your users will never know the difference. There are standards, and we need to recognize them and not get so lofty as to assume we are above them. I think this idea would work. Comments welcome.

  44. SEO is good for users; what’s good for users will keep them at your site, which could create more linking opportunities like google maps has a “link to this map” on every map

  45. Heather Paquinas

    Here is the solution:

    http://www.ragepank.com/articles/45/ajax-is-here/

    to quote:

    This fix is actually quite straightforward. If you are interested in getting your site indexed properly by search engines, then it has to be crawlable with javascript off. So all hyperlinks on this site are static-looking, and crawler friendly.
    Next, I run a piece of javascript after the page has loaded which runs through the page and converts all the links to the javascript based links.
    If you don’t have javascript, then the links won’t get converted, which is exactly what we want. Because the javascript links return the same content as the regular links, this can’t be considered black-hat SEO.

  46. Great Info Matt! In Uk I see more Yahoo adverts on tv than google. Maybe google could market there engine over europe more.

  47. I’m working on a new analytics platform explicitly designed for Ajax applications – http://www.ajaxmetrics.com – and I’d love to hear what you have to say about it.

  48. Alex, i checked ur analytics platform, i doubt people will put the javascript everytime.. ajax isnt the solution for cms, its good for emails, chat, maps but i find it’s usage very limited

  49. Hi Matt nice post you have got in here,

    Johny, i totally agree with you ajax is for specific type of websites and web developers who are using ajax technology normally don’t care about the page views and so, all what they care about is to give the web visitors a better experience’s using the website and come again and again and it’s not limited.

  50. I was wondering what heavy AJAX pages would do the GoogleBot. Google does index maps from Google Maps, doesn’t it?

  51. ^
    I dont think AJAX causes any problem to the GoogleBot. I’ve seen lots of AJAX pages which are indexed perfectly

css.php