You may have heard of Peter Norvig, who is the Director of Research at Google. He’s the guy that did the Gettysburg Address as a PowerPoint presentation. Or you might have used his artificial intelligence textbook in college.
Recently Peter used several folks’ logs (including mine) as a baseline to estimate the skew in Alexa due to the self-selection bias of webmasters or SEOs installing the Alexa toolbar. The results are really interesting:
The difference is quite profound. For example, I get about twice the pageviews of mattcutts.com, but his Alexa pageview ranking is about 25 times more than mine … What that means is that people with the Alexa toolbar installed are 25 times more likely to view a page on Matt’s site versus mine, but overall, all users view twice as many pages on my site. That’s a 50 to 1 difference introduced by the selection bias of Alexa.
Read the entire post for more details. Something to think about when you use any metrics that allow for self-selection.