A word about metrics, part III: market share of Google Docs?

I’m not sure what Google Docs market share is, but I thought it would be interesting to mention a couple data points and add a new data point.

Data point #1: Compete. Compete just estimated that 4.4M visitors stopped by Google Docs in September, which is just a hair below 2.4% of the U.S. online population, according to them. Compete buys data from ISPs, among other sources, but doesn’t reveal which ISPs sell their surfing data, so it’s hard to tell if those ISPs’ users tend toward tech-savvy vs. newbie or affluent vs. lower income. One other metrics service (Nielsen//NetRatings) has claimed that Google Docs users tend to skew toward higher-incomes and are more likely to be technology early adopters.

Data point #2: ClickStream. A recent press release from ClickStream Technologies that claims that 1% of internet surfers use Google Docs. Honestly, this felt a little low to me. So I read about how they collected their data, and I have a hunch why ClickStream might have come up with lower numbers. From the press release:

“From May to November 2008, ClickStream Technologies recruited 2,400 U.S. internet users over the age of 18 to complete a survey and install ClickSight® …. Participants were recruited through a market research firm which awards cash and prizes in exchange for completing online surveys.

– Sample is self-reported (in initial recruitment survey) as 65.5% female, 34.5% male…

A few things come to mind:
– 2,400 users is not a ton of people.

– Tech-savvy, more affluent users are probably less likely to agree to click-monitoring in exchange for cash and prizes. I would go so far as to say most tech-savvy users would actively avoid such offers. If Google Docs users really do skew more toward affluent/tech-savvy (and I think that they do), that would result in fewer Google Docs users in ClickStream’s consumer panel.

– 65.5% female users sounds way too high. I think a more representative number is something like 52% of the online population. If ClickStream is getting 65%+ female users and not even in the 50% range, there could be all kinds of sampling errors in the data, e.g. if users were recruited from sites that didn’t represent the overall internet population.

I was thinking about ClickStream’s study and how it got a fair amount of coverage that implied Google Docs might be struggling, despite the fact that ClickStream recruited a relatively small number of users by offering cash/prizes to complete online surveys. And I asked myself: “Matt, are their any application monitoring services that tech-savvy people do use?” As soon as I asked that, I remembered that I signed up for Wakoopa recently. Wakoopa is a Web 2.0 website + client-side download that lets you track and share which applications you run. It’s the sort of service that tech-savvy users like Louis Gray are likely to use, and Wakoopa just recently started tracking web apps.

Datapoint #3: Wakoopa. Let’s see how many people are using various applications on Wakoopa. A little bit of searching turned up these stats:

Windows Explorer: 23,985 people
Finder: 6,254 people => 23,985 + 6,254 = estimate of 30,239 active users

Word: 14,985 people
OpenOffice: 3,762 people
Google Docs: 1,516 people
Corel WordPerfect: 80 people

I don’t think Wakoopa says how many active users they have, so I took one popular-but-Windows-only app (Windows Explorer) and one popular-but-Mac-only app (Finder) and added them to estimate that Wakoopa has about 23,985 + 6,254 = 30,239 active users. The reasoning is that if you’re running Windows or Mac, you’d expect that Wakoopa would see you running Windows Explorer or Finder at least once. Now let’s see how ClickStream and Wakoopa compare:

Application % of users (ClickStream) % of users (Wakoopa)
Word 51% 49.6%
OpenOffice 5% 12.4%
Google Docs 1% 5.0%
WordPerfect v.12 3% 0.3%

It’s easier to see this as a graph:

Google Docs Market share graphs

According to ClickStream, users are 3-5x more likely to use WordPerfect than Google Docs. But Wakoopa’s data suggests that Google Docs is about 20x more popular than WordPerfect. So who’s right? Well, both sources of data have self-selection bias. Wakoopa gives data on at least 10x as many users as ClickStream, but you have to bear in mind that Wakoopa’s users skew toward the tech-savvy. If you have friends that sign up for cash/prizes in online studies you might lean toward the ClickStream numbers. If you run with a more Web 2.0 crowd or don’t know anyone that runs WordPerfect, you might believe the Wakoopa data. If ClickStream disclosed their percentages of (say) IE vs. Firefox/Chrome/Safari/Opera, I suspect that would also help calibrate the differences. The correct answer is probably somewhere in between the 1% estimate from ClickStream and the 5% estimate from Wakoopa.

Google Docs is clearly the underdog in this area. But I’ve talked before about how Google’s tech-savvy user base can skew usage metrics. It would be a shame if people read the ClickStream Technologies press release and failed to consider some of the additional factors in estimating market share.

29 Responses to A word about metrics, part III: market share of Google Docs? (Leave a comment)

  1. P.S. This is off-topic, but I noticed it just after publishing this post: if anyone from ClickStream Technologies stops by to read this post, it appears that your blog has been hacked by spammers. View the source on http://clickinsight.clickstreamtech.com/ to see it; sorry to be the bearer of bad news. Judging from the generator meta tag, it looks like you’re running a pretty old version of WordPress (2.0.4), so I’d also upgrade to the latest WordPress version after fixing the hacked content.

    (I’ll delete this comment after ClickStream’s blog is cleaned up.)

  2. Dave (originial)

    RE: “So who’s right? ” Probably neither are any better than semi-educated guess work. There is SO VERY MORE to producing *meaningful stats* than you have posted here.

    In fact, I would say (for now) there is no way to get meaningful stats about online tools used. Google probably has the most information to produce meaningful stats.

  3. I used to be 80% google docs/ 20% open office. However, there seems to be a lot more document incompatabilities in google docs now and I end up opening them in open office. Plus 3.0 is nice. Since I click on the documents in google docs my ISP would think that I am using google docs. However, since it usally fails my real work is in Open Office. If other people are experiencing the same thing that might explain the results.

  4. The Google Docs market share may actually be higher than even Compete’s data would indicate, and the reason relates to the other three pieces of software. The thing with Google Docs is that you have to be online to use it…which means it isn’t bundled software or something that comes with a new PC or (worse) something that comes with other software to be installed (e.g. OpenOffice).

    There is no meaningful statistics or measure in this case. The best bet would be an average, possibly a weighted average, of various stats, and even that wouldn’t be accurate.

  5. It also depends on what you mean by Google Docs use – I had tons of people checking out our baby name poll, charts, and graphs, all built with Docs. The ability to embed and share stuff like that is one of the killer parts of Docs but it doesn’t really compare to the normal use of an Excel spreadsheet.

  6. Knowing (as best I can) the proclivities of my wife and four sisters, an online, phone, or direct mail survey response (paid or otherwise) that asserts a 65% female sample seems not unreasonable.

    Ignoring the anecdotal, what’s the female percentage of all online purchase transactions?

  7. Slightly off topic, but to answer Adam above:

    You do not have to be online to use Google Docs, it uses Google Gears so you can run the apps offline. It will synch the files once you connect to the internet again. I think this is a huge misconception that is (along with the fact few people have even heard of it) holding google docs back.

    I have been using Google Docs since April, as I didn’t want to buy MS Office. It is good for basic use but lacking for more advanced users. After fairly heavy use for a few months I found I still need MS Excel and Powerpoint. I look forward to when the product is more mature though.

  8. Quite true, Jason Morrison. Just in the last couple weeks I knocked out two presentations using Google Presentations. When I was done, I exported them each to PowerPoint format. A bunch of people saw me do presentations at Web 2.0 and the WebmasterWorld Pubcon, but they didn’t know that I did all of the presentations in Google Docs. And it was quite nice; I like that you can copy a slide in one presentation and paste it in a different presentation to quickly move a slide between presentations.

  9. Interesting comparisons Matt. Your numbers seem about right. Although, one could argue how tech-savvy our users really are. We’ve got a lot of gamers and home users on the site as well who (believe it or not) don’t even know what Flickr or Twitter means. But I get your point, very cool comparison.

    We’re actually working on a seperate trends section on Wakoopa where you’ll be able to navigate through our data much more easily in the future. Posts like these are really fun for us to read, thanks.

  10. Having no stats at all would often be better than having skewed stats like Alexa or Clickstream, because then at least people would realize they don’t have any good numbers, instead of believing in the wrong numbers. But I guess there will always be these kind of services for people buying into wrong but authoritative-looking numbers: http://www.esolutionsdata.com/statistic/8153

  11. Doing stats right is hard to get a resonable result you need a decent sample size 1k is very small 3k i s common for political polls – you also have to normalise your data for age and sex so you dont get data skewed by having a biased population.

    For example I would guess that use of open office/docs is baised towards youger people not in fulltime employment.

  12. Wow – that gave me a headache. I guess I’m statistically challenged. I have to use MS Office in my business so want compatibility over the board. I also suspect that a lot of people don’t know about Google Docs which may indicate the need for getting the work out better.

  13. While questioning the data offered by Compete and ClickStream is certainly worthwhile, why not make your point more easily by providing actual Google Docs usage statistics? Bad data wouldn’t need to be dismissed if Google provided official metrics.

  14. Zac: I didn’t really word that very clearly, and that’s my fault. The point that I was trying to make is that, at some point in time, you have to be online to consciously download something in order to be able to use Google Docs, and that components required to run Docs do not come bundled with any other software that a user might have on his/her computer, either online or offline.

    OpenOffice comes with the JRE…or at least a link to an installer does that a lot of people scratch their heads as to how it appear. I’ve had at least half a dozen people ask me what it was and whether it was spyware because they had never seen it before.
    Both Word and WordPerfect can come bundled with a brand name PC, although the latter seems to be slowly phasing itself out thanks to manufacturers like Dell who allow “nothing” as the default for productivity software. Some clone builders will include one or the other as well in an effort to upsell.

    In other words, users generally have to choose to run Docs. They don’t necessarily have to choose to run the others. That’s what I was trying to say about usage.

  15. By the way, does anyone know what the percentage of people who believe in statistics that are skewed or misleading is? Just curious.

  16. What about frequency of use, too? I do use Google Docs to share / keep some of my personal data mobile, but I don’t use it very often at all compared with, say, Excel or Word which get regular use due to work. If I signed up to a system that used automated tracking over x days or weeks, chances are it would fail to register my usage of Docs as it’s so infrequent.

  17. Former Microsoft employees…

    Be careful, it’s a ‘Microsoft study’ that pollutes the Web: http://boycottnovell.com/2008/11/16/former-softies-anti-google/

  18. Morris Rosenthal

    Matt,

    Quantcast estimated 1.3 M unique visitors for Google Docs last month, just to throw another number in the ring. Seems to me they do fairly well on high traffic sites that don’t require log-ins. Of course, you could always drop by the Docs sub domain when they aren’t looking and add the Quantcast code to their pages. But maybe it would be easier to just take somebody out for drinks and ask:-)

    Morris

  19. Roy Schestowitz, interesting article. It’s true that the CEO of ClickStream Technologies worked at Microsoft for several years, but my hope is that any survey results came from unrepresentative sample data (a small number of users who self-selected to get cash/prizes) rather than any deliberate bias.

    Philipp Lenssen, that’s a hilarious site.

  20. Dave (originial)

    Well said, Philipp Lenssen.

    It amazes me the sheer volume of, otherwise intelligent people, that are taken for a ride by stats, JUST because they have nice pretty graph or alike.

  21. Google’s got a pretty big sample size that it uses for Google Trends. But I do this:
    http://trends.google.com/websites?q=doc.google.com%2Cwakoopa.com&geo=all&date=all

    No data. I think that’s because Google doesn’t report data for its own properties, however — perhaps time to rethink that policy? Doesn’t help you with the issue of tracking apps as used on the desktop, though.

  22. ok.. my head is going explode.. boooom

  23. A suggestion. I try quite often to open files that potential customers email me in google docs. I have over a 50% fail rate.

    Now, I know some of what the problems are: many files are version dependent and call on additional files the customer/client did not send/embed correctly. But, at least I can bugger with the files and get it to at least open, even if it is not presented as the client intented, in open office. At least I get some information I can respond with.

    I’m just about done trying google docs. It just fails to open to many files that are sent to me (files gmail cheerfully offers to open in google docs).

    If, at least google docs could show me the content, even if the formatting is skewed, I would use it more.

    Maybe then, market share will increase.

    Toby

  24. The most interesting result to me is that there are at least 30% of users who don’t use a word processor (as measured by Wakoopa) or use some other word processor than the ones you list. I’d expect most, if not all, technically savy users to use some sort of word processor. (Also, some people may use several word processors and be counted more than once.) IMO, there’s a problem with the statistics.

    There is another problem–an online service like Google Docs may best define “market share” as the number of minutes (or bytes, or keystrokes, etc) used by each subscriber, where a product like Word will be concerned with units purchased and OpenOffice may be more concerned with the number of installs or downloads.

    But is market share what you want to measure? Do you care about outpacing Microsoft, or providing value and uniqueness for your users? Or should you discard the idea of market share and be concerned with the good things that people have done with Google Docs, especially the things that are hard to do in other word processors?
    .

  25. Hey Matt first of all i have to say you rock, keep it up! I recently installed clicktracks on a website for a client of mine http//skinandhairproducts.com i used to have some other cheap analytics programs but i just wasn’t working. Over all we really love clicktracks, there numbers seem to be pretty accurate (i still question some of them), the only problem we have is that it really is not as user friendly as it should be. Hopefully they can fix that, but is there anyway i can test to see if there analytics are accurate? Sometimes i wonder…..Anyway thanks for everything!

  26. My definition of Google Docs user: someone who created (!) content using either Google Documents (former Writely) or Google Spreadsheet or Google Presentations. If something was shared with you and you only view it, you are not a user. If a Spreadsheet was published as a Form and you filled it, you are not a user.
    You may want to call it “active user” instead.
    I would love to know how many those “active” users Google Docs have. Zoho ( a competitor) reported 1M in Aug 2008.

  27. Michael Kariv

    Up to date (May 15, 2009) data for Wakoopa:
    zoho-office-suite,1350
    microsoft-office-word,25098
    google-docs,8097
    openoffice,6484
    windows-explorer,39939
    finder,9981

  28. Michael Kariv

    I have just made a spreadsheet with the original Matts data and new data I gathered today.
    http://spreadsheets.google.com/pub?key=rvaDvIdh6E8qqluzLDpo5yw&gid=0
    I plan to update it periodically. My interest is to see the trend. Of the 3 data sources Matt uses, only wakoopa can be used for tracking.

  29. Users generally have to choose to run Docs.

css.php