Andy Beal and Shuman on click fraud

Andy Beal’s recent article about click fraud is notable. Not just because he talked to Shuman Ghosemajumder, a Google program manager who works on click fraud. The more notable reason is that it led to Shuman posting on his blog to clarify and offer more info:

Our top priority is to protect advertisers, so that means not disclosing any proprietary methods which would allow click fraud perpetrators to reverse-engineer our systems. However, there is still a great deal of information we can share. I and others on our team have spent literally hundreds of hours on communications and sharing such information outside Google.

Read Shuman’s post for more details. I’m sure he had to take the precaution to run his post by the legal team, but personally, I’m delighted to see Shuman talking about this issue on his blog.

31 Responses to Andy Beal and Shuman on click fraud (Leave a comment)

  1. Matt

    Talking about Shuman Ghosemajumder. Does he participate in webmasters forum discussions? which one?

    I would love to ask him few questions.

    Thanks.

  2. While I have found Googles detection to be much better than its competitors on a pound to pound comparson 🙂 as Loren points out the numbers are still huge becuase of the volume going through Google.
    http://www.marketingpilgrim.com/2006/12/google-click-fraud-rate-two-percent.html#comment-7436

    Can Shuman answer why Michael Anthony Bradley got off with little more than a slap on the wrist?

  3. Thanks for the link, Matt. I’m glad to take this topic to my blog as well. The more information and understanding there is out there about click fraud and click fraud protection, the better off everyone (that is, Google, our advertisers, and the industry overall) will be.

    To Harith – I haven’t participated on Webmasters forums, as the topic of click fraud protection is generally of more interest to advertisers. But you can contact me through my blog and I’m happy to answer whatever questions you have.

    Best,
    Shuman

  4. Teddie, regarding the Michael Anthony Bradley case, I’m not an attorney, but I can offer this high level explanation which was given to me by our legal team, which I also covered at SES last week. Basically, this was a special case due to the extortion aspect. The legal definition of fraud requires there to be damages, and our analysis of his software indicated that we would have caught most or all of the clicks generated by it, so there would have been little or no damages. Because of the extortion aspect, this case isn’t typical of most, and legal action remains part of our toolkit for going after click fraud perpetrators.

    Shuman

  5. Thanks to pointing to the text . . . it explained a couple of questions I had about AdSense

  6. While Google’s anti ‘invalid click’ system is impressive, it generates a lot of false positives. Our company was wrongly accused of click fraud, and there are increasing numbers of other people saying the same. The problem is, Google won’t release any information to the publishers in question. This means they miss out on valuable feedback which could be used to make the system better.

  7. It´s funny, Andy Beal’s title and conclusion don’t match. His title says:

    “Google’s Click Fraud Rate is Less than 2%”

    and in the end of the article he says:

    “Google is already filtering more than 98% of invalid clicks.”

    Technically he is right of course. 2% of invalid clicks is less than 2 % of all clicks.

    If his assumption of 8 or 9% of all clicks to be invalid, and 98% of all invalid clicks are cought, then the percentage of invalid clicks not cought is like 0.18 %.

    And likely a small percentage of that is click fraud.

    So you´re looking at 1 in every 1000 to 10.000 clicks that is an undetected fraudulent click.

    200 million searches per day at a CTR of lets say 1 % is about 400 undetected fraudulent clicks per day. What´s the value of that?

    20 cents per click on average? would make it $80 per day in undeserved clicks…

    That´s interesting,.. about $2400 per month. Of course you can argue higher CTR and higher percentages of undetected fraudulent clicks, but still, it’s peanuts compared to Google’s revenues.

    If you calculate the value of all invalid clicks Google charges for, it would be a number of like 3600 invalid clicks per day. at 20 cents per click we´re looking at $720 per day or $21600 per month. Still peanuts.

  8. @Peter (IMC)

    To google it’s peanuts, But that is still money that advertisers are paying and still money google is charging.

  9. Hmmm… Interesting stuff. I do not understand how you can determine “good clicks” which originate from behind a router/firewall in which many, perhaps hundreds, of employees may be viewing a single domain, e.g. going to a single site which may be relevant to something in the current news of the day.

    It is certainly not abnormal to expect that occasionally, many different people with the “same” IP address may be clicking on a set of ads on a single site.

    How do Google et al distinguish this “valid situation” from bots operating from a single IP ?

    Cheers to all, Holiday Greetings

    William
    London, UK

  10. Good link to Shuman’s excellent blog. I still think much of the issue (as with Organic SEO complaints) is related to vocabulary. For an advertiser the technical aspect of “fraud” vs “worthless clicks” is of far less significance than to Google where IMHO Google very actively goes after fraud but only passively goes after “worthless clicks” because they pay a lot of the bills. Excellent sites can deliver bad clicks if they get a lot of their traffic from questionable sites/spyware/etc and junky looking/acting sites can deliver very high quality traffic.
    But I don’t have access to good data so like everybody I’m “profiling” rather than researching. Google could change that with an ombudsman function that would allow ‘trusted’ outsiders (e.g. a panel of retired judges/Stanford professors) to evaluate and confirm the methodology that says: “Google has no significant PPC fraud and people who say we do are just exaggerating things!”

  11. Harith, I know Shuman is way busy, but he still goes to a ton of webmaster conferences.

    Folks, I’m not gonna talk about click-fraud stuff myself in this thread. 🙂

  12. Joe, I answered your question on my blog post also. Cutting and pasting here: using spyware to deliver AdSense ads or drive traffic to your site is strictly prohibited by our program policies, and we regularly ban AdSense publishers for such violations (on a daily basis, in fact). That being said, the quality of the content on a site is a separate issue from click fraud. A site with low visual quality can actually have very valuable traffic, just as a high visual quality site may be engaging in click fraud. The key thing is that these are independent factors. In terms of addressing the relative quality of traffic from different sites, in addition to determining whether or not we believe truly malicious traffic is coming from a site, when traffic isn’t marked as invalid, our SmartPricing system still provides sliding discounts to advertisers. The “lower quality” a site’s legitimate traffic appears to be to us, the steeper the discount to the advertisers. By providing such discounts, we normalize ROI across different parts of our network. The end result is that the amount advertisers pay for sales or leads (the metric of ROI) is roughly the same on AdSense as it is on Google.com. SmartPricing reduces both Google’s and our AdSense partner’s revenues – in order to provide better ROI for our advertisers, and promote the overall health of our online advertising ecosystem. This isn’t something that is very well-known – thanks for bringing it up!

  13. S.E.W., here’s a handy tip. If you use bold 3 or more times in one post, I will probably delete it.

  14. Matt wrote,”S.E.W., here’s a handy tip. If you use bold 3 or more times in one post, I will probably delete it.”

    Now that was funny enough to be worth the price of admission…

  15. Matt

    Thanks for inviting Shuman to join us. Hopefully we see more of the bright Googlers here in Europe.

    Shuman
    Thanks for visiting us. Much appreciated.
    I understand some advertisers worry about click fraud, but we are still talking about 2% or lets say the double 4%.

    I really don’t know of any online business operation without at least 5% risk of some kind. AdWords/AdSense campaigns are not an exception.

    My first question: have you made that point clear through your discussions?

    IMO, advertisers should worry more about optimizing their AdWords/AdSenses campaigns which allow them to increase effectiveness of the campaigns with factors like 25+%. Thats not a theory but what any successful advertiser might have already discovered and practice.
    In that connection, I wish to mention that our local Googlers and the folks in Dublin have been of great help and helped us several time to understand and resolve important issues.

    My second question: have you encouraged through your discussion advertisers to communicate with their local Googlers asking for help and advice?

    The problem which we have faced isn’t originated from Google, but from few compititors which started using some of our sites names as part of their keywords. And sometimes even they copy most of our Adwords/AdSense text. To prevent that is more important for us than worry about 2% click fraud.

    My third question: have you asked what worry advertisers most, the 2% or some other issues?

    Many thanks in advance.

  16. I have a question on this: do legitimate clicks from illegitimate sites count in the fraud equation? For example, if I set up a scraper site and run GAS down the right side and a naive user stumbles upon my site and clicks the ad, does that count?

    I can’t see the number being that low for precisely that reason. I don’t think it’s all that high, but not that low either.

  17. Harith Said: “I understand some advertisers worry about click fraud, but we are still talking about 2% or lets say the double 4%.”

    If I understand it correctly Harith, it´s 2% of 8 or 9 % of all clicks. Not 2% of all clicks.

    Pro-SEO Said: “@Peter (IMC)

    To google it’s peanuts, But that is still money that advertisers are paying and still money google is charging.”

    It´s not those amounts on one advertiser Pro-SEO. It´s like 0.18% of your adwords budget on average per advertiser. It’s peanuts. (0.18% is like $1.80 in every $1000 spent. It´s peanuts.

    When you have your car fixed they probably write half an hour more of work than they actually did which costs you like $30. Nobody ever complains about that,. 🙂

    I have had one experience with click fraud (or invalid clicks). It was in a campaign that was set to spent $300 per day, then one day 10 times the usual clicks came in at a huge cost. A simple email to Google and about a week for them to investigate resulted in Google agreeing with us that these were in fact invalid clicks and they didn’t charge for them.

    It seems to me that if you do have a real problem, Google is the first to admit it. But if you try to proof that a percentage of your clicks were invalid or worse, fraudulent, then you´re wasting your time. On the other hand, these couple of really picky people do keep the Google team sharp,. 🙂

  18. Excellent reply and clarification, thx Shuman!

  19. The best analogoy I ever heard (and I wish i could remember who said it and where)

    All of the people complaining about click fraud don’t complain about non customers taking their free pens and conferences.

  20. Peter (IMC)

    “If I understand it correctly Harith, it´s 2% of 8 or 9 % of all clicks. Not 2% of all clicks.”

    We are still talking about mini-risks or in fact micro-risks within AdWords/AdSense system. That micro-risks could be easily equalized by SEO/SEM specialists by optimizing their AdWords/AdSense campaigns.

    Strange enough you don’t see many advartisers compaining about that mini-risks, rather bloggers etc… 😉

  21. I was wondering if Google was going to do anything about those stupid SE’s that extract results from the Big 3, and then are listed in SE results. And at the top they’ll have zillions of Google Ads. It’s kind of redundant to gave SE results of SE results.

  22. Ryan,

    The two situations aren’t comparable. When a company puts out pens at conferences, it’s promotional — designed for branding. Even if the people taking the pens aren’t current, or even potential customers, the people who wind up using the pens might very well be. Pens are lent to coworkers, family, friends, etc. Furthermore, the number of pens at a given conference is fixed and determined ahead of time. Your analogy might carry more weight if conference attendees could cause arbitrary numbers of pens to be manufactured on demand at the promoting company’s expense.

  23. CPC, I think they are comparable.

    The main point of a pen is to get your company name in front of people… Even if a person is clicking on your ad to make their own adsense account money, they’re still seeing your brand name in front of their eyes.

    there IS a limit on spending… if you’re running an adwords account without a max spending amount then you’re doing it quite stupidly. (granted my company spends more in a month on adwords / overture than I make in a year.. but we still have limits)

    I’ve run adwords accounts for many customers… and you’d be suprised how many people count every non-purchasing click as fraud. I think that’s the main problem here… there’s too many varying differences in the definition of fraud.

  24. Ryan,

    Perhaps I’ve missed it, but I don’t see a hard limit on AdWords accounts in the documentation. There is a daily limit, but I don’t see anything that allows one to spend a fixed amount of money on a campaign, prior to the start of that campaign. In the pen analogy, one spends a fixed amount of money on the pens that will be provided at the conference. In this case, someone who steals the pens while no one’s looking, or takes more pens than those who run the booth prefer (because they want to ensure enough pens for the expected amount of people who will visit the booth) is acting comparably to a click fraudster.

    Furthermore, I’d be surprised if there were a sizeable number of advertisers who view click fraud as a form of branding. One must also consider that programs can be written that generate clicks that are never viewed by humans.

    Actually, I’m aware that there are advertisers who will attribute nonperforming campaigns to click fraud. A problem here is that fraudsters can take advantage of the fact that ordinary web surfing need not necessarily result in conversions. So the nonconverting clicks may or may not be fraudulent; we just don’t know.

  25. ZZPrices Said,
    December 14, 2006 @ 1:40 pm

    I was wondering if Google was going to do anything about those stupid SE’s that extract results from the Big 3, and then are listed in SE results. And at the top they’ll have zillions of Google Ads. It’s kind of redundant to gave SE results of SE results.

    ZZPrices,.. I think Google agrees with these types of sites doing this as some even say they work together with Google on this. And I agree, it’s creating a bigger internet with completely redundant information, only order is changed.

  26. Shuman, thanks for the info on click quality – I keep reading about click fraud, but not much out there on click quality, and wondering how you classify.

    I am getting these questions all the time from our customers; I’d say about 1/2 of our small to medium sized AdWords users do not use the content network prior to signing up with us, we typically do turn it on for them, but the conversion rate questions invariably come up.

    What I am curious about is Google’s position on sites with only ads and nothing else, how they are classified. Just because they are ads only does not prove click fraud. But, we did a little analysis with a half dozen of our customers and found they convert very rarely…way way less than your standard content click – ok, not a huge relevation, but nice to prove it wih real customer data. Ads only sites – are they classified in some special way, or just as a poor quality site, I wonder.
    Ron
    http://www.adisem.com

  27. As far as I’m concerned, the issue with invalid clicks and click-fraud is highly overstated in popular media. AdWords is still a highly effective means of lead generation for our clients (some of which spend over 200k per month). Combine Google’s efforts at eliminating invalid clicks with comprehensive ROI analytics, and you continue to have a winning formula.

  28. Matt, can you please respond to this video we’ve been seeing online?

    http://www.youtube.com/watch?v=PuXilEu2EfI&eurl=

  29. I have just received an accusation of “Click Fraud” from google by email and wondered is there is any way I can respond to it? I know I haven’t personally been involved in click fraud (other than the odd mistaken click everyone makes) and it hasn’t happened on any site I publish adsense on without my knowledge .(Why would it, there is no gain AFAICS to anyone other than me)

    The problem is that I run a Web Building Site called WebClickGo.com which allows users to create and publish their own sites (Just like google pages etc) and I place google ads on the free sites to cover costs. (I don’t allow users to publish there own ads and actively suspend users that try)

    I have had periods where I have had much higher then average click-thoughs but can explain these completely. One site for example publishes a weekly print newspaper in India on the weekend and for a few weeks (while it was using our service) there would be large amounts of traffic as readers of there newspaper visited the site at the weekend. This generated lots of click through for a while (The content very specific and the ads very targeted) even i was surprised. (though it seems they have now got a professional site developed and redirected their domain). This is the only unusual traffic my account has seen but as I said is completely explainable.

    Problem is without info from google I cant determine if this is the “trigger” or something else completely. I would love to sort this out as like most people I don’t like being accused of something I’ve not done.

  30. Sam M Said,
    August 31, 2007 @ 2:56 pm – You should contact them through google webmasters. I am sure if there is a reasonable explnation you will have your adsense acc. back to good standing

  31. Shuman thanks for clarifying this “The “lower quality” a site’s legitimate traffic appears to be to us, the steeper the discount to the advertisers. By providing such discounts, we normalize ROI across different parts of our network.”

    Never new google had this in place. But one question if any body can answer would be great.

    Is that does google condemn website that have low visitors but high CTR even if 90% of those visitors are from keywords through google SE.

css.php