Watching a story, part II

Make sure you read my previous post about this story first. Jim Hedger admits that his click-fraud story isn’t really about terrorism. In his post on Threadwatch, he said:

The story is really only tangentally [sic] about nefarious use of ad-sense or ysm dollars. The story is about advertiser alegations [sic] of massive click fraud. The part that got us going on the terrorism angle happened in our initial interview with source Clarence Briggs, CEO of AIT Corporation (extra-large hosting and web-services firm based in N.Carolina), he noted a USAToday article about terrorist organization Orkut sites, noting they often contained AdWords advertising. Clarence then asked the question, “Where does the money go?”

Good question eh?

We began a long investigation into AITs various claims, most of which were extremely strong suggestions Google was allowing and charging for way too many off-shore clicks when AIT had specifically requested a US and Canada only campaign.

So a huge part of Jim Hedger’s story is that AIT requested a US- and Canada-only advertising campaign, but AIT got lots of ad clicks from other countries.

Jim sent some log files from AIT to Google, and our AdWords team checked it out. It turns out that AIT didn’t set up their ad campaigns right. They may have only wanted clicks from US and Canada, but multiple AIT campaigns were set for “All Countries and Territories”.

See how I did that? Just a quick, short rebuttal.

Disclosure: Just to be safe, I ran this post and the last post by someone from our legal and PR teams.

25 Responses to Watching a story, part II (Leave a comment)

  1. So are we just supposed to take your word for it? He has shown proof, I would expect the same from G.

  2. Matt,
    If you are going to try to debunk every idiot who comes up with an outrageous story you are going to have a very long and tiring time doing so.

    I’s something like whack a mole.

  3. rishi, dude, shhhhhhh!!!

    Let him keep doing this. Between Matt’s subtle wit and the sheer stupidity of the original stories, this is high comedy at its very best, my friend!

    Where else will you find stuff that is as thoroughly entertaining as the last few posts have been, uninterrupted by commercials, news updates, or other annoyances?

    Keep ’em comin’, Matt! I can’t speak for anyone else, but after the week I had, I needed an excuse to ROTFLOL!! And my Segway’s broken.

  4. Cameron, the AdWords team investigated, and multiple campaigns were not set to U.S./Canada only. I’ve seen MySQL queries and the resulting campaign IDs and data.

    Multi-Worded Adam, I hear that YouTube is highly entertaining. ๐Ÿ™‚

  5. Mikkel deMib Svendsen

    Matt, I have personally experienced exactly the same thing as AIT claims on campaigns where I do know everything was set up right – US only campaigns that showed ads to users outside the US.

    What I found back when I first saw it was, and Google at that point confirmed it, that US targeted ads was presented to users from the US + users where Google do not know the origin. I don’t think thats what advertisers expect. Are you telling us that do not happen?

    What I also found back when I saw this first time was, that IPs Google should know about they claimed was unknown to them – for example my own IP from the largest ISP here in Denmark. I asked and Google told me it was unknown. That same IP is perfectly mapped on even the cheapest IP lists you can buy. Are you telling me that Googles IP mapping is not as good as what I can pull of the web for $30? I don’t believe that ๐Ÿ™‚

    In any case, this would be much less of a problem if Google would provide us advertisers with proper documentation of what exactly it is we pay for – a log for every invoice with the IP, timestamp and referrering domain of each click. Is that so unresonable to ask for? Phone companies can do that (on the numbers you call and are being charged for) and so should Google. If “bad” clicks are then being charged for at least both parties have the same data to work from. It would take a lot of the guessing out of cases such as the AIT-case.

    To me this is about transparancy – or the lack of it. I haven’t seen any good reasons why we advertisers can’t get proper documentation of what it is we pay for. All other companies I deal with can give me details of what I pay for and sooner or later I believe (and hope) Google will have to too.

  6. Cameron, I see “proof” that an Al Qaeda-friendly site is carrying Google AdSense. I don’t see “proof” that Google mismanaged AIT’s AdWords campaign.

    Now, Matt has picked up on only one aspect of the story. I’m actually more curious about the terrorism connection. While I have questioned the validity of Google’s third-party validation of its click-fraud management practices (a researcher who, by his own admission only got his information from Google’s side of the issue and apparently made no effort to look at how click-fraud networks — the technology for which is older than Google — actually work), I would hope that Google makes a quick response to the terrorism-supporting AdSense allegations.

    Not that I have been mislead to believe that Google willfully or knowingly supprts terrorism. I just want to know that something has been done to ensure that any known Al Qaeda-friendly sites are not getting money from the program.

  7. Mikkel, I couldn’t speak to your complaint; I’ve only seen the AdWords team’s conclusion from the logs that Jim sent over from AIT. It is certainly possible that country-targeting mismatches can happen (no geolocation database is going to be perfect), but I believe the explanation in this case was misconfiguration of the campaign.

    On a side note Mikkel, I understand your desire to get a log with an invoice for every valid click. However, given that you are well-known as a self-admitted cloaking blackhat, I think it would not be smart to tell you as an advertiser exactly which clicks were detected as invalid. No offense intended, of course. ๐Ÿ™‚

    Michael Martinez, no third-party on Orkut is paid for ad clicks, so even if I started a “I think Michael Martinez is mean” group on Orkut and lots of people clicked on the ads, it wouldn’t benefit me. I haven’t seen any mention of offensive non-Orkut sites showing AdSense at this point, but I’d be happy to hear of places that shouldn’t be showing AdSense (for spam reasons or otherwise).

  8. Mikkel deMib Svendsen

    > it would not be smart to tell you as an advertiser exactly which clicks were detected as invalid.

    Matt, I am asking for the oposite – a log of what we DID pay for. Is that too much to ask? My phone company can do that and any other company I pay for anything will provide me with details of what I pay for. I think Google would save itself a lot of truoble if you where more open about this. And its not just me, Matt (black hat or not – in fact, as you know, I prefer much more colorfull hats than borring black ones :)) – most advertisers I speak to don’t understand why we cannot get details of what it is we pay for.

    On the IP list issue I am not asking for perfect lists but I must say that I was very surprised to learn that my IP was not mapped by Google, as confirmed by the support team. I am on the biggest ISP here in Denmark covering over 50% of the market. Their IPs are mapped on all lists I’ve seen – except for Google’s. I just find it very hard to believe that your data is so bad ๐Ÿ™‚

    Also, this does not answer my main concern – why do Google show ads to users they do not know where is coming from. I would assume, that if I (correctly) set up ads to target just one region Google would only show it to them – not to the confirmed users + anyone Google don’t know about. Users Google don’t know where is coming from should only see global ads, in my opinion, and I think most advertisers think thats how it works. But from what I’ve sen this is not how it works.

  9. Mikkel – Let’s imagine that a blackhat is trying to reverse engineer Google’s anti-click-fraud algorithm. He can open an Adwords account, and run a bunch of test clicks. He knows what he’s clicked. He compares that to the hypothetical click list you want to get from Google. Now he knows exactly what clicks are filtered, and what clicks are not filtered. This is a very bad thing for every Google advertiser, me included. Please Google, do not give out this information.

  10. Mikkel deMib Svendsen

    I understand that argument, Jonathan, but I just don’t think it will stand in a charge back case. Conspiracy theories such as that is not a valid argument when it comes to documenting the deliveries you charge for.

    Besides, I don’t need that list to test to test how to break click fraud algorithms, if thatโ€™s what I wanted. There are many other ways to do this much more efficient. However, I am personally not into that game at all. I am just manipulating organic results the same way most PR-agents are trying to manipulate the traditional medias (but thatโ€™s another discussion we should probably not start polluting this thread with :)). To me, there is a big difference between trying to manipulate organic results and committing click fraud – stealing money from advertisers. But thatโ€™s just my personal ethics.

    Anyway, there will be a lot more on my views on this in tomorrows StrikePoint with DaveN and I (Matt, it’s StrikePoint we host – not SEO Rockstars)

  11. Matt,

    Sorry to say that in September of this year I had my own battle over geographic targeting and click fraud with the Adwords team. My campaign was set up correctly for US and only US clicks.

    I detected a click from a Telstra IP (more documents available if requested) which was clearly in Australia. I submitted my log entry and the whois information to Adwords support. Their reply?

    “Since our tests indicate that this targeting is accurate in the vast majority of cases, we do not credit for geographic targeting.”

    Google intended to keep my money for an item I did not wish to purchase. It is as simple as that.

    To correct this I engaged in an effort involving 30 plus e-mails and multiple hours of telephone calls with Google fighting me most of the way.

    For the record, there was another click in this “disagreement” involving keyword matching, or not (on that slum known as a parked page), as was the case here.

    But most of the disagreement was about this single click and the single credit for which I had to fight Google tooth and nail. Unlike your AIT example, my campaign was set up correctly. I sent the log entry. I sent the whois. Google never denied the click occurred, the IP, my data or the location. It simple stated as quoted above – in essence, too bad, we are keeping the money.

    At the end of the month long crusade Google finally admitted it was wrong and issued the click credit and a credit to my account. But this experience illustrates a very different factual environment than that you depict in your post. Instead of looking at the evidence of a clearly offshore click on a clearly US only campaign, one click mind you, Google forced a battle before they decided to do what was right.

    This illustrates an environment where, far from trying to do everything with a little guy like me to appear to be avoiding such problems, Google has reinforced, with concrete and rebar, the appearance of intentional click fraud.

    I know you are not part of the Adwords team, but you need to know there are reasons why Google’s self proclaimed “do no harm” motto is scoffed at by so many as blatant hypocrisy.

  12. Aeronautic, I’d rather not get drawn into discussions of other situations (I don’t work on the ads side, as you mention), but I’m sure ads people will read your comment.

    Mikkel deMib Svendsen, I think Jonathan explained even better than I could why Google doesn’t break out whether any individual click was considered invalid. I’m not aware of any major search engine that does what you’re asking.

    One thing Google does do is show each advertiser the aggregate percentage of their clicks that are being discarded without the advertiser having to pay. That’s something that no other search engine does, I believe.

    Another thing Google offers is autotagging, which tags every ad click with a unique identifier in the “gclid=” parameter. That lets advertisers distinguish between the original click on an ad vs. a page reload or hitting the back button. I believe that’s also something that no other major search engine offers. Hmm. Perhaps I should do a blog post about Google-only features that go above and beyond other search engines.

  13. Matt,

    My issue was put to bed after the battle. So the Adwords team needs not to read the comment, but change their policy.

    They and the corporation have been accused in legal documents (I’m not a party) of a pattern and practice of intentional click fraud (I’ve read the law firm’s press release and some news coverage – if I’ve got that wrong I’m happy to be corrected). If that is not their intent and habit, perhaps they should respond in a manner different than they did to me?

    Like the others posting earlier, there is zero credibility in the claim of inaccurate geographic databases coming from an organization like Google.

    In my specific case, Telstra is a gargantuan ISP and I found the geolocation in two seconds.

    Keep in mind also the actual written response from Google I quoted exactly in my first comment.

    I translate the meaning and options here (through my own bias, of course): Not an “Ooops, we will fix that, the database missed it, awfully sorry about that old chap, care for a cuppa coffee?” (What I would call a “do no harm” approach.) That would have ended my issue in two e-mails.

    Instead, I was told in essence, “We are keeping the money and always do since we are right most of the time.” (The “we are the giant monster Google and go stick it in your ear” approach.)

    Actions in literature and life define intent, not words. I judge Google and the Adwords team on their actions.

    And you too. Your ethics and good will are illustrated perfectly by the way you allowed my comment to run unedited. And I thank you for it.

    ๐Ÿ˜‰

  14. No worries, Aeronautic. If you ever see me at a conference, please come up and say hello. I’d enjoy talking more sometime.

  15. Being one who doesn’t think too highly of Google as of late, for specific transparency reasons * I just gotta say that in my past experience, where I showed reasons why clicks should not be charged for, I did get refunded. The facts don’t lie it seems. ๐Ÿ™‚ The message I got was one of if you want a refund, show us why we should and we will refund you.
    Doesn’t exactly inspire confidence in the product though does it? Also begs the question of how, if I could identify suspect patterns via access log analysis then why couldn’t a multi billion dollar corp do the same and save me the hassle, or even say hey, you had a lot more clicks but we didn’t charge you for these, cos we believe in fairness and transparency. A cynic would argue that a primary motive might be money and greed, but I wouldn’t be so unfair as to suggest that this is a reason, but I’d certainly like to know, as I’m sure would a few million other advertisers. Luckily for Google this isn’t an issue that is firmly embedded with the mainstream media advertising world. Maybe thats whats required to push forth a policy shift.

    Maybe you are all bogged down in circuitous internal debates from competing internal constituencies , so perhaps some of the perceived foot dragging on this could be attributed to that at least, it just doesn’t wash up good though does it? Kinda gives the wrong impression, at least this a perception I glean from giving things a quick once over, trying to understand possible reasons why.

    Perhaps its for some of these reasons that Google seems to be coming under sustained attack from all sorts of angles and corners of the webmaster community. It only seems like yesterday when the love for G was still in the ascendancy, the days where the ‘do no evil’ mantra was actually believed and smiled upon for the breath of fresh air it was. Issues like this do certainly dilute the belief that such an ethos still exists. Perhaps this cynicism is inevitable; is it possible for a commercial organisation thats aggressively expanding into all manner of markets gobbling up marketshare as it trammels, in such a cut throat win at all costs world, truly capable of sustaining such a perception? Perhaps not. Money and power do have a way of changing people, sometimes not always for the best. The reverse is true too, jealousy and ruined business models have a habit of generating ill will.

    Personally, ( and you may not care even) I genuinely do not doubt your integrity Matt, just going by what you’ve said and written over the years its clrea that you are a sterling team player with a firm hand on the buttons of truth and integrity. Its obvious too, that in your view many of your fellow Googlers are of a similar disposition. It might well be that given the opportunity, you yourself would probably agree with many of the more accurate and reasonable concerns that some (Mikkel demib Svensdon) have made on this very issue of click ‘fraud’ (hate that word). Although, as I don’t know what your personal take is, one can only summise and make assumptions. Only you would know whether you have sufficient internal mindshare to impress a view for policy change, or perhaps there really are compelling arguments that make such disclosure next to impossible at best or destructive at worst.

    Ha! Who’d want to be the spokesperson that shot the golden goose? Its kinda understandable that for now at least, silence on the mechanics might be viewed as the better option.

    It’s interesting that you choose to debunk/challenge this story especially on a topic that doesn’t appear to have too much to do with your core constituency of webspam.Perhaps you just enjoy the punchup! ๐Ÿ˜€

    I wonder if we will see those other search leviathans in the space take a more proactive view as a result. What with all these ‘Google won’t tell you’ accusations flying around, what better way to promote an alternative.”Spend money with us, we show you where your clicks came from, and how much they cost” has quite a compelling ring to it.

    We can all dream I guess ๐Ÿ™‚

    * -31 ๐Ÿ™

  16. Don’t dream too much longer Rob. What you want is indeed coming. Whether the current top guns like it or not.

  17. Matt wtf …

    – Threadwatch started a discussion called โ€œIs Google Funding Al Qaeda & Hezbollah Terrorist Groupsโ€. Threadwatch is co-owned by David Naylor, who co-hosts the Strikepoint show on WebmasterRadio.

    Debunk all you want but don’t try make out this is some BH hat thing against google …. thats all I will say In Public .. on this at this time .. in fact if google needs infomation then they can go Via the correct channels.

    DaveN

  18. added you said that

    The first phase of the story was fascinating because other than Loren Baker, pretty much the only write-ups the first night were from WebmasterRadio (WMR) hosts:

    http://blog.searchenginewatch.com/blog/061207-180311
    http://www.clickz.com/showPage.html?page=3624150

    hmmm the date on that seems tell a different tale but then again it depends what your job is, Yours is fighting spam not reading the news I guess..

    http://www.google.com/search?hl=en&lr=&safe=off&client=firefox-a&rls=org.mozilla%3Aen-US%3Aofficial&hs=YQZ&q=mattcutts.com&btnG=Search

    you might want to clean up the proxy stuff and tiny url stuff and the edu stuff .. ๐Ÿ™‚ but then again bad serps is good for adwords …

    DaveN

    I’m glad these post where pointed out to me

  19. I appreciate Matt’s comments on geotargeting. As it turns out, the AdWords documentation confirms his statements that it may not work correctly all the time. In fact, there are research papers that discuss the practical limitations of geotargeting, such as http://www.ccsl.carleton.ca/~jamuir/papers/TR-06-05.pdf . Results can vary widely for many plausible reasons. Click fraud can have an even worse effect on geotargeting.

    Perhaps the AdWords documentation can be reworded to describe geotargeting as a best effort service, making it clear to all concerned parties that while efforts are made to target, results cannot be guaranteed.

    Too bad you’re not on the ads team, Matt. BTW, I find the operator in the spam protection field difficult to read. I realize this is intentional, but it might pose problems for people with poor eyesight.

  20. rob, nice comment. I think about this stuff all the time. When I talk to the AdWords folks with thoughts or suggestions, I usually come away thinking that they’re smarter than me. Which is good–I’m glad when they’ve already thought of something that I suggest.

    I think your larger point is also more interesting: how does a company try to stay true to itself and our users as it grows? How do we make sure that we advocate for our users, and that the outside world knows that? When we have discussions internally, I’ve been a fan of the idea of trying to bake the culture into our corporate DNA in different ways, so the culture doesn’t rely on the personality of a few people.

    DaveN, the Baker/Boser/Shoemoney/TW stories all appeared the night of the 6th. That agrees with what I said in the post, then it got quiet for a while as the web mulled over the claims. The short mention in SEW was on the 7th, and the ClickZ article was on the 8th. I didn’t say that it didn’t get written up, just that a) the first write-ups were mostly WMR folks (not BH folks; I don’t mean to imply that this was a BH thing), and b) not many other people wrote it up, given the considerable claims made for the story.

    CPCcurmudgeon, I agree that geotargeting is a best-effort service; if you see documentation that claims geotargeting to be 100% perfect, please point it out to me so that we can make sure the language is as accurate as possible. In November of 2000 I evaluated geolocation vendors for Google’s initial implementation of geolocation, so I’m well aware that no solution is perfect. On the other hand, do you remember the Nazi/Yahoo France case several years ago, where Yahoo claimed it was impossible to determine country-of-origin from an IP address? Measured by that incident, it’s wonderful that geotargeting has gotten so much more accurate since then.

  21. Matt, to your credit, I have never seen any documentation claiming that Google’s geolocation works 100% of the time. My suggestion is that on pages such as the Local Business Ads help page, there might be some text that gives a caveat that geolocation is a best-effort service. That way, it’s much more likely that potential customers will see it.

    I remember the Nazi/Yahoo France case. In fact, it is mentioned in the geolocation paper I provided a link to. My understanding of Yahoo’s statements is that they believed it was impossible to guarantee any level of accuracy of geotargeting results. This doesn’t mean that it would never work; certainly, there are instances where the mapping corresponds to the physical location of an (alleged) user. But it would lead (and has led) to problems when the mappings fail, and advertisers are charged for usage they were (perhaps accidently) led to believe they would not be charged for.

    BTW, Jonathan, there are parallels to click fraud in the telecom industry, such as 1-900 phone scams. However, this doesn’t stop phone companies from providing detailed billing statements when that is warranted. I think there is some merit to providing detailed click charges against IPs, and allowing advertisers to block on an IP range, provided that they understand that this might lead to the blocking of “legit” traffic if that range is transferred to another organization. I would also understand if Google needed to charge more to provide this service.

  22. a) the first write-ups were mostly WMR folks ..

    nothing to do with the fact that it was wmr that found the story ..
    or the fact that wmr called on people they are close to, to look at the data.

    the terrorist side of the story is not what i wanted to get into .. The NSA are all over that with the FBI ,, and before we broke the story I believe.

    I’m interested in the Clickdata from AIT .. and combining that with some other clients, in fact I’m just getting the data for over 100 million dollars of ppc spend.. which I said to Adam i would share. But man the TW thing imo was a low blow,., the story was nothing to do with me being posted over there and deep down you know that.

    DaveN

  23. Mikkel deMib Svendsen

    The question is not weather the geo targeting is 100% accurate. Its not and I have never claimed it could be. The question is what do Google show when they do not know the origin of a user. Right now, as I have experienced, Google show ads to users they do NOT know where is coming from even for ads that have been targeted to one region only. That is just plain wrong to do, in my opinion. Google should only show global ads (non regional targeted) to users they do not know where is from.

    Where on Googles TOS does it say “We will show you ads to users in the region you select + any other user we do not know where is coming from”?

    Google have so far claimed, oh but thats just a small number. In most cases we do fine! How do you know? How do we, the advertisers, know? For all I know Google is indeed not doing very well on this given the fact that IPs Google should, and can find out about they apparently say they have not mapped. So, Matt, is Google’s mapping bad or is Google not telling the truth when they claim that IPs from the biggest local ISPs (such as in my own case) are not in your IP database? Sorry, but as I said before, I simply do not believe that Google have data that is not as good as what I can pull of the web for a few bucks.

    The second claim from Google is, oh yes, we may have shown your ad outside the region you targeted but we never charged you for the clicks that came through. Does anyone really believe that Google will waste its inventory on commercial ads they do not get paid for? I don’t! From a business as well as a technical point of view it dosn’t make any sense. Why not give that inventory to ads that are either targeted to that region or no region at all – ads that Google could in fact legitimately charge for.

  24. Hello Everyone,

    Sorry to chime in here again but this hand wringing about the 100% “promise” of accuracy of geotargeting is irritating to say the least.

    When selecting regions the UI provides little in the way of warning or disclaimer about accuracy, but that is a classic problem of AdWords disclosure – like that of telling, or not telling, clients when their ads will run on parked pages. But I digress.

    As I think it’s fair to infer from Mikkel’s comment, I agree the issue is where in the world does Google get off charging for something they are uncertain meets the client’s criteria?

    Why does the slop spill on the customer, not Google? If Google is not 100% sure of the visitor IP, by all means don’t show the ad or don’t charge for the click.

    Why is the burden on the customer to police Google? And to do so without detailed reporting of click IPs?

    In my case reported above in this thread the IP was 124.176.218.XX (last two munged for privacy of visitor)

    How long does it take to confirm it was not in the USA? Yah.

    Google could not handle that one?

    And again, most importantly, and in direct contradiction of public statements, irrespective of the issue of how accurate their look-ups are, how does Google really handle properly reported credit requests for bad geotargeting?

    Folks, this is a direct quote from an e-mail sent to me by AdWords Support:

    โ€œSince our tests indicate that this targeting is accurate in the vast majority of cases, we do not credit for geographic targeting.โ€ (quote here real)

    They keep the money. That is the only way that can be understood. If I order a salad and the waiter delivers a steak, charges my credit card and refuses to reverse the charge, I’d view that as fraud. Especially if they tell me “Our data suggests that folks who order salads really wanted steaks in the vast majority of cases, therefore we do not credit for steaks served you did not order.” (quote here fictional)

    As stated earlier, Google did finally reverse charge in this case, but it took a month, over 30 e-mails and hours of calls. That behavior, along with the real quote above, speaks volumes about Google’s intent.

    They did not, in my case, quickly, easily, willingly or cooperatively reverse the charge and it was impeccably documented and politely requested at the outset. Read the response quote – they had no intention to reverse the click. None. And an official representative stated in writing (the quote) that this was official company policy. And if past is precedent that response was far from a personal message to me, it was pure boilerplate (template) text that reflected official approved policy.

    Frankly, the class actions don’t seem as an appropriate a remedy as seeing a bunch of State Attorney Generals investigate this a la Microsoft.

    Part of why this irks me so much is the hypocrisy of it all.

    Yet another reason is this is doing significant harm to small AdSense publishers, like me, in the content network. When advertisers pull out of the content network for fear of being ripped off by who knows who, the little publishers also get hurt. And that’s before the nightmare of so-called smart pricing.

    Thanks for the electrons Matt. I’d be honored to meet you at an event.

  25. It does seem strange that Google would not know about IPs allocated to well-known ISPs such as Telstra. However, I can imagine how this could happen. IP address blocks change hands, and it is possible that a block came into Telstra’s possession without a change to whatever databases Google was using at the time. This is one among many reasons why I feel geotargeting shouldn’t be relied on, because one cannot be sure that one’s database correctly reflects allocations that are in use.

    In terms of actually determining if a geotargeted ad should be shown, unfortunately advertisers are not provided with enough information to make comparisons. In the case where a request came in from an IP address with no mapping, but the query contained the targeted location, the ad could be shown, according to the AdWords documentation. However, advertisers don’t get a report of what comes in on the HTTP requests, so they have no way to determine what factors were applied in deciding whether an ad was shown.

css.php