My thoughts on this week’s debate

Earlier this week I was on a search panel with Harry Shum of Bing and Rich Skrenta of Blekko (and moderated by Vivek Wadhwa) and the video now live. It’s forty minutes long, but it covers a lot of ground:

One big point of discussion is whether Bing copies Google’s search results. I’m going to try to address this earnestly; if snarky is what you want, Stephen Colbert will oblige you.

First off, let me say that I respect all the people at Bing. From engineers to evangelists, everyone that I’ve met from Microsoft has been thoughtful and sincere, and I truly believe they want to make a great search engine too. I know that they work really hard, and the last thing I would want to do is imply that Bing is purely piggybacking Google. I don’t believe that.

That said, I didn’t expect that Microsoft would deny the claims so strongly. Yusuf Mehdi’s post says “We do not copy results from any of our competitors. Period. Full stop.”

Given the strength of the “We do not copy Google’s results” statements, I think it’s fair to line up screenshots of the results on Google that later showed up on Bing:

Google Screenshot
compared with
Bing Screenshot

and

Google Screenshot
compared with
Bing Screenshot

and

Google Screenshot
compared with
Bing Screenshot

and

Google Screenshot
compared with
Bing Screenshot

and

Google Screenshot
compared with
Bing Screenshot

and

Google Screenshot
compared with
Bing Screenshot

and

Google Screenshot
compared with
Bing Screenshot

I think if you asked a regular person about these screenshots, Microsoft’s “We do not copy Google’s results” statement wouldn’t ring completely true.

Something I’ve heard smart people say is that this could be due to generalized clickstream processing rather than code that targets Google specifically. I’d love if Microsoft would clarify that, but at least one example has surfaced in which Microsoft was targeting Google’s urls specifically. The paper is titled Learning Phrase-Based Spelling Error Models from Clickthrough Data and here’s some of the relevant parts:

The clickthrough data of the second type consists of a set of query reformulation sessions extracted from 3 months of log files from a commercial Web browser [I assume this is Internet Explorer. --Matt] …. In our experiments, we “reverse-engineer” the parameters from the URLs of these [query formulation] sessions, and deduce how each search engine encodes both a query and the fact that a user arrived at a URL by clicking on the spelling suggestion of the query – an important indication that the spelling suggestion is desired. From these three months of query reformulation sessions, we extracted about 3 million query-correction pairs.”

This paper very much sounds like Microsoft reverse engineered which specific url parameters on Google corresponded to a spelling correction. Figure 1 of that paper looks like Microsoft used specific Google url parameters such as “&spell=1″ to extract spell corrections from Google. Targeting Google deliberately is quite different than using lots of clicks from different places. This is at least one concrete example of Microsoft taking browser data and using it to mine data deliberately and specifically from Google (in this case, the efforts of Google’s spell correction team).

That brings me to an issue that I raised with Bing during the search panel and afterwards with Harry Shum: disclosure. A while ago, my copy of Windows XP was auto-updated to IE8. Here’s one of the dialog boxes:

IE8 suggested sites

I don’t think an average consumer realizes that if they say “yes, show me suggested sites” that they’re granting Microsoft permission to send their queries and clicks on Google to Microsoft, which will then be used in Bing’s ranking. I think my Mom would be confused that saying “Yes” to that dialog will send what she searches for on Google and what she clicks on to Microsoft. I don’t think that IE8′s disclosure is clear and conspicuous enough that a reasonable consumer could make an informed choice and know that IE8 will send their Google queries/clicks to Microsoft.

One comment that I’ve heard is that “it’s whiny for Google to complain about this.” I agree that’s a risk, but at the same time I think it’s important to go on the record about this.

Another comment that I’ve heard is that this affects only long-tail queries. As we said in our blog post, the whole reason we ran this test was because we thought this practice was happening for lots and lots of different queries, not simply rare queries. To verify our hypothesis, rare queries were the easiest to test. To me, what the experiment proved was that clicks on Google are being incorporated in Bing’s rankings. Microsoft is the company best able to answer the degree to which clicks on Google figure into their Bing’s rankings, and I hope they clarify how much of an impact clicks on Google affect Microsoft’s rankings.

Unfortunately, most of the reply has been along the lines of “this is only one of 1000 signals.” Nate Silver does a good job of tackling this, so I’ll quote him:

Microsoft’s defense boils down to this: Google results are just one of the many ingredients that we use. For two reasons, this argument is not necessarily convincing.

First, not all of the inputs are necessarily equal. It could be, for instance, that the Google results are weighted so heavily that they are as important as the other 999 inputs combined.

And it may also be that an even larger fraction of what creates value for Bing users are Google’s results. Bing might consider hundreds of other variables, but these might produce little overall improvement in the quality of its search, or might actually detract from it. (Microsoft might or might not recognize this, since measuring relevance is tricky: it could be that features that they think are improving the relevance of their results actually aren’t helping very much.)

Second, it is problematic for Microsoft to describe Google results as just one of many “signals and features”. Google results are not any ordinary kind of input; instead, they are more of a finished (albeit ever-evolving) product

Let’s take that thought to its conclusion. If clicks on Google really account for only 1/1000th (or some other trivial fraction) of Microsoft’s relevancy, why not just stop using those clicks and reduce the negative coverage and perception of this? And if Microsoft is unwilling to stop incorporating Google’s clicks in Bing’s rankings, doesn’t that argue that Google’s clicks account for much more than 1/1000th of Bing’s rankings?

I really did try to be calm and constructive in this post, so I apologize if some frustration came through despite that–my feelings on the search panel were definitely not feigned. Since people at Microsoft might not like this post, I want to reiterate that I know the people (especially the engineers) at Bing work incredibly hard to compete with Google, and I have huge respect for that. It’s because of how hard those engineers work that I think Microsoft should stop using clicks on Google in Bing’s rankings. If Bing does better on a search query than Google does, that’s fantastic. But an asterisk that says “we don’t know how much of this win came from Google” does a disservice to everyone. I think Bing’s engineers deserve to know that when they beat Google on a query, it’s due entirely to their hard work. Unless Microsoft changes its practices, there will always be a question mark.

If you want to dive into this topic even deeper, you can watch the full forty minute video above.

268 Responses to My thoughts on this week’s debate (Leave a comment)

  1. as i know its about bing only (as he copying Google). Wish some copycaters learn something and leave copycating

  2. I watched the live feed and I couldn’t believe how Harry came across. MSFT should have held their hands up it is very clear that they were “piggy backing”. I am not saying they are right or wrong but outright denial wasn’t a good move.

    Matt you are a great public speaker! In fact every Google employee that I have come across are good public speakers!

  3. Thanks for this post and showing some screen grabs. Even if Google clicks account for 1/1000th of Bing’s algorithm, whats the probability that seven tracked ‘test’ terms found their way into Bing’s results? Were there only 7 test terms used or is this only part of the story?

  4. Excellent summary, Matt. I agree especially with the last bit. If it is truly only 1/1000th of their algo, they should drop it….not just for PR’s sake but also for the sake of unique and diverse search options.

  5. I think you got right to the heart of the issue in your last paragraph. The information they got from Google was so insignificant and unimportant that their results should not be hurt at all by removing this data. This whole thing was just silly and shows how weird Microsoft can be.

  6. I still think your examples using synthetic keywords are not generalizable to non-synthetic keywords (e.g. real words, which webmasters have created content about). Sure, there is still “copying” being done, be it through clickstream data or toolbars or whatnot, but I’m very skeptical that this is actually occurring on meaningful queries. It doesn’t change the fact that copying is being done, but if it only affects gibberish queries, it also greatly reduces the impact & severity of these allegations. If you really want to make this point, show us at least one example of it happening on a real word….

  7. My concern as a search user is that by trying to return search results that are so similar to Google we are losing additional research sources, what people in the trade would call a “search voice”. Bing seems to have become more similar to Google as time has gone on.

    Incidentally our Bloxx content filtering at work blocks Blekko as “gambling related”. Maybe a comment on the financial aspects of trying to run a search engine!

  8. Graywolf

    So are you willing to say that google doesn’t use anytoolbar data gathered from non google sites to influence google SERPS? Because that’s what it seems to me is what really happened.

  9. Rokesh Jankie

    Hi Matt,

    indeed your frustration was sometimes visible and you don’t need to apologize for your hard work.
    With the screenshots being so clear, I don’t understand why the MS guy couldn’t admit.

    Keep up the good work.

  10. Marco Tulio

    I agree with you Matt. On another note, though, would you care to explain why/how such random queries give results at all?

  11. Matt, could there be any legal repercussions of Microsoft’s “alleged” actions? If so, do you think Google may pursue them?

  12. Sorry Matt you lost me after showing N single result nonsense SERPs… and none that were longer and still exactly the same.

    If Bing shows just one result and it’s the one result that was cued by user feedback submitted by Googlers (via toolbar or whatever), then how is that “copying”? You’ve got an uphill battle here because you called them out. They admit to using user behavior. For nonsense queries, what else is there to go by? And if the BING SERP for those nonsense queries is solely is driven by your submitted activity, what else could they be but what they are? That’s a far cry from “copying Google’s SERPs”.

    I want to find a better argument in here, but example after example of nonsense queries showing single results that were planted by Googlers.. well, it’s just not compelling. In fact, it’s evidence of Googler’s hacking at the Bing algo and then publishing discovered vulnerabilities, no? At least more akin to that than “copying” ??

  13. Harith

    Matt

    In case Bing just continue copying Google serps in future. Does Google has any options to force Bing to stop doing that?

  14. Matt,

    I have read that during Google’s trial they had 20 engineers tested 100 fake keywords but only 9 came back in Bing over the trial period. That seems like a really important piece of information that was left out of this article.

    Can you give more details on what was tested? It seems like having 20 Google Engineers test 100 keyword searches and getting a 9% response rate would warrant Google, with its immense resources, to get a few (hundred) more engineers involved before going after Microsoft.

    Without full disclosure of what was tested in every article Google is coming across as trying to redirect public attention away from spam issues they have had in the past few months.

  15. Raphael DDL Oliveira

    Just a thought:
    Even if Bing uses only 1/1000th, isn’t still be tagged as “copycat” and then, stealing other company’s (Google) work and intelectual property? I mean, the query code that sorted the websites at DB to be shown as results is a big intellectual property of Google, no? So the result of a search, in that specific rank and factor is only in that way because was google’s engineers work.

    Copyright infringment maybe?

    Best regards

  16. Andy

    One experiment I really wish you had done before releasing the results (and possibly causing Microsoft to change their algorithm):

    1. create another set of webpages on random domains (e.g: random-domain.com) with each page clearly targeted to a single made-up word: (e.g.: random-domain.com/ghjuiproes.html with Title: ghjuiproes and Text: ghjuiproes)
    2. create a link on this pageto some random unrelated site.
    3. Click that link as well, and see if it shows up as a result for “ghjuiproes” searches in Bing

    If that experiment happened and didn’t work, that would make me a lot more comfortable blaming Bing.

  17. Carlos Mello

    And how does IE Suggested Sites differ from Google Toolbar’s behavior, Matt? Doesn’t they work essentially the same way?

  18. Tristan Perry

    I haven’t got much to add since I’ve expressed my views previously over at Hacker News, plus this blog post does a good job of calmly pointing out the issues. Having said that, I really love the penultimate paragraph (“Since people at Microsoft might not like this post, … always be a question mark”).

    I agree with this entirely. Given Bing’s resources, it seems bizarre that they were relying on Google’s results even a little bit. And until they sort this issue out, I definitely agree that it’s difficult to know when to give credit to Bing.

    (Plus it’s made all the more difficult considering that Bing keep sort of denying this whole mess, despite the quite conclusive proof. I suspect that if they do stop using Google’s results, we won’t know about it considering the hole that Bing have dug themselves with their denials. Ah well, que sera sera).

  19. AnonCuzImScaredOfGettingOnYourBadSide

    Google copied Overture (AdWords)
    Google copied Twitter (Buzz)
    Google copied Flickr (Picassa)
    Google copied iPhone (Android)
    Google copied MapQuest (Maps)
    Google copied Yahoo (Finance)
    Google copied Alta Viata (Translate)
    Google copied AIM/ICQ (Chat)
    Google copied Friendster (Orkut)

    I could go on, but you get the idea. Glass houses, an so on.

  20. Whatever way Bing are doing it, either through a unethical but not technically wrong method or by simply copying the end result is that the results are copied and therefore Bing trusts Google’s results more than there own.

    What is strange is that Bing haven’t even bothered to come up with a half decent excuse they simple “deny any wrongdoing” which is just ridiculous.

    I do have a question for Matt or any other Googler:

    Was this announcement deliberately timed to coincide with the recent Future of Search conference for maximum impact and media attention?

    Thanks

    Mark Collier

  21. vetinary

    Hi Matt,

    Besides the technical evidence, just by looking at Harry’s reaction one can see that the guy KNOWS they are doing something they shouldn’t be doing… It bothers him that its been revealed, and he gets very emotional about it…

    keep up the amazing work!

  22. Raphael DDL Oliveira

    @Marco Tulio
    I think you should’ve read the start of story and you will find out.

    The random queries were inserted by google’s engeneers to really check if Bing was copying Googles results.

  23. So new SEO optimisation consists in clicking on his own website in Google, using IE, to improve ranking in Bing (and Yahoo). Welcome to clicrank.

  24. Ben

    So Google caught Bing out looking over their shoulder on a couple of queries and Bing, by their silence, has admitted that what happens on its competitor’s website does affect the quality of their own product.

    What I don’t understand why Google is being so self-righteous about the whole thing – acting like they have never been evil in their life. As someone that lives in Australia, lest we forget that Google “accidently” collected personal Wi-Fi data: http://tcrn.ch/fxKb9C . Having your own engineers that work so hard accidently code a piece of functionality into the system does not really come off as the truth, more like bad PR spin.

    And much like you respect what the Bing team are doing, I respect that Google has some of the most talented programmers (and marketing) people in the world.

    It’s just had for me to understand why Google felt like they had to “dob” in Bing to the public. How does this serve the public’s interest or provide a better search experience to the general population?

  25. Jon

    Matt – This debate has been fascinating. One point of criticism towards Google that you’ve neglect to touch on, however, is that Google also collects click-stream data through Chrome & the Google toolbar. Google may or may not use that data in your ranking algorithm, but I’ve heard very little from the Google team regarding the types of data you collect and what you do with it. I think its safe to assume you could also extract query/result-clicks on Bing (or any other search engine), and possibly even collect the domains people mark as spam or content farms on Blekko, and much more, I’m sure.

    Google’s silence on this issue does strike many observers as hypocritical. We know you have the data, but I have yet to see any official policy or statement on what you are willing to do with it.

  26. Brian

    I still don’t see how those screenshots are conclusive yet.

    My first question would be, “Why is Google turning up an actual match and not a ‘No standard web pages containing your search term’ and ‘Did you mean…’?”

    The answer to this would be key, because then Bing could be picking up on the same actual text somewhere, possible a typo in a url somewhere linking to it, or a product name in and older version, that Google did.

    To contrast, if I mash the keyboard and search, I’m not getting actual matches like above.

  27. Paperino

    Nice idea putting the screenshots of the tests you have been running. The method though looks a little bit propaganda to me, because:
    1. it’s showing the screenshot of only the successful outcomes of the tests, when it only affects 7-9% cases of the sample
    2. it gives extreme relevance to it when the statistical relevance of it is void
    It seems to me that you don’t have any idea on how non-PageRank based search engines work. Maybe you are trying to discover it, that’s why you employed 20 engineers to proof what you claim is “copying”.
    Also the timing of this disclosure is quite suspicious: why now, when people are asking if PageRank is succumbing to WebFarms and your only reply so far has beens “Our metrics say that our results are better then ever”? The answer to the question on how Google plans to solve this would be far more interesting than defining a “7-9% of the cases Bing algorithms considered relevant enough many user clicks on a Google fabricated link to show the link in their search results” a proof of copying.
    I also imagine that Chrome and the Google Toolbar report on user clicks as well. What do you do with that info? Do you discard as not useful the links of certain sites like Bing? How do you determine which sites should be discarded as a valid source?
    In the meanwhile, just try for once to not be evil…

  28. I watched the whole debate and feel that it was pretty obvious that Bing has been caught red handed. It’s not an issue of whether Google owns user click-through data or not, it’s an issue that Bing purports to be a first-class search engine, “The” alternative to Google, and yet defers to Google’s own ranking algorithm instead of its own search results. To me, it’s yet another illustration that there’s simply no reason to use Bing, when you can simply use Google and get better search results. I also didn’t feel Harry Shum’s character attacks on Matt Cutts toward the end were appropriate or professional. If anything, this only discredited his argument further.

  29. First of, yea.. Bing copies the results..
    BUT, their job is do what their competitors do, just better. So can’t blame ‘em.

    Secondly, please take a look at your own disclosure on you Toolbar page: http://www.google.com/toolbar/ff/index.html – Where does it say that you gather information?
    -Concerning:

  30. Peter Kasting

    @Jon: I work on Chrome and we absolutely do NOT collect clickstream data through Chrome. Not even when you turn on the off-by-default “anonymous usage statistics”. It is very important to us to protect our users’ privacy.

    I can’t speak for Toolbar since I don’t work on it.

  31. Sucks doesn’t it Matt when you know your hard work is being used by third parties to support their search. Now think about local search, the IYP and travel industry and how much it hurts that our data is being essentially scraped on a, “Places” page. You shouldn’t throw stones from a glass Google house.

  32. If your the dominant player you should bite your lip and ‘put them down for later’ years ago i was asked by a very senior PR in BT to respond formally respond in some online forums over an instance of the Met Polices brand new switch (PABX) being left with a default password and hence VMB’s had a field day.

    MY employer unfairly was getting stick from the press – probably via ‘friends’ in the media to deflect the polices incompetence. Its probably for the best that our internal security decided that having a phone company employee posting in alt.2600 might not be for the best :-)

    This did not show either You or Google in a good light – you should leave stunts like this to the Calacanis’s and Arringtons of this world.

  33. Paperino

    It is very important to us to protect our users’ privacy.

    said the guy working for the company of the “privacy nightmares” (Google Buzz the example that comes on top of my mind). :D

  34. Very well said. But I do have to ask, is any of this really surprising at all? I mean, it’s Microsoft. Nothing is sacred at Microsoft. They have no “ideological core”, so to speak. Google does. It’s blatantly obvious. Even the infrastructure of Google’s simple mantra is built upon foundational ethics. There are no foundational ethics within Microsoft. And the expectation just isn’t there. I think that’s why this hurts Google users, fans, employees, and share holders so much… we like to believe ourselves to be an ethical bunch. Sure, the individual ethics can be argued and debated independently (and, they will always be argued and debated), but the expectation for ethical questions to actually exist at the heart of any given Google subject will always be there. Was this good? Was this evil? Was this right? Was this wrong? Moral questions will always surround Google, both internally and externally. Microsoft has never existed in a synonymous environment, and they aren’t about to move into one now. Don’t get me wrong, I’m glad you’re calling them out on their wrong doings… I’m just suggesting that they don’t give a damn. And that we shouldn’t be surprised by their lack of giving damns.

    With that said, do you know if I can use Google Checkout to purchase the Dookie Rope Chains listed above? I bets they’s tight, yo.

  35. jojo

    Excellent response Matt.

  36. Matt, I read what Google says that they did to get these specific search results on Bing. But what was not mentioned (or what I didn’t pick up on) is the fact that Bing customizes search results, just like Google does. So, how much of what Bing did was because of customization: it’s logical for Bing to show these particular search results because 100 percent of the search query was a click on a particular URL.

    Because of customization (Bing’s customization), the query could have been run on Google, it could have been run on Blekko, it could have been run on some other search engine. And Bing would show the same search result (which is based on customization–because the user in this case clicked on the search result during this “test”).

    I personally need to see more information about the test and what exactly was done, what was clicked on, etc.) before I make up my mind on this. From what I can tell, and what you’re explaining above, it looks to me like Bing is guilty of customization and not necessarily copying Google’s search results.

  37. anon

    The Mom test?! Seriously?

    Does my mom understand when she clicks on a web page with AdSense on it that she is “granting permission” to send her queries or clicks (probably that brought her to the page, from wherever) to Google to improve ad matching and probably many other things?

    “I don’t think an average consumer realizes that if they say “yes, show me suggested sites” that they’re granting Microsoft permission to send their queries and clicks on Google to Microsoft, which will then be used in Bing’s ranking. I think my Mom would be confused that saying “Yes” to that dialog will send what she searches for on Google and what she clicks on to Microsoft. I don’t think that IE8′s disclosure is clear and conspicuous enough that a reasonable consumer could make an informed choice and know that IE8 will send their Google queries/clicks to Microsoft.”

  38. Matt,

    I feel very strongly that you are treading on thin ice with this “evidence”. You may have a valid argument that users could be unwittingly sending information to Bing. That I can agree with, but the same is true with Google’s sites and tools as well. However, that is a completely different argument.

    I could do 10, 100, 1000, 10000… random searches and take screenshots from both Bing and Google. I think if you asked a regular person about these screenshots, Microsoft’s “We do not copy Google’s results” statement would ring completely true.

    I would be curious to see the 90+ failure tests. If Google only returned what user’s wanted less than 10% of the time, Google wouldn’t be happy with those results. Why is that success rate okay in this instance? Why did Bing fail to “copy” them? Would the same test on a random website with a similar URL minus the “google.com” work? What, exactly, was your control in this experiment?

    With a 100% success rate on google.com and a 100% failure rate with a similarly formed URL on a random domain, you could definitely prove that Bing was giving special treatment to google.com. With your experiment that is pure speculation.

    I think Bing will continue to do whatever it is they are doing and you have only hurt your reputation and given Bing some much needed publicity.

    Side note: I just noticed some of the Bing results actually returned possibly more relevant actions than Google in the related searches. Interesting parsing they’ve got going on.

  39. harald

    I suggest google should start encrypting the query with some kind of one-time-pad. the q?=… should still work, but if the query comes from the usual box all the parameters are encrypted with a random key. if the server changes this random key every few minutes it’s impossible to track the queries based on the urls alone.

  40. Bill

    I have no issue whatsoever with Bing analyzing and using the data sent in by (opt-in) users (of Bing toolbar) that is further processed to help increase search relevancy results.

    “Copying” and “making use of” are not the same thing.

    What you are in effect doing here: is claiming that Bing scans Google results 24/7 every day using organic and synthetic search terms. And that they then use such results as their own.

    We both know this is false. And anyone that has looked into this and has a reasonable sense about him/herself also knows this to be false.

    I don’t see why it’s an issue for Bing to analyze and make use of the behavior of Bing toolbar users.

    All this proves is that users’ Google search behavior holds *good weight* as a metric WHEN the search terms product non-existent results. And even then, it’s only 7 or 9 out of the 100 “honeypot” terms that made it to Bing’s results page.

  41. @Renaud JOLY

    Very true, but all Google would do would be to discount second or third or whatever number of links from a particular IP address or discount high number of clicks from IPs close to each other.

    Effectively they will look for click spam.

  42. knowirelessguy

    Matt, honestly, does this really matter? they are not stealing any secret sauce, simply trying to find a way to get obscure results to obscure queries. Would anyone ever actually enter these search strings, other than the synthetic ones that google hard coded (which you have always claimed was impossible) results? So in obscure cases they might use click data from IE or toolbar to find out what users did and did not find useful from other websites around the world. Is that so much more different from other methods, spiders, back clicks, page views, etc? It is a tool at the disposal of MSFT to make their tool better that should not have been a shock that they do.

    I would think it would be more interesting for you to discuss the amount of man hours spent on this sugar trap and hacking your own code to create these. I find it more interesting that you have a mechanism that would allow synthetic results for keywords searches, that would be something that China would love to have so they can force results when people look for Tiananmen Square. Why bother with the ‘great firewall’ when google can simply insert the results they or customers want.

  43. I would be more concerned if these results were showing up where there was more than one result. This whole scandal seems contrived by Google engineers, I’m surprised that Google is even worried about Bing copying them, Google has their own problems they should be focused on.

    I’m sure most statisticians would say that you should not trust outliers in data and that is exactly what this is + entrapment. And yes Bing/MSFT track clicks everywhere, including Google. If the user opts in to that so be it. Google collects so much user data it is staggering.

  44. @Brian

    True, but I highly recommend you read this post here: http://googleblog.blogspot.com/2011/02/microsofts-bing-uses-google-search.html which explains it more in-depth.

    But to summarize, Google typed in these searches on Google and Bing and got no result and then they manually placed these totally non related results on Google (overriding the algorithm).

    Then a couple of weeks later they went to Bing and magically they returned the same results Google placed manually.

    Basically proving that Bing is copying.

    Hope this helps

    Mark

  45. Morris Rosenthal

    Matt,

    I want to rank for jkhwqerbdhbaj:-)

    Seriously though, thanks for fixing the autocomplete thing. I was sick of seeing the #1 completion for my name being Torrent.

    Now, if you could just do something about eHow and their 100,000 minorly differentiated articles on each subject under the sun, like:

    How to repair the exahust on a ’67 Mustang
    How to repair the exhaust on a 1967 Mustang
    How to fix the exhaust on a ’67 Mustang
    How to fix the exhaust on a 1967 Mustang
    How to etc…

    I realize none of those individual pages get serious traffic, but it adds up, or they wouldn’t be doing it. In the meantime, they have more (NOFOLLOWED) links to my site than I have pages. Maybe an algo tweak that looked twice at site with more pages than incoming links would be useful. According to Yahoo Site Explorer, their pages to incoming links ratio is about 2.5:1, and that includes a lot of spammy links from their article authors who were encouraged to link their own work any way possible. That link ratio is about 1:2 for WikiPedia, five times better with what would normally be thought pretty bland, if useful, content.

    Morris

  46. What I’m interested in is how Google uses Web History. If I participate, I (and Google) can currently see the queries that I perform on Bing and the places I click right after.

    Google uses Web History to personalize the search experience. My question is do you use any of the query and click history from Bing (or other search engines) in that personalization?

  47. Hi Matt

    @Bill makes a great point, that Bing could be tracking what pages a user visits and customizes their results to that. Using the toolbar to track these sites.

    Did the Google research piece/sting actually click on the results or did they simply do the search?

    And maybe if there is a Bing representative out there, maybe they could confirm whether or not they use the sites that a user visits as part of their algorithm?

    Thanks

  48. Chad

    Sorry I cannot see how this is copying at all. They are using the CLICK data of users… NOT the search data from google.

  49. Caleb LaVergne

    Some counterpoints:
    1) If someone asked Google to say “What % of your ranking algorithm is specific signal X?” (as people ask constantly), aside from the high-level classifications that have been published in the past, Google categorically does not respond (and as one of the biggest mouthpieces on this, I’m sure you know this). Expecting Microsoft to tell you exactly how heavily weighted clicks are seems unfair, and I’d hardly expect them to respond with anything other than the broad “we use lots of factors” that ALL search engines routinely respond with. Note that I’m not criticizing the non-disclosure: there are very GOOD spam-related reasons not to reveal the algorithms.
    2) Google Toolbar and Google Analytics both capture ludicrously large amounts of consumer clickstream information as well, and Google Analytics (along with the rest of the tracking industry) doesn’t message its data collection to users in any way. Similarly, Google Toolbar isn’t particularly overt in showing a big “Do you REALLY want us to collect every HTTP request you make while you’re using the Toolbar?” message. So calling out IE for not flashing big huge warning messages seems a bit hypocritical, or at the very least naive in today’s Web.
    3) As an indicator of relevance, watching searches throughout the rest of the web to see what people actually click on (vs. the search terms used) seems like a pretty darn good factor, I’d guess. I could see Microsoft looking at Amazon searches, at YouTube searches, at Baidu searches, at torrent search engines, and yes, at Google searches. Now it turns out, Google happens to have a VERY high % of the search share globally, much like a certain company’s OS share. Google certainly uses its OWN search click data for ranking, doesn’t it? I imagine Microsoft uses their own click data for ranking, too, but they’re combining it with click data from external sites.

    Some agreement:
    1) I think your strongest argument is actually a privacy argument. The reason I’m assuming that Google doesn’t use Analytics or Toolbar data to create search factors is likely because users haven’t intentionally given Google the right to use that data to feed search. I’m sure both Google & Microsoft privacy policies are written such that both COULD use it, but Google is trying to do the right thing and not mix data sources. That said, privacy on the Web often takes a strong second place to functionality, and if giving some clickstream data that’s only used in aggregate to provide better results is the price, I imagine most people would actually make that trade off (wrong as I think they are to do so).
    2) If you suspect that Microsoft is copying RESULTS, and not just consuming click data, then as others have suggested, run some searches where you click on ads instead of natural results (the keywords should be cheap :)), and see whether they index the natural results or the ads. If they’re actually scraping the results, that’s certainly another matter entirely.

  50. I think if you asked a regular person about these screenshots, Microsoft’s “We do not copy Google’s results” statement wouldn’t ring completely true.

    That would be because you’re emphasising the words “Bing is copying Google” without emphasising “because 20 Google engineers submitted this data to Bing.”

    One thing you’re not mentioning is that if the intent was to copy Google, why use clickstream data? If the majority of the people using the Bing bar click on the third result in Google, wouldn’t that rank higher in Bing? Also, in the case of the misspelled word “torsorophy”, notice how *none* of the results except for the first are the same – because the majority of people clicked the first result, and none of the others. Again, if the aim was to copy Google, it seems like a poor way of doing so.

    I am far more likely to accept that Bing is sending search terms and clickstream as a way of learning what keywords users associate links to – something you don’t seem to consider whatsoever. Especially since it seems it collects the search terms for many website’s search functionality – and not just search engines, as I will explain later.

    The paper is titled Learning Phrase-Based Spelling Error Models from Clickthrough Data and here’s some of the relevant parts[...] This paper very much sounds like Microsoft reverse engineered which specific url parameters on Google corresponded to a spelling correction. Figure 1 of that paper looks like Microsoft used specific Google url parameters such as “&spell=1? to extract spell corrections from Google.

    Something you fail to mention is that this is also done for Yahoo! and Bing. In this case, there is no indication of specifically targetting Google alone.

    In the original post to the Google blog (which annoyingly doesn’t have comments – take a cue from Bing here, please!) you or your co-workers indicated that Microsoft *wasn’t* in fact spellcorrecting the specific search term. Therefore, this whitepaper is completely irrelevent and serves only to produce an ad hom attack against Microsoft.

    This is at least one concrete example of Microsoft taking browser data and using it to mine data deliberately and specifically from Google (in this case, the efforts of Google’s spell correction team).

    No, it’s not – rather, it’s an indication that someone from Microsoft had this idea and published a whitepaper about it. Unless you have definite statements that Bing is doing this (and you guys provided evidence to the contrary in the Google blog), then this has absolutely no relevence.

    Another thing: I’m sure your engineers noticed that when they visited Google and searched for something, the Bing bar put their search query into it’s search bar. It seems that, at least internally, the Bing bar knows what search term was entered, probably from the URL’s q value. Is it ethical to send that data along with clickstream?

    They may or may not have have noticed that this happens for Yahoo!, for Bing, for Wikipedia, for Amazon, and for eBay, among others. What about for them? Does it suddenly become ethical, if the Bing bar knows the search query for Wikipedia or Amazon, to send it then?

    I don’t think an average consumer realizes that if they say “yes, show me suggested sites” that they’re granting Microsoft permission to send their queries and clicks on Google to Microsoft, which will then be used in Bing’s ranking. I think my Mom would be confused that saying “Yes” to that dialog will send what she searches for on Google and what she clicks on to Microsoft. I don’t think that IE8′s disclosure is clear and conspicuous enough that a reasonable consumer could make an informed choice and know that IE8 will send their Google queries/clicks to Microsoft.

    Do you even know if they’re doing it? In the blog post from Google you guys indicated that it was “some combination of Internet Explorer 8[...] the Bing Toolbar [...]
    or possibly some other means…” It sounds like you’re unsure of the exact mechanism. Why not have an only Suggested Sites control, or an only Bing bar control, or a neither control? Why single out Suggested Sites’ dialog?

    (As an aside, I believe if your blogger had put “We suspect that Bing is using some combination of IE or the Bing bar to send data to Bing on what people search for on Google and the Google search results they click” at the beginning of the page, then presented the evidence, I doubt you would have gottern as much of an uproar as “Bing is copying Google” does. Because collecting data on what people click on, whether it be Google or any other site, is not copying Google’s results. [To quote Mehdi, full stop. Period.])

    In any case, it doesn’t matter if the user knows what data is being collected. This is a misdirection – the accusation is that Microsoft is copying *your* results, not anonymous user data. (Of course, anonymous user data is fine to collect, as I’m sure you know full well. Do users know you’re collecting their data when they search? Why, yes, it’s all in the Privacy Policy linked at the bottom of the page! Wait a minute…)

    Unfortunately, most of the reply has been along the lines of “this is only one of 1000 signals.”

    Er, no. Their response is that (and this is important) clickstream data is one of 1000 signals. The keywords here are “clickstream data”, and there’s nothing wrong with collecting that.

    I think Bing’s engineers deserve to know that when they beat Google on a query, it’s due entirely to their hard work. Unless Microsoft changes its practices, there will always be a question mark.

    Nice turn of phrase, but it’s easy to twist something to make a competitor look bad. To wit:

    I think Google’s engineers deserve to know that their company and lead engineers don’t misrepresent their competitors or use libel against them to damage their reputation. Until Google changes their practices, there will always be a question mark.

    That’s not my actual opinion on the matter, and I’m sure you really mean well, but you lace your words with venom and emotion. It makes you guys look bad, as well as incredibly silly when you critisise a competitor for legitimate data collection.

  51. Matt, even within your own argument you invalidate your earlier claims.

    You’re claiming that Bing is copying Google’s search results when in reality they’re only using as you put it “Google’s clicks.” That’s not copying your results, that’s parsing the search query entered on Google and tying that to the users click data.

    Again, does it use information gathered from your site? Yes. But that’s not copying and your tests don’t show anything else at work.

    And to be fair, you guys do the same thing pulling data from Facebook & Twitter etc.

    Of course, since this was simply a PR stunt aimed at deflecting criticism of your own rankings, I suspect you don’t actually care about being fair or accurate in your statements.

  52. Do you have any evidence that the Suggested Sites feature sends this information back to Microsoft? I ran some packet captures and they didn’t seem to show that.

    OTOH, the Bing Toolbar clearly sends that information back… but it also asks you explicitly beforehand if it can send back information about “the searches you do, websites you visit.” That information is URLs and sometimes link titles, only.

    Full description at http://projectgus.com/2011/02/bing-google-finding-some-facts/

    I think it’s also drawing a long bow to say “Google really account for only 1/1000th” based on Bing’s assertion that they use “more than 1000 signals”. It’s just as plausible that all clickstream data only “accounts for only 1/1000th”, and some Google query URLs just happen to be in that clickstream.

    (Sorry for the repeat comments, I’m trying this a third time as it seems to maybe being eaten by a spam filter!)

  53. Tim

    Matt – Has Google released 100% of its test cases and results? Did Google test more than these handful of terms and were they 100% successful? I’m not sure if Google is showing us a one sided version of the story or this is really everything.

  54. You make a good case Matt, but is it safe to say you are a little biased?

  55. Mark

    I think the best part about this whole episode is the resulting dialogue. The masses are finally being heard…we all want the best search possible. It’s going to be fascinating to watch who will listen and get it done right.

  56. Couldn’t wait to see that discussion, thanks Matt for posting the video.

    As much as I am a fan of Microsoft, the company collectively lacks much of the common sense for communicating with the public and for delivering great user experience in my opinion. I wish they got better at both… One tries so hard to find the cool and one is inadvertently let down.

    In any case, collecting anything from Internet Explorer users is only going to get you as far – anyone who is at least a bit tech-savvy and their grandma use some other browser. Thus, at the end of the day, you end up with the input of but a specific demographic group. The question is, do you really want to learn from or tailor to that group?

    Finally, thumbs down for the “Suggested Sites” policy. Such a shame :(

  57. Martin P

    I think Google did a very lousy job at their “test”. There was no control group. They also set it up so the phrase was always the first result –I suspect had you clicked on a sixth ranked result, for example, on Bing it would have been the first result (i.e. not copying your data). You also did not test against other search providers Blekko or Dudkduckgo, for example.

    Anyhow, Daniel Eran Dilger (Roughly Drafted) outlines some pretty good examples of hypocrisy in your crying wolf. http://goo.gl/SBzW4

    This stuff of your thinking they’re good engineers while then on the next line you smear them with shoddy data is indicative more of Google than Bing. If you want to convince people Bing are riding your coattails, prove it with good data, not some mock up.

  58. Bob

    @Matt Cutts, so you’ve used Google’s resources (20 engineers + 20 laptops) to prove that Bing Bar works. Congratulations on your great discovery, sir.

  59. If they are copying Google results than either they have a search engine without homework or they just wanted to get in the market. Either the case i dont think its a good practice.

  60. Hi Matt,

    While, I understand that the search result was generated by Google, once it is sent to the user, it is the user’s intelligence which is determining which result is useful. In many cases the amount of user intelligence is minimal, which is exploited by people with Web Spam, but in other cases it cannot be discounted. It is really nothing more than somebody reading a text book and paraphrasing. Can the text book owner claim copyright over the knowledge gained?

    The second part of Bing not informing the user about the clickstream being sent to its own service. I do not agree with Bing’s practice, but at the same time do not think Google can use this as an argument. Most normal users dont have any inkling that their searches are being tracked or every click on a search result is being sent back to Google. Does Google use a “Big Red Text” to indicate this on the homepage? No it doesn’t.

    Cheers!

  61. paul

    I’m really sorry, this is childish and has to stop. Even if Microsoft has an special adapter for google within its clickstream analysis, so what.

    And about the privacy aspect of things: Which company has this thing called Analytics which runs on about every site in the web.

    Don’t get me wrong, I love and use Google Search, Mail etc. but this for me is pure, bad, schoolyard fingerpointing.

  62. Plagiarism is copying somebody else’s work and not giving them credit. Back to shoveling snow…

  63. Ed

    As others have mentioned, Google really doesn’t have a reason to complain. In one way or another, they’ve used other’s “intelligence” for their own gain in the past.
    The reasoning that bing shouldn’t have added the results it was “fed”, because it’s 1/1000 of the input data, is just idiotic. It seems to me that bing showing search results for those terms is actually pretty damn smart. It noticed multiple “hits” for those terms and added it them to it’s index. Unless I’m mistaken, that’s exactly what a search engine is supposed to do.

  64. Matt, I’ve got to congratulate you on your civility. If it had been me, I would have reiterated the results that you got and called Mehdi a baldfaced liar. As for the charge that Google was “whiny” to complain about Microsoft’s actions, that was precisely the same charge that Microsoft and its supporters made against Netscape in the 1990s. Subsequently, two U.S. Federal courts and the European Union found that Netscape’s charges were completely substantiated, and that Microsoft was acting as a monopolist against the interests of consumers and in violation of several laws.

  65. PVRK

    The tpoic of the discussion is supposed be search SPAM and content farms with Google ads. There can 100 bings trying to copy you. It always happens in other businesnesses. First stop trying act smart answer real questions about almost useless goolge search results now a days and how google is profiting from spam

  66. Bill S

    Arguments brought in by people that deny the legitimacy of Google’s concerns create a lot of noise around the clickstream and the signals and scraping vs copying vs capturing, etc. All of that is really meaningless. Consider the inputs and outputs of this experiment: the honeypot was fair and it did catch Bing red-handed. The honeypot would NOT have worked IF and ONLY IF Bing did not use Google’s search results in its own ranking. (You can attack “use” vs “copy” vs “scrape” vs “using anonymous data”: it doesn’t matter).

    * You cannot remove Google from in-between the honeypot query and the result, no matter how you try. Period. Full Stop. *

    (And please, don’t grind your personal axe with either Google or MS: who got better or worse and who “stole” what idea, etc., there is a very specific issue at hand with very specific set of evidence; deflecting the conversation elsewhere removes credibility from your position. And please, saying Matt’s words are “laced with venom…”: do re-read your own comment!)

  67. p

    Here is a suggestion – Run a test which would create a link to a crazy website on a legitimate search word. Take an example – Search for pizza, but send results for cars, say Toyota on the first page. Now click on that… You an repeat this process from say 5 users. Now Bing for pizza to see if Toyota comes up on that page. If it does, then you know the weight of Google result. If it does not, stop Whining!

  68. Brad Brening

    Unless Google can repeat this experiment without first clicking through the Google search result while usung the Bing toolbar, there is no issue here. I would expect for Bing to make use of their clickstream data, from wherever derived. And for Google to be attempting to call out another company on data collection and privacy issues seems to me to be at the height of hypocracy..

  69. Ken Aston

    It would be very interesting to see a summary of the results for all of the apparently 100 search terms that Google used for this test.

  70. Lee

    First off, who is Blekko and why do I need another search engine. However I agree with their idea of blocking out entire sites, until they decide to block mine. I guess if Google blocked entire domains, it would just be another law suit. But it did make sense that if you know a site is spamming, let a person block it, as a person has to write the code to do it anyway. Or manually block the site, than write the code to block any new ones.

    I tried to block sites when that option was available on the Google search page [last year?], but I don’t think that lasted but a few months. Sounds like it’s coming back in Chrome, great but I also use IE and Firefox too.

    I also think that some of the problem with spam is due to Adsense as they indicated. Each new site I find copying one of my pages, is just an Adsense site pulling in data from all over the internet. I don’t find many [with my content] that have any original content. I’m an Adsense user too, but it just seems that many of these ‘newer’ sites are generated just to run adsense. Do they show high in the SERPs maybe not, but the ones taking my graphics serve them right off my server, using my bandwidth.

    A standard quality test seems like a good idea. Have a bunch of college kids search a randomly chosen key word once a year on each of the different search engines ~ what could it hurt…

    I always use Google for text searches, but, starting mid last year, I use Bing for pic searches [seems faster]. I don’t use any other search engines.

  71. Paul

    Matt, you seem to think that “respect,” whatever that means, is more important than delivering the best possible results to search engine users. I would hope that both Microsoft and Google use every means at their disposal to sort through the content farms and scrapers and other chaff to determine what real users find most valuable for every conceivable query. A week after Demand Media’s IPO, surely that’s more important than ever.

    Microsoft has a Web browser, and that Web browser has a search box. A user types a query into the search box, and selects, say, the sixth result on the resulting SERP. In fact, let’s say ten thousand users all over the English-speaking world type exactly the same query into the search box over some period of time, and fully a third of them select that exact result, no matter where it appears on the SERP. That’s valuable information! I don’t care if the search engine is Google or Bing or anyone else. If the information is there for the taking, take it and use it to help you cut through the crap for the benefit of your users.

    Peter Kasting, above, says Chrome does not perform this kind of clickstream tracking. I’m stunned and disappointed to learn that this is true–although it might help explain the general slide in the quality of Google’s search results a lot of people have been noticing lately.

    Google is still the only search engine I ever use, but it won’t be for long if you guys keep choosing to adopt a prissy attitude about your competitors instead of using the information at your disposal to give us the best possible search results, period.

  72. Matt,

    I enjoy the webmaster videos and blog posts and do hope that you continue to increase those as a result of this conversion.

    Also, I think it is comical to see all of the results that now appear for “mbzrxpgjys.” I thought that I would be clever and write a funny post about it. :-P Apparantly, a couple thousand people were a head of me. lol

  73. Matt,

    Thanks for the thoughtful post. I wrote some more from one Bing developer’s perspective at http://t.co/RsNA8xE

    Also, as a brief note, no one (as far as I know) claimed that the clickstream was 1/1000 of the weight of any particular model — this was just the common shorthand (used by you, I think, as well) to say that our models have lots of signals and features, and that it’s tricky to do credit assignment. I discussed this in my first blog post (http://t.co/RIlGYo8) and this one as well.

  74. Im new to building sites and how Search Engines Index!
    But it has always seemed to me that Google Indexes my sites first then I appear on other search engines later and slower! (therefor I have always assumed that other engines use Google’s results).

    Due to this, Google will always be #1

    Great site BTW Matt, :)

    PokerGuy

  75. Kevin

    Matt,
    I’m on your side here, but the point’s been made and it’s time to move on. Anyone who knows search knows that Google is leagues above Bing in terms of relevancy, it’s no contest. But you guys are 1990s Microsoft now. Blowing the whistle after the foul on the ‘little guy’ isn’t going to help you out.

  76. What @MarkKB said. His comments and criticisms were spot on and I really hope you address them if you address the various comments and concerns in this thread. I also can’t help but feel that this isn’t about what this is about … and I wonder if/when we’ll find out what the real issue with Bing is. You’re the leader in search quality and market share, there’s no obvious need to create controversy, so what is this really about? I’d hate to learn it’s just about needing a Q score lift.

  77. Excellent, but still circumstantial evidence. Still, imitation is the best flattery. We love you Google, that is why Bing wants to be like you! :D

  78. Is using Twitter firehose cheating? In a black and white world, were using calculators in a test is considered cheating, then using Twitter firehose is cheating. Everybody knows that Google’s search results have suffered due to spam, content farms, black hat SEOs, social media marketers, trolls, and gypsies. It is also know that Google uses a lot of different data points to improve their search results, not just crawling from a href to a href. Google has tracking information on users, from every side of the click. Google often has and collects information when a user clicks a link on its search result page and on the visited page (if that site uses any of Google products such as Analytics or Adsense). Google is sitting pretty collecting data from every angle, because it has the market share to do so and tells competitors “No Soup for You.” The orchestrated “synthetic” outrage from Google and associated Bing sting borders into monopolistic behavior.

  79. Matt

    here’s the microsoft fud at it’s core:

    Google copied Overture (AdWords) – they made their own
    Google copied Twitter (Buzz) – it failed and sucks
    Google copied Flickr (Picassa) – they made their own and via acquisitions
    Google copied iPhone (Android) – this is actually open source and not closed like iphone
    Google copied MapQuest (Maps) – they made their own and improved on it substantially – you think mapquest had live gps and turn by turn?
    Google copied Yahoo (Finance) – they made their own, and it’s different from yahoo, and it’s from public data. anyone can do that.
    Google copied Alta Viata (Translate) – are you joking? Google has almost made a universal voice translator. has alta vista done that?
    Google copied AIM/ICQ (Chat) – it’s wrong for other people to be able to make chat programs?
    Google copied Friendster (Orkut) – I don’t know about this one.

    I could go on, but you get the idea. Glass houses (misinformation by someone pro MS), an so on.

    fixed up the BS reply with reality.

  80. Rob

    For a company which makes most of its money unethically (google) with all the click fraud, search farms, squatters etc, this is like a pot calling the kettle black.

  81. As an SEO I would like to express an extreme distaste with MSN’s general manor of adressing this whole situation. Rather than concede to the plain truth they have two representatives contradict each other and completely lose face in my eyes. He sounded like the average pushy salesman ducking questions and answering questions with questions and placing blame by ephasizing and obscuring facts like saying Google uses click data so why can’t we. Yeah data from their own engine where their inovative alto delivered the results in the first place. They once again prove that they will use their hold in the OS world to decieve users and manipulate an industry with underhanded and sneaky tactics that lack original thought and inovation. Can someone say GUI. As reference I provide a page that Blekko might manually dismiss because of user feedback. http://lowendmac.com/orchard/06/apple-vs-microsoft.html :-)
    Thanks Matt!!

  82. I really can’t even believe you continue to bring this up.

    You engaged in a honeypot test that basically amounted to manually affecting your own search results through deliberate and coordinated click fraud. You then picked the longest of long tails and still could only achieve a 7-9% detection rate with no explanation as to why 91-93% of your long tail tests failed. It’s really that simple and everything else is the PR machines of two huge companies spilling out over the public web.

    Google does not own user behavior, they do not own click behavior – stop acting and talking like you do. If you want to “own” these behaviors, then continue to improve on your great Chrome browser, great Android mobile platform and get users to stop using Internet Explorer. You make great products, your spelling suggestion technology is top notch, but your company continues to make blunder after blunder when communicating with the public.

  83. Katrina

    A tad hypocritical. Google certainly takes a fair chunk of everyone’s content under fair use and makes money off of it without sending traffic necessarily. Pot Kettle Black.

  84. WB

    MATT WHY DON’T YOU GET TO FUCK.
    Seriously all this bs just told me that BING matters

    Google – You shot yourself in the foot.

  85. Mike Bell

    Funny how Google cries when they’re being copied and yet they copy Apple every chance they get.

    Google copied iPhone and iOS.

    Google even copied black MacBook with their Cr-48.

    Hey Cutts, why aren’t you covering that?!

  86. I think this is always happens, when you are on the top others try to copy you to be in the competiton. If you are providing top services than others will also interested to know how you will doing this and they try the same tricks.

  87. rl

    Strange but today i observed few results beeing :
    - indexed by google , bing 0.
    - cached by google and few minutes later indexed on bing.
    even if you add heavy visited url to bing, surf with tollbars, it won´t show up until google have cached this sites.

    strange

  88. jojomonkey

    It’s nice to see engineers w/ bigger balls and brains than most politicans. Cool debate/discussion!

    I personally think Google does above and beyond w/r/t providing info. to the public/industry.
    And I don’t think Bing’s tactics are that big of a deal.

    Google could actually screw Bing over by creating junk searches and letting Bing copy it over but serving up legitimate users w/ correct searches.

  89. I think 99.9% of people would see this as a pretty compelling case for copying – what was toted as the future of search discussion seems to have turned into another Google PR machine though, and a MS own goal – a conspiracy theorist would think that all the noise in the build up to this had been pushed by Google (knowing that they were going to get fingered [unfairly] as the source of all spam) with the intention of shifting the story and leaving MS with egg on their face as publicly as possible ;-)

  90. Just Trying

    Since the comments are not showing up. Just trying if Matt just stopped taking the comments, given that not many people are buying his argument.

  91. Aaron Lauper

    Seems Google is asking Bing to reveal their signals without google also revealing theirs. Is that a fair request? Same goes for Google requesting information on browser / toolbar usage without also sharing their own. Seems the stream of openness should go both ways here.

  92. Cody

    I find this event / “debate” to be very disappointing. While I’m not a fan of many things MSFT has done or their policies, I think that this particular effort to attack them is misleading and manipulative.

    I agree with your point that MS needs to be more honest and open about using browsing data and how it’s being used. I agree that they aren’t handling this issue quite as well as they could; but they never do…They are kinda like the geeky kid running for class treasurer being made fun of during a speech, they just don’t know how to handle public attacks of this kind. As an aside, if privacy issues are REALLY what you care about, perhaps your time is better spent calling out all of the iPhone/Android app developers who are harvesting and selling much more identifying and personal data than MSFT has been (at least MS puts it into their EULAs).

    Here’s the problem…The claim being made here is that search terms that point to already highly ranked websites are appearing in Bing results and that must mean they are copying Google’s results. First flaw, the presentation anybody has made doesn’t actually amount to the end claim; meaning, there’s not actually a correlation between the “evidence” and the accusation. Second, these are manufactured terms that don’t exist in the wild, meaning no competition, but that issue has been bandied back and forth plenty and not worth re-discussing. Third, it’s falsified data being fed into MS’s servers, which some could argue is really shady for other reasons, but I’m only going to reference it in the context of incorrect data rather than maliciously-formed data. Fourth, the sites being linked to are pretty highly ranked. The thing common sense begs from these points, is there any reason Bing should NOT have indexed them beyond the fact the words don’t have meaning? From a clearly definable logic, if somebody uses a given search term and goes to a website, it’s reasonable that somebody else might use that term to also go to the same site. There are a lot of other ways to argue the same thing, but ALL of the perspectives I’ve read and anything I’ve been able to come up with equal the same thing, this is no different from the infamous “Google Bomb”.

    The thing that bothers me most, this is NOT THE FIRST TIME that this flaw was exposed in Bing. I read about a guy pushing bogus search results into Bing with data collected from user activity about a year ago, but he didn’t even plant faked data inside another search engine first. That difference actually tells me that either MS only closed part of the hole, or somebody at Google is more interested in fabricating a story and needed to create this connection.

    My point…a reasonable person can see this event as somebody at Google exploiting a known bug in Bing’s ranking system by feeding it data in a way that can’t be properly examined or proven, with a LOT of information being left out or heavily misrepresented. Instead of having a good and honest laugh at pranking Microsoft, Google is turning this into a witch hunt.

  93. Glad I don’t use IE! Chrome 9 FTW!

  94. Brent

    After reading MarkKB’s excellent rebuttal, I’d say we’re done here. Please drop this Google, and don’t add to your embarassment.

  95. i watched the video and read many of the recent blogs about search issues.

    here’s a summary of my thoughts and impressions.

    • Matt is a great speaker, and Rich Skrenta too.

    • Vivek Wadhwa and Harry Shum would greatly benefit if they take some effort to improve their English accent (if they intent to do a lot public speaking)

    • Harry Shum seem a little pushy in this presentation. (certainly, Matt’s been a bit accusative and defensive too)

    • Google really shouldn’t disclose anything about their algorithm. I do not think others have a right to claim Google should. Personally, i’d even go as far to suggest that Google shut down their webmaster blogs and videos, which often are pure discussions about how Google search works. I do not see any need at all for Google to discuss how their search engine works. I do not agree with any of Harry’s point about how “as industry leader, Google should disclose their algorithm” (or something close to this sentiment). As Matt said, Google disclosed far plenty already. I actually held Google responsible for creating and fueling on the entire SEO gaming market.

    • Matt has always been great, but this recent incidence about MS copying, i felt that Matt/Google is really whining a big deal. Am not impressed by all these evidence Google put forth about how Bing is copying Google. I do not see MS have done anything unethical about using bing bar to get data for what user searches under Google. Even the MS paper matt mentioned as a evidence of MS’s little “reverse engineering”, i do not see anything unethical about that. What MS has done is not stealing, nor plagiarism, in any sense of these words. As far as i can see, what MS have done as presented by Google is normal competitive activity in software industry.

    • there are lots heat on Google about how Google isn’t blacklisting content farms due to profitting from them. Personally, i still have faith in Google and believe that Google’s action on content farms isn’t that much effected by financial gain, short term or long term. Ι certainly think that money do indeed effect Google’s behavior and decisions, consciously or unconsciously, but not enough for the public to blame Google as being greedy or unethical, at this point. Of all things considered, of all practices of big software companies out there, i’d still rank Google as number 1 in “do no evil”.

    • it is possible that Google is hinting that MS is actually copying Google search results in some massive scale (with or without Bing toolbar). If that, then i think it’s a problem. But i think so far Google isn’t making that claim.

    • i learned about Blekko, their interesting approach, as well many other things from this video. Thanks a lot guys.

    Xah

  96. Seth Johnson

    Google beat out Yahoo and other previous search engines (Webcrawler, etc.) because they provided a better user experience, by delivering more relevant and desirable results. Wadhwa and the other panelists tried to push some academically defined metric to compete on, but you were right, the user experience is the ultimate defining metric (although admittedly Google does have consumer habit and presence going for it as well). I know that point of competition is not the “hot point” of the debate but I at least wanted to bring it up.

    As for the copying aspect, Microsoft’s strongest argument is that it has as much right to user behavior as anyone. Also, they have the argument that it uses its dominant browser presence (i.e. IE) to further its competitiveness in the search engine area the same way Google likely users is strength of presence in other areas (such as video sharing, search listings, etc.) to take strategic actions to strengthen its position in other areas (like Google product features sponsored results, etc.)

    Anyway, watching the whole video, props to you for coming off as collected and straightforward. Google sent the right guy. Unfortunately, Shum came across as accusatory and condescending. As for Skrenta, well…how could you not like Rich?

  97. “Do you have any evidence that the Suggested Sites feature sends this information back to Microsoft? I ran some packet captures and they didn’t seem to show that.”

    Angus: yes, we do have that evidence. I believe you’re looking in the wrong process. Also, the process sends the data in a batch to Microsoft periodically, so you won’t see it as you click from result to result.

  98. Gator

    In reply to Xah
    “Vivek Wadhwa and Harry Shum would greatly benefit if they take some effort to improve their English accent (if they intent to do a lot public speaking)”

    This comment of yours makes me feel that the world would greatly benefit if there are a few xenophobic and narrow minded people like you who assume that American/British accent is the only accent out there.

  99. So if Google applies a penalty to Bing, and Bing supposedly copies Google, will that create an infinite loop of zero rankings?

    Will you reply to each other’s re-inclusion requests? LOL

  100. Mike

    I’m the first one to call Google out when I disagree with their actions (hello, net neutrality?), but I also believe firmly in speaking out when companies do the right thing, or are in the right. Matt, you did an excellent job in a very awkward situation, with, in my opinion, an interviewer (Vivek Wadhwa) that came off as neither fair nor polished. Well done.

    Harry Shum came off to me sounding like a used car salesman.

  101. Spem

    No expert here but I do wonder about the following,

    you used the Bing toolbar to inject “signals” into Microsoft’s search results. I have to assume these result could not get into their dataset (or whatever it is called) in any other way.

    If you would than search for those results isn’t it obvious you would get the exact copy of what you injected in their system?

  102. Matt,

    Let’s assume the following:

    Domain.com/page.html receives 1000 visits per month via ‘XYZ’ keyword. 100 of those visitors spend an average of 1 min on page.html and they end up bookmarking the page in Chrome. Would Google use each of those bookmarks as a ranking signal?

  103. Dan

    I will always be on the google side and don’t even log into bing – and what i feel is what bing is doing is ethically wrong and they shouldn’t be doing that but if you look from their side – they don’t have any options left to get more market share in search – and they are just pissed off on how google is doing that :) so it’s better to copy them rather then building something which they are not capable of :)

  104. Hi Matt, I believe that saying that “Bing uses [or even "copies"] Google search results” is utterly misleading, since what your spam test proved is that Bing uses IE opt-in clickstream data for ranking web pages:

    We asked these engineers to enter the synthetic queries into the search box on the Google home page, and click on the results, i.e., the results we inserted.

  105. I personally think Microsoft, or Harry here is trying to shift the weight and exchange concepts. It’s not about who owns the data or whether you have the right to collect the data, it’s about what you do with the data. In a nut shell, if Google’s data is weight this much in Bings algo to have produced this similarity in search result, Bing IS copying Google. I mean if they keep doing this (by getting the taste of candy and lured on), there is going to be not much difference between Google and Bing’s results. What good does this do?

    Is Bing expecting to share the market with Google 50:50 by doing this (no matter how stupid this may sound, it might work in that direction)? I mean if we swap the position between Google and Bing, what is Bing going to say today?

    I am Chinese, and I really hope Harry has nothing to do with this cheating strategy.

  106. João Pinto

    Matt,

    Despite your claims of respect and consideration for MS engineers, your allegation that they are copying google’s results is abusive and disrespectful.

    Google indexes unauthorized copies of thousands other’s valuable creations (web contents) and you call it “searching”, someone else indexes data resulting from Google’s own creation (search results), and you call it “copying”.

    You raise ethical questions on how MS collects data using their browser without *properly* informing the user. How does that differ from the data you collect from users when they visit a Google Analytics site ?
    Do you really believe that you do a better job informing users on how you collect data from them when they reach a GA enabled site ?

    You do a respectful work, so please be respectful and considerable about other’s work too, disagreement about data indexing methods and values is not a strong reason for such bold claims as you have done.

  107. Interesting stuff, we’ve been following the debate since it began. Doesn’t it seem obvious that a new algoritm would take something from other well worked ones? Since they can’t copy directly, they work in parts that they do know about. I was under the impression that Google basically copied and updated another algoritm in order to create the YouTube search? If there were other search engines that did as good a job as Google, i’m sure Bing would work indicators in from them as well.
    I think it should be taken as a backhanded compliment, and I look forward to the rest of the debate!

  108. Although it is obvious that Google has got a stronger and valid position against Microsoft. But I would like to share the article from Aron Wall, on who is copying who:
    http://www.seobook.com/google-vs-bing

  109. Chris

    I though Google deliberately sent this data to Bing by making a search via Internet Explorer with site suggest turned on. This sends the web sites a user browses to Microsoft for them to be able to build up a suggestion list, and in turn this is how Bing received the “search term” used and “link clicked” to get to the non-relevant web page.

    Based on the fact that it was the only result that came back for the nonsense search term, because the sites being linked to didn’t relate to the search term, it is no wonder the results are the same, as Bings algorithm had A) no other point of reference, other than a user clicked a link using a search term, and B) that no other result matched that search term.

    I don’t see how any copying was done. Don’t all search indexes have to start from somewhere – a hyperlink to a web site, for instance?

  110. Hi Matt,
    the screen shots prove your point!
    However, I am not I am convinced that Bing copies the results.
    Its looks link Microsoft is using IE + other “obscure” tools to gather information. Since none of their other algorithm parameters helped, the information they gathered, by spying on web users, got them this results.

    Microsoft is using IE and other “obscure” tools to know where and what we do. This is disturbing, even more than Bing copying Google results.

  111. Bob

    I’m glad I found your blog, Matt.

    Google the product is a great search engine. Its power and performance leaves Bing in the dust. Google the company, on the other hand, has reached a new low with this week’s PR campaign.

    I hope you or another reader will correct any misconceptions I have of this event: After you opted in to the Bing toolbar data-sharing program, you invented some alien, sterile words that had never appeared anywhere in known human history. Then you used opted-in Bing toolbars to expose each of these words to Bing, for the first time ever, along with a single URL disguised as a search result. Aside from this artificial user experience data, you knew that Bing would have zero information about any of these search terms.

    After repeatedly sending this faked user-experience data to Bing, you queried it for each of your alien terms. You found that, for a small minority of these terms, Bing returned the pages that you had fed it.

    You thus proved that Bing uses information from opted-in toolbar users as a factor in its search index. Of course, the only way you could easily demonstrate this effect was with cases in which Bing had none of all the other sources of information it might ordinarily be expected to have about a search term. Even then, you could only demonstrate this effect with a minority of your alien terms, demonstrating that Bing usually attaches no discernable value to such isolated, and thus questionable, information.

    At this point, you deserved some credit for documenting a clever, although unquantified, use of information on Bing’s part. Probably the most scandalous inference that a normally logical person might make is that Bing’s BS filters aren’t perfect, since they worked only about 90% of the time. But that wasn’t nearly enough for you. You made a huge deductive leap and said you had proven that Bing copies Google.

    I won’t bother to rehash the way you released all of this to the world, except to note that you did so through a blogger. And then you sat by while the blogger, followed by many news organizations, proclaimed that Bing copies Google search results. You didn’t bother correcting those who implied Bing had crawled Google search pages, or might even be scraping Google searches in real time. No, you just sat by and let the ignorance flow.

    If I may, let me suggest a few studies that could have provide a small context of reality to your own. You could have checked your own server logs to see if your bogus search pages had been spidered by the Bing bot, or had even been accessed by anyone other than your testers. Hmmm, something tells me you did this but didn’t mention that you hadn’t found anything. You wouldn’t want to give away any actual facts that might counter the now-common misinterpretations about just how Bing “copies” Google.

    You might have generated Google result sets with hundreds of fake hits on your fake search terms, clicked repeatedly on a single low-rated link, and then checked to see if this link appeared at the top, or all alone, on subsequent Bing searches. Well, that wouldn’t make for persuasive, easy-to-spin screen shots, would it? And you’d have to explain why Bing hadn’t “copied” Google’s top-ranked bogus results.

    Similarly, you might have used genuine but extremely uncommon words. Using many opted-in toolbar-enabled browser sessions, your testers could have searched the non-juiced Google, clicking on the same low-ranked result. Then you could have analyzed before-and-after Bing search results for any discernable influence on their rankings. But then you’d have to deal with noisy data, and might have to acknowledge that perhaps Bing uses factors in addition to its “stolen” Google results. That would be way too hard to explain, so forget that.

    You might have confidentially asked professional colleagues at other search companies to repeat your experiment with their own engines. But their own positive results wouldn’t help you imply that Microsoft specifically copies Google.

    In brief, you declined to obtain — or at least to publicize — any information that might detract from your message of an evil Microsoft stealing from a defenseless Google. This, as much as your grand leap to the conclusion that Bing copies Google, shows that your “experiment” is little more than a staged publicity stunt.

    I keep asking myself why I should care enough about this episode to spend hours thinking and writing about it. Then I realize that I’m hugely disappointed in the behavior of a company whose product, back in the nineties, I encouraged hundreds of people to try. And, as much as I disdain Microsoft for its characteristically miserable business ethics, it is sad to see Google stooping to their level.

  112. Antonio

    Don’t forget that this experiment was ran under the condition that the user has the separately downloadable Bing Toolbar add on with the search optimization feature enabled. This brings more targeted search results based on what the user searches for. Only under this circumstance is there a 9% chance of Bing copying Gibberish results that it has no results for (namely because the assemblage of letters means nothing). This has nothing to do with the stand-alone Bing search engine. Lets remember, only a 9% success rate, even under these strict conditions where a normal user would most likely not find themselves.

    If Google wants to re-do this experiemnt, sans the independently downloadable Bing Toolbar, then they would have a story. Seeing that they could only get a 9% success rate, that test would most likely end with a higher failrate (Whcih would be a 91% failrate = A- in failing).

    I love how people forget to add all of that “other” information into the equation. It’s very depressing for a general user and advocate of Googles applications to see this company come to such blackhat pr tactics. Regardless of this i will still be a Google advocate as they have given me the tools and tech to delve into an area i so much love. SEO! I am just very dissapointed right now.

  113. In essence, Microsoft/Bing management is saying to their search engine development team
    ‘We don’t have faith in your ability to build a great search engine, [independent of Google's lead]‘ #ouch

  114. RL

    @CmsBuffet
    I think google don´t really worry about that, if microsoft clearly declares to all clients :
    “suggested blabla is powered by your own google search history, clickstream or what else we use”. Bing powered 98% by google or something like that.

  115. Matt, one highly probative element seems missing, either from the test of these unique tokens, or from reports about the testing. I may be misreading the reports, or missing something fundamental — and, sorry if it’s explained in the video, but without a transcript that video might as well not exist. (I don’t have forty minutes to watch something I could read in four.) In any case, if I missed something so basic, many other people will, too.

    There appear to be valid inbound links to all the sites where unique tokens were seeded. This affords Bing effective plausible denial: they could have spidered their way to every page. If Google’s test had seeded tokens on pages with no inbound links, Bing’s discovery of such “orphan” pages would be conclusive of their “borrowing” Google’s results.

    In fact, the tokens would matter little: if Bing indexed a page known within Google results but which could not be discovered in any other way, such proof would be definitive. That might have been buttressed by embedding tokens in the URL — to assure no defense that “some unnamed person using MSIE” randomly entered that URL.

    I’ve read several articles (from Danny’s release through this entire thread) and have not seen this mentioned, or at least not clearly. If it’s implied or stated, sorry — somehow, I missed it.

    And we’re still gonna deal with this whole violation of Googlewhack policy thang! ;-)

    Gary Stock

  116. Q

    I am not a fan of Microsoft but they do make some very nice products.
    I am wondering does any one have a list of these over 1000 input that they’re using as defense?
    Wouldn’t surprise me one bit if they couldn’t even come up with 100.

  117. Hi Matt,

    Great discussion there. I particularly enjoyed the way you jumped right into the issue and kept referring to the issue quite periodically in comparison to the rest of the panel who didn’t discuss much about it. It is interesting to see how Harry Shum went on the defensive and raised the tempo a little at times. It gives away a lot about how you could stand to be vindicated.

    Secondly, on the issue of web spam. I don’t really understand why the panel couldn’t agree with you when you mentioned about doing the filtering process algorithmically. Given the scale of the number of webpages in Google’s index, I don’t really see an alternative to algorithms if every webpage on the web has to be given a fair chance of being ranked. I think it will help in preventing webpages or sites from being classified as spam incorrectly. Doing it manually like Blekko.com is a challenge with limitations to it. It will also put a lot of dependency on the individuals who do the filtering. Something that person A classifies as spam may not be the same way person B might look at it. Plus there is also the task of defining and maintaining standards to the manual filtering process. This in turn becomes difficult as the trend of spam may keep changing. I agree with you on the algorithmic approach to tackle this problem.

    Great post by the way. It throws light on a lot of things that web users may generally overlook.

    Cheers,

  118. Bob

    Gary,

    I don’t see how either a positive or negative result on this test would help support, or cast doubt on, Google’s claim that Bing copies their search results.

    A negative result would not show that Bing doesn’t capture information from Google searchers; it would only suggest that Bing isn’t dumb enough to put a page that doesn’t exist in its index.

    A positive result would not add newly probative evidence to Google’s claim, which, by the way, no one is even disputing: that Bing captures information on relationships between search terms used in a query and the links that the putative searcher chose to view.

    However, even I must admit that if it were shown that Bing was directing its search users to non-existent pages, based only on Google’s say-so, that would get my attention, and I’d be waving pitchforks at Microsoft along with the best of them.

  119. Happy Lunar New Year

    This is a very asymmetrical response to Bing’s 3% marketshare growth since re-launch. I’m loath to imagine what/how Google search would respond to another year of equal growth from Bing and another year of Google Search stagnation. Quite some re-direction at a very convenient time.

  120. Leigh Jones

    I’d have thought that any “Bing copying” would more likely center around Google search results for perhaps the top 100 or so search targets. It wouldn’t seem likely that Bing intentionally searched for “hiybbiqaj” on Google and retained the results in case someone requested a search on Bing for that target. The number of retained searches required to provide comprehensive coverage of pseudorandom search targets would be prohibitive.

    So, what you’re suggesting can only be explained with live search redirects. But, then, the descriptive text attached to the search result link differs here. That would mean that Bing would have to perform a search on Google, then capture the linked web page and summarize its contents. Search results on Bing would take prohibitively long if that were occurring.

    Bing, of course, is supposed to be running the Yahoo.com search engine — a well equipped database with long traditions. I know a lot of people that use Yahoo.com habitually and swear by it vigorously. The very notion that Yahoo.com might be copying from Google really difficult to swallow; Yahoo prides itself in providing “filtered” search results that are more highly likely to satisfy than their competitions results. But, then, if Bing were to rely on Google results whenever the Yahoo.com engine yielded no results, then I think that might begin to explain your observations.

    Or, alternately, “hiybbiqaj” might actually be found in the web page.

  121. It clearly looks like Bing is incorporating some aspects of Google’s search results into their algo; what I’m not sure about is why Google is so upset… are there really any secrets on the internet? I suppose Bing could have been more upfront about the route they’ve chosen, but we all know what’s going on, an outright admission isn’t going to make a lot of difference at this point. Google engineers have to walking pretty tall right about now, knowing the competition has resorted to ‘copying’ them – yikes!

  122. Brian R

    Too bad Vivek cut Matt off and didn’t let him respond to all the misleading jargon that was coming out of Harry’s mouth. I haven’t watched the whole video but I think this topic deserved a lot more than 10 minutes.

  123. Duplicate content eh ;)

    Maybe I have just been super-unlucky, but on the few occasions I tried to find stuff using bing, it was too inferior to Google, for me to use.

    Whilst it’s not uncommon for people to try and emulate an industry leader, this looks a little curious to me. It’s also got to be pretty embarrassing for Microsoft.

    Thanks for giving this issue some oxygen.

  124. Michael

    “Did the Google research piece/sting actually click on the results or did they simply do the search?”

    This is the only thing that matters. If they clicked on the page, no problem. If they didn’t click but merely did the google search, then Bing is copying results.

  125. Check out my recent post on YouMoz/SEOMoz called: Why Bing aren’t copying Google’s search results: http://www.seomoz.org/ugc/the-bing-sting-facts-why-bing-arent-copying-google-11924

  126. I think the reason why the Microsoft algorithm copies Google on the extreme long-tail search, such as mbzrxpgjys, is that there are no other signals except for the one from Google. Where in most cases, the Google signal would only account for 1/1000th of a point, for mbzrxpgjys it accounts for 100% as it is the only data available for that term. Therefore it looks like a complete copy, when there is a “justifiable and logical” reason for it. Now, what Microsoft should do is to adjust that algorithm so that it has to have at least two signals to be valid.

  127. Here I differ

    >>First, not all of the inputs are necessarily equal. It could be, for instance, that the Google results are weighted so heavily that they are as important as the other 999 inputs combined.

    The first point does not seem valid. In provided screen shots, effect is only happening for search queries resulting into only one match, this reflects either other inputs didn’t have any say in generating results (i.e., only one input had a say, lets consider for ease that being Google) or more than one input concluded single match (including Google’s input, may be). So the first point’s assertion seems over stretched.

    >> Second, it is problematic for Microsoft to describe Google results as just one of many “signals and features”. Google results are not any ordinary kind of input; instead, they are more of a finished (albeit ever-evolving) product

    Even if someone is using finished product then why does it seem as a problem, is Machine learning not applied on raw data (without being mined scientifically) instead of only information (mined data).

    >> “We do not copy results from any of our competitors. Period. Full stop.”

    This sentence is still valid as its not copying but only one input, can you generate many results with similar ordering (i.e., ranking) between two search engines, if not then the above sentence is valid.

  128. MarkKB nailed it, “20 Google engineers submitted this data to Bing.”

    All it proves is Google successfully decoded 1 signal Bing uses to rank system. Congrats – there you have it – Google just practiced SEO.

    Now, if Google can show me a search with a real keyword and a minimum of 1000 results, and then this story will have some validity relevance. Until then, this is just public whining about a competitor.

  129. Thanks for posting the 7 screen shots. Reports were never really clear on the number of actual results were found to be exact and mostly put it between 7-9 out of 100 tested. Personally, I think it’s bad reporting on Google to release such a major statement without providing exact data and the fact that reports claimed to be between 7-9 sounded like they were exactly sure or didn’t even know. Since you posted 7 actual results, I’m going to take this as there were only 7 out of 100 that looked similar.

    1. I’m not sure what happened to the other 1 or 2, or if they ever existed at all, but the fact that the actual article didn’t even know makes it hard to believe and rely on as fact at all. Next time try making it an exact number, rather than guessing or providing vague results that are meant to sound like they are more then they are.

    2. 93% of the results didn’t produced an exact match. This means that it is less likely to find a copied result on Bing than it would be to find one that is so-called copied. Not really selling me on this point at all.

    3. User data is better for users. Google’s model is built on the haves and have-nots instead of what users are doing. If Bing wants to build an engine that takes into account what users are actually doing on the web, and people want to opt in for it, then great for all of us. These wonderful people and Bing are making the search results better for all of us and I say thanks to both Bing and the users for sending in this data.

    4. Manipulating results for even the purpose of doing a test is still manipulating results and spamming the internet. Google created a fake result, which is against their policy, and now opens the door to more speculation that this is going on. Google for a long time said it couldn’t manually insert pages into their Algo, but now it seems that they can. How long has this been going on? This is the first public admission of this happening, but I suspect it has been going on a lot longer than they finally let us know about.

    5. Petty stings makes Google look petty. Is Google so worried about Bing and their results that they will violate their own policy, game users with their fake results, and call out a competitor for something that doesn’t even measure as reasonable amount of data to be proven as true while running the risk of further brand backlash and petty child like squabbling? I guess so. Wow, I didn’t realize Bing was so big and powerful now.

    Way to go Bing for innovating user data, and so sorry that Google didn’t think of it first. Next time maybe Google should try innovating something new instead of stealing their competitors ideas or crying foul when they come up with something that works.

    Go Bing yourself!

  130. Mihir

    Matt,

    Your claim that the IE suggestion dialog does not make it clear that it is sending information to MSFT is good. But then, the Google.com page does the same – the fact that my search history is being anonymously collected is hidden deep in the Google ToS somewhere – why is that not on front page either?

  131. Etienne Dupuis

    @Simon Templar: You are wrong. Google got suspicious because many of they’re rank #1 result was ALSO showing up #1 in bing, with popular queries.

    If you cant beat the big guy, join him.

  132. Did Google opt out of their index?

  133. If you kids can’t play nice I am not going to let you play in my wallet any more!

  134. I think that this really matters because from Google’s point of view, Bing is stealing the work that Google is putting into their algorithm. So searchers on Bing could perform a search and say to themselves… “I like these results, I’m going to keep searching on Bing, they do a nice job.” When what really happened is GOOGLE did a nice job, and they are stealing their mojo in regards to offering up relevant search results to searchers.

  135. m_live

    hehehehe
    It is very important to us to protect our users’ privacy.

    said the guy working for the company of the “privacy nightmares” (Google Buzz the example that comes on top of my mind).

    Talk about Privacy !!!!!!!!!!!!!!

    How does it feel like now, Google ?
    You scan millions of other publishers and authors books to your library and YOU WANT THEM TO COME TO YOU and tell not to publish in your site ? Is it not ILLEGAL – living on other peoples work ?
    You allow Youtube users to upload any video, and YOU WANT MEDIA TO COME TO YOU TO TELL to remove it ? Is it fair ?
    Think people think.
    http://community.nytimes.com/comments/pogue.blogs.nytimes.com/2011/02/02/on-googles-bing-sting/?sort=oldest&offset=2

    People out there who don’t know how Google makes money are IGNORANT or IRRATIONAL”

    Does’nt have a clue about online business… Google gets angry if “its” contents area “copied” although it is their CORE business… Get it ? Take Youtube or Google books or those million blue links. Are you kidding….

  136. Sam

    @Matt,

    what is your response on Danny Sullivan’s new post “Bing: Why Google’s Wrong In Its Accusations”.

    Earlier he said Bing is copying, now he says google is wrong.

  137. Well if everyone else is dropping links in here:

    Bing: Why Google’s Wrong In Its Accusations
    http://searchengineland.com/bing-why-googles-wrong-in-its-accusations-63279

    That’s the follow up to my original story that kicked all this off, from having some long talks with Bing. In it, it covers what Matt was hoping to get clarified — a clear denial from Bing that it is targeting Google URLs specifically. It also looks at the difficulty in coming up with a “Google-free” clickstream, given that Google powers results off its own site.

    It also highlights that I’m still waiting for Google to clarify what exactly its Google Toolbar data contributes, or doesn’t contribute, to search ranking. That also goes to what Matt wrote about above, this entire thing that his mom might get confused that Suggested Sites with IE doesn’t make clear that Google searches might flow to Bing. Well, the Google Toolbar isn’t that much clearer and potentially is doing similar things.

    When you install it, you get told this:

    “PageRank and future page-related services are part of the enhanced Toolbar. For enhanced Toolbar features to work, Toolbar has to tell us what site you’re visiting by sending Google the URL.”

    Nothing in there discloses that the toolbar will be used to gather site speed information which, in turn, influences how sites rank in Google. Nor does it say that your searches on Bing will get sent back to Google — which they will, even if Google says it doesn’t use them.

    By the way, the toolbar doesn’t tell you to say yes or no. It says “Ask Me Later” or “Enable Enhanced Features.”

    So, I’m not feeling like Google’s claiming any higher ground by continuing to highlight IE’s logging tools. What they log, as I covered in my original story, is covered in their disclosures. It’s a waste of time to dwell on that.

  138. Test results are certainly disturbing, but does this happen with significant queries? Not happened to me so far.

  139. D E Peters

    I take quite a few issues with what is said in the video. #1 is that most of Google’s top results are Google properties such as Wikipedia and You Tube. But these are rarely the authority sites that should be shown first in the search results. #2 is that Google’s core business is advertising, not search. That in itself skews Google’s rankings and results in problems such as stated in issue #1. Search is simply a means to an end that Google can use to piggy-back ads on. #3 is that non-original content such as a directory of articles isn’t relevant to a site. Niche sites that concentrate on providing answers to specific questions are much more important than sites that are simply spewing repackaged articles about anything and everything. When you have a writer’s great writing, there is no reason to wreck it by rewriting it. That is plagiarism. In a sense, Google and others do this by collecting books and books. It’s not their work, and yet they pretend it is by branding it as Google Books.

    But instead, by collecting the best related works on a particular subject, a site should be seen as trying to provide the best information available WITHOUT plagiarism existing works.

    I agree, this is an overly simplified analogy, but it is an important and accurate one in order to make a point. You may see the books as products, but as books are commonly associated with Copyright, Google Books becomes an alleged authority when all it does is repackage data and articles the same way any spammer does. To reprint said books written with significant changes might be plagiarism, but the movie industry does this constantly. Examples are common place, “Animal Farm” takes all sorts of liberties with the original book, so how come it stays title as “Animal Farm”?

    I think you guys are missing the point, and just pointing your fingers at each other instead of trying to understand what we need. We want the best resources, not sites that show ads to the websites that pay the most for traffic. This model has worked well for Google, but it is an extremely poor business model. Microsoft’s Bing is copying the same severely flawed model. But once people have a better alternative, these models will show their problems.

    Right now, there are no alternatives. Blekko may seem like one to some, but I have to look at that brand and say that is not a very well thought-out brand. It sounds like “Blech-O” and it is hard to spell with 2 k’s. So there doesn’t seem to be any creative talent behind this brand. And that’s what search is missing. Creativity.

    And instead of banning any spammy site that isn’t malicious, they should be categorized as spam sites at the very bottom of all results (with no shortcut to get top the spam listings). This way the owners, admins and webmasters who look for their sites and don’t see them and wonder why get an immediate alert that their website is in trouble and can then address the “Webmaster’s Tools” section of the related website.

    You all have mighty idealistic ideas of advertising, and no fresh ideas about search. VERY DISAPPOINTING. I’m very discouraged.

    -Doug Peters

  140. live4life

    Every search engine out there copies google in results style.

    Google influence on search is huge and they remain the people to catch. The non advertised main screen has become a legendary theme. The continuous play with logo and the presentation of search is driven by google. The only difference I find with bing is the picture in the background on their home page.

  141. PuppyHunter

    What if Bing uses Google Serps as one of 200 factors in their algo. That one factor could be comprised of 50 different signals (from various engines).

    If I were trying to improve my SE’s results I would start with Google’s and try and improve upon it… cough.. Blekko.

    Let’s explore the most obvious conspiracy theory (@jennings) because over the past few months Google has taken a big hit on quality and called out by many of the top SEO leaders for their spammy results.. i.e. Sullivan, Wall, + others.

    Here’s what could have happened in the underground chambers….

    Google engineer: “We’re screwed, our algo is so convoluted we can no longer create filters to reduce the number of spam sites without hurting legitimate businesses. We’ve tried – it’s not working!”

    Matt Cutts: “We need to wag the dog, buy some time to figure this out”

    Google VP: “Got it covered, check out this little experiment. We spent 5 million over the last few months – I could hardly believe it myself but Bing may be using our results to improve theirs!!”

    Larry: “F’em. Call them out at Bigthink and let’s try and fix it. If we can’t, revert back to Florida and start over”

    What if Google gave click through and bounce rate the weight it does links? We’d see a completely different search landscape, arguably a better one.

  142. Great topic for discussion. Matt, do you feel that Google will move towards utilizing a search algorithm that incorporates a more social aspect to it such as Blekko has done? My fear is that competitors will hire people to mark websites as spam on large scale to get them removed from search results. Having a socially integrated search engine may be the best answer but I feel that having a personalized search is the most efficient and economical.

  143. Isaac Levy

    Yep. It’s also kind of ridiculous how much microsoft is monitoring in IE. Consider the flak that google got for the gmail ads, and then look at the vague ‘suggested sites’ feature. This doesn’t change my opinion of microsoft much though; it was already low, now it’s a bit lower.

  144. The great thing about a story like this is that it illustrates a deeper issue with society in general. Two people (or in this case, companies) get involved in a pretty heated debate, people take sides, the debate gets heated, personal attacks are issued from both camps, and no one seems to realize that everyone who even takes a side is wrong.

    Google is wrong for even publishing this information in the first place, which 1) allows Bing to correct it if it’s indeed going on and 2) doesn’t allow for the opportunity to use the honeypot to their own advantage if it is indeed going on by serving up personalized, irrelevant results as a form of countersabotage. If they didn’t ask permission from the websites which they used for their experiment (and spy intelligence suggests they wouldn’t), that’s wrong too. No one seems to have commented on that, but it’s wrong.

    MS is wrong for not coming up with a proper denial right away, or really properly denying it at any point. MS is also wrong for using browsers and toolbars to monitor user behavior without any kind of upfront disclosure. They’re also wrong for using it as a search signal when it’s not really relevant and will also open the game up for all sorts of manipulation (Bing Toolbar = Alexa Toolbar v. 2.0).

    Both parties are wrong for using political strategy tactics (i.e. timing), rather than putting the issues on the table as quickly as possible so that they could be dealt with rather than worrying about when to reveal the information. If this was all discovered in late December, early January at the latest is when it should have been revealed. Both parties are also wrong for using a third party to report any aspect of the story in the first place, although I think that’s a bigger X against Google since they went through Sullivan rather than dealing with the issue upfront.

    Anyone who even took a side in all of this is also wrong, because neither party is right in the first place. I’m a fan of products and services issued by both companies, but this is not either side’s finest hour.

    People who went off on rants and wrote blog posts and dropped links for self-promotional purposes are especially wrong. It’s one thing if you’re Matt trying to explain your side of the story (sorry, Matt, but I’m not with you on this one). It’s another thing entirely if you’re some wannabe who decided to post a bunch of bullet points backing up your opinion onto either your blog or someone else’s and then go around putting the link anywhere that “seems relevant” (translation: anywhere that won’t delete your comment for the spam that it is).

    The bottom line is that unless either side is prepared to do something positive with this, and it doesn’t appear likely, this story is best left ignored. All it does it make the search engine industry even more political than it already was, and that’s the last thing anyone should want. This went from mildly amusing at best to annoying on the Gilbert Gottfried’s voice level.

  145. What’s the problem?

    Is Microsoft violating any laws? No
    Is anyone getting hurt? No
    Is anyone worse off? No

    Is Bing getting better? Yes.
    Is that helping -the few- Bing users? Yes!
    Is online information now easier to find than before ? Again, YES! (for IE users with Bing sort-of pre-installed)

    Than what’s the problem?
    Stick to your motto Google: Don’t be evil,
    just enjoy, because you are the winners here.

  146. I can’t really understand why Microsoft keeps changing the subject, the 1000 signal thing is just a poor excuse, fine, use Google’s search as a signal, but what Microsoft is failing to answer a simple question: If you are using Google results to improve yours, why do you display Google’s results DIRECTLY on your users pages?

  147. Kay

    From the evidence Google has shown, It’s only appropriate that Bing gives due credit to Google.. like so.. “All *Nonsensical* results powered by Google”

  148. Harith

    IMO.. especially SEOes owe Danny Sullivan much for his original story that kicked the Google-bing saga off.

    Its very informative and educating to read and hear Googlers and “Bingers” discussing publicly matters related to Gooogle & Bing search algos, Google and bing toolbars contribution to search ranking etc…

    And as one might have expected, the discussion has been sometimes very “passionate” indeed. Nothing wrong in that as long as we keep in mind:

    We members of SEO, SEM and Search industries should never resort to demonizing those who disagree with us.

  149. AKAMe

    “I think if you asked a regular person about these screenshots, Microsoft’s “We do not copy Google’s results” statement wouldn’t ring completely true.”

    Keyword here is “regular” person, so the above argument is nonsense.

    A regular person won’t have the knowledge to know whether this is a copy case or not. The engineers in Google are not regular people on this issue: they are experts on search engine. They know how search engine works and they probably do the same thing to other search engines too.

    To the Google People: if you want to complaint somebody next time, please do more investigation to avoid becoming a crying baby.

  150. I always selected “No” on that dialog, on my PC and on PCs I’ve setup for people. But anyway, rarely do I use Internet Explorer. I mostly use Chrome and Firefox.

    About the subject, I couldn’t care less if Bing is copying anything from Google to improve their search engine results. But I am definite that they’re not just “copying”, and their goal is to produce the best result for the searcher. Nonetheless, I understand that the procedure they are using with Internet Explorer is not very honest.

    BTW, the auto search (history) feature while you type in the address bar of the Firefox browser is awesome. While Chrome sucks badly at that. I hope to see an improvement in that with Chrome.

  151. You may have a point, but this is presumably just data collection through browser toolbars. Is Google able to say they don’t do similar data collection?

    I hope you’ll forgive me for saying so, but I can’t help feeling that Google’s alleged grievance is a little rich, when copying other people’s products is seemingly part of Google’s core business, and I suspect other people feel the same way.

  152. >Well if everyone else is dropping links in here:

    Ha! I bet that’s pretty much what Bing engineers said when they wrote the ‘Google Signal’. eg: “Everyone else is doing it”.

    Google uses toolbar data – why shouldn’t Bing?

  153. Van

    Security is never a “convenient” choice … that said … just surf Anonymously

    Both Google and Microsoft acquire user content and localized data.

    However, Nothing is “Free” on the net. I see a new set of countermeasures
    for the end-user that addresses what data or information is sent … “For Free”.

    Google asserts that they do not use their “Bar” to collect “Bing” users data.
    Microsoft uses their “Bar” to collect “Google” users data.

    Most folks’ PCs have Microsoft OSs on them with IE as default.
    How secure are you?
    How diligent is the end user to clear out all its key or important data from
    their PC?

    While the latest browsers (IE included) do allow you to delete private data
    and cookies … it requires user diligence OR a choice or decision to Surf
    ANONYMOUSLY
    .

  154. I think it’s pretty evident that Bing is copying search results from the images above. This kind of leaves me wondering if this has effected Googles search traffic at all? This doesn’t make me want to use Bing anytime soon. My homepage always and forever will be Google.

  155. Joe

    Matt,

    Are there any legal implications in the utilization of Googles intellectual property for Microsoft”s profit? Also, is it legal for them to scrape data from user behavior without being explicit about their intentions?

  156. Srini Raghavan

    Dude
    If Google and its storied founders had not so blatantly plagiarized Overture’s ad system & pay per click model, Google wouldn’t have existed today and you wouldn’t have a job at Google.
    Bill Gross of Idea Labs was the real genius that Larry & Sergey merely copied the technology.
    STHFU.

  157. Google and Bing should work together to resolve the privacy implications of this business before the government feels they need to get involved in preventing these companies from tracking what we are clicking to improve our search experience. The reality is, if I do a search on Google – which I do 95% of the time, I am not going to all of a sudden then start using Bing and think its better because it list the site I visited as the first result.
    What isn’t clear and should be addressed, how were these sessions administered for the experiment? Were users logged into a Microsoft account, or just linked via the IE browser they were using?

  158. justin

    Microsoft has a long history of thievery. From all out stealing the code from Mosaic web browser to stealing code and ideas from Apple. This seems overwhelmingly incriminating. Even if bing copied EVERY search result from bing I still wouldn’t use it. Thanks for bringing this to light, Matt.

  159. Dear Matt,
    Thanks for the post and especially for the screenshots. Now I understand exactly what tests you performed to prove your case.

  160. I personally don’t believe in having closed search results system with users having to search in different search engines to find what they need. The point is users should find what they want or need easily. Users would be happy with the results as long as it produces value to them – without worrying about whether it is from Google, Bing or XYZ.

    Another thing is that search engine is not like the grocery store that every other one needs to have something different. Over the internet, website are either local or universal and so should search engines be.

  161. Bob

    It’s more like improved upon and isn’t that one of the greatest things with the internet, it’s evolving as we work. Being part of such a amazing time in history is just beyond my wildest imagination. Besides, when you reach a site that doesn’t answer your questions or present the information in a manor understandable to you, don’t you hope that someone will improve upon it? I surely do!

  162. hi..! Matt Cutts
    I have been following up this news for the whole week, and from the evidence that you (Google) disclosed. I think this shameful and abusive on Microsoft part…if Bing want be a real competitor to Google……! the easiest way ……Using the media to say sorry can help restore a damaged reputation. ..
    Thank you
    best regards,
    jeffy wu
    06/02/2011

  163. Google should stand above. IMHO, it was bad communication from Google.

    Microsoft didn’t copy the search algorithm, but search results. I could be mistaken, but you don’t have a copyright on that content.

    People in the street will remember “Bing works equal good as Google”. Which isn’t true, not even close.

  164. Chris

    I think an effective/cost-friendly process would be a search algorithm uses a combination of user input + its own algorithm to determine site quality. For example, by allowing users to input/determine if a site is spam (add-on for a browser) then if alot of spam signals are sent, someone from Google takes a look at the site and comes back with a quality score. However even if the site is considered spam by alot of users, and the person reviewing gives it a bad score the page isn’t automatically delisted from the search results. Instead it has lost a certain amount of ‘trust’ and reduction in rankings (by somewhere between 15-25%) the other 75% of its ranking is determine by search engine algorithm. I still believe Google can make huge improvements with its current algorithm, that would make it unnecessary to conduct manual reviews.

    I don’t like Blekkos approach (which is similar as i have said), but they have removed alot of sites from their index. Alot of the keywords i have entered, have come up with quite irrelevant results (small test)

  165. Bob Quigman

    I’m still trying to find a way to explain this to my Mom, or to some other Regular Person who is bound to ask about it at today’s Super Bowl party.

    Matt, since you mentioned both of these archetypes of the “I’m not very technical” crowd, how should I answer them when they ask me questions like this:

    Yeah, but how did Bing know it should search Google for those funky words?

    How did Bing get the answers from Google? Did Bing Google the words?

    I made up my own non-word, askiflaxitrob. Google didn’t have any answers. Neither did Bing, but it had some answers for similar words. Was Bing being desperate because it couldn’t get any answers from Google?

    I just know I’m gonna get some of those questions. And the questions will start with “OK, Mr. Computer Smarty Pants, explain this to me…”

    Matt, I want to be ready for these challenges to my Mr. Computer Smarty Pants crown. So help me out here, Matt or Friends-of-Matt. How did you explain this stuff to Mom and other Regular Persons?

  166. Swat

    Matt , I was a fan of yours, but this whole episode has been very flimsy. David has updated a post with the other side of the story, and I think its pretty well told.

    The evidence you post above with screenshots can at best be called as childish. You havent cared to explain or refer to the other 91 queries which does not suit your argument. Google complaining about this sounds more like the Devil preaching the bible.

    You have been selectively answering a few questions which suits your side of the argument here in the comments above. Leads me to believe you do not have answers for other questions about Google’s own disclosures, and if mom’s data is any better off , or for that matter if she is any better informed about what data Google tool bars sniff. Care to explain these ?

  167. I’m not sure what to take of this, Matt. Sure, google has definitive proofs that the sting worked, but the legality of the issue is a little bit more complicated than we might think off hand. For example, people on HN have a feeling that if there was custom code in the bing toolbar that tracked google searches only, that would be a bad thing. However, if it tracked clicks (as consented by the tiny ToS when you signed up), and it tracked all clicks, not only Google searches.. it means they’ve basically covered themselves up.

    Is what they’re doing trixy and not purely Goodie-do company stuff? Yes. Is it black hat tactics? I’m not quite so sure of that, at least with the evidence/retorts we’ve got now.

    I’ve love to hear what you think on the clickstream issue (or if you commented somewhere, a link towards it).

  168. As an SEO guy I spend most of the day thinking about Google but it makes me feel good to know there is another search engine that will give me a different answer to the same questions. Now that Bing and Yahoo merged we have fewer options and we need variation to keep the world sane.

  169. Always tough to argue with straight evidence…

  170. Certainly a great post Matt however I have to agree with some of the other commenters that omitting the mention that +- 9% of the test search terms were concluded as ‘copied’ by Bing is important. In your piece above it reads (and looks like) all the test search terms returned the same result/s in Bing and not the 1 in 10 that were shown to return the same SERPs result in Bing.

  171. Hiren

    Well this is true but meantime bing also doing some really great stuff. So we can think of it.

  172. Lamont Kialome

    Who uses Bing anyways?

    Everyone I know says “I’m gonna google it”. No one uses Bing!

  173. Sunny

    Google DOESNT own Clickdata. Users own there clickdata and by participating in the opt-in program, they are transferring ownership or atleast right to use to MSFT.

    Is it ethical to use Google’s clickdata if ever Google owns it? Yes.

    What was break through in search? PageRank. How does it work?

    It uses outbound and inbound links on x.com/blah to rank all the pages that linked by x.com/blah. Isnt Google using x.com/blah’s data to rank other pages? Is it ethical of Google to use x.com/blah’s links to rank the content of pages it is linking to? Yes. Because it helps in better ranking, on the same lines, its ethical to use clickdata from _ANY DAMN SITE_ on earth to rank results.

    There is lot of difference between clickthrough and scraping.

    Get it straight, every real time sensor lacks context.

    User was displayed 10 links and he clicked on 4th link. So what? To use this well you need to know what else was displayed to him. A guy queries for his girlfriend’s name and there is celebrity with name whose facebook profile is first link and his girlfriend’s facebook profile is 2nd, he clicks on second. Does that mean 2nd was top result for this query? No.

    Also, clickdata is for static ranking which is query independent. It doesnt make any sense to associate it with a query.

    Also imagine, how Bing can royally screw its result by just including Google click data. It has so much noise. Its hard to normalize, people from X country tend to click more than Y country. You cant normalize it.

    Google results are specific to geography. They are query dependent. Given that you cant analyse log in real time, click stream is majorly a static ranking feature. And static rank is always query independent.

    More over, if I were to use that feature, I wont do it on Google. Web results dont have any context. I would rather prefer clickstreams on a specific site. For ranking technical articles, I would prefer TechCrunch clickstream or some tech blog than generic web result.

    I do agree that Bing benefits from Google’s ranking, but not because its Google, but any search engine or any popular site. If any item is getting more clickthrough on eBay search, its will get better rank in Bing. So I dont find why Google is so cocky about there IP. Its not there IP. Its user’s click action that is benefitting Bing.

    Now the point is, Did Google did there homework before going public? Did they stand by and let media took over and blame Bing?

    No, the are incomplete in there accusations.

    None of there experiments prove that Bing uses Google as a special signal. Those are so gibberish queries, that only clickstream signal had a non zero score. More over it was clicked only on Google, so only Google’s clickstream mattered. No where it signifies that Bing uses Google’s clickstream data in special way.

    I believe, you would agree, a monster search engine should have done a full fledged experiment before going public with such half baked result. Unless, they have planted a really huge PR coup to either get away from hot topic of spam on Google or to hamper Bing.

    Oh, to conclude. By all means, Google is still best. I wont move to Bing. But Google lost the respect it had :(
    Can we get back to something more productive?

  174. Clearly they are not copying. All you need to do to test this is type a set of random letters into Google. Then copy and paste those same random characters into Bing. You get different results every single time. I have yet to find a set of random characters that show any common results between the search engine. All of the examples provided seem to be hand picked.

  175. You can prove it by supplying the steps to duplicate it. Screenshots are not enough evidence.

  176. PJ

    Humm.. Very interesting post Matt.. I really enjoyed the screen shots.. you made bunch of good points.. but here is something to wonder.. what if Google is stealing Bing’s search results? think about it.. Bing can make the same screenshots and call them a conspiracy with Google’s search results.. right? I mean at this point algorithms being so close, how can one search be differentiated from another?

  177. But these are some unique/alone queries. What about general/popular ones? I doubt that they will be similar… Although very interesting experiment!)

  178. PWashington

    That was like a street fight.

  179. 9 out of 100 results are as per your point of view ..which you showcased…why not talking bout remaining 90 results which DID NOT prove your point :) Do you have any reason why not 100 percent results matched? If Bing is stealing why it steals only 10 and not 100

    talking bout disclosure….Since for which Google product we have a specific disclosure? I can not find a single one..!!! We haven’t forgotten Google Buzz yet …Google Toolbar smuggles user data and we have no disclosure :P http://www.seobook.com/i-turned-google-toolbar-it-kept-spying-me

    By giving limited details, a regular person may get fooled by you guys but a technical person will only laugh at you…like we are laughing …. :)

  180. ROY

    Well, you guys just don’t see the side effect of Bing’s “copying”/ “stealing”/ “learning”, whatever you’d like to call. THIS is not only matters the search results we can get today, but the direction of search engines’ development in the future.

    Building their own work based on the others efforts, will that benefit us? In short-term, yes we may be able to get better results from Bing which may cause some users flowing to Bing. But in long-term, definitely not. Bing starts an unhealthy tendency acquiescing which gains almost directly from other search engines hard working.

    I know they don’t just paste Google’s results on their sites that simply, they uses “the users clicking habits”. But think about it, your habits actually don’t work while you’re searching for something because the desire of getting a proper answer just overwhelming your other “hobbies”. I don’t consider this desire as a part of habits for even if I hate to click the Wikipedia all the time, I’d still click on it if I have to. It’s unlike you click a random blog’s link from many others after all.

    Let’s back to the long-term effects. If Microsoft doesn’t stop what they’re doing, it would be no doubted that they’d kill Google someday. I’m not trying to scare anyone, but the users would finally go to the one’s results with better relevance and less spams: Bing does because their results are of the best from others by “borrowing”. They may buy Google’s PR in a low price then “BingIsNowGoogle”. Or if they somehow could not been able to get it, they could continue their “learning” from other smaller while might be better search engines like a new borned “Google” by the same way. Small engines couldn’t afford such user-loss so they may simply die soon. Thus innovation would be inhibited again. Bing wouldn’t copy everything at once, maybe they are just starting their experiment to see whether they can make it gradually then just caught by Google in time.

    It is dangerous to say a “yes” today without thinking twice because tomorrow we’ll be still exist and maybe still need search engine’s help.

  181. SpinMaster

    So this is what I could gather from Matt’s explanation

    7 to 9 % (partial) success rates (don’t have a clue on how it actually worked)
    Take a wild guess on how Bing works (we don’t know why it failed more than 90% of the times)
    Set a really, really, reaaalllllyy low standard of statistically relevant data
    Do the test only on Google and Bing
    We think we are the best and every one has to follow our way of ranking or collecting data, only our way is the legal way. We do not think what Bing did is illegal, but it is just different from how we do it so it must be declared illegal.
    ….

    I tried really hard to look for the evidence that would lead one to conclude that Bing was outright COPYING (a very strong word in this case). None of these (and some) points are close enough to make that call. So, now I get it Matt, you are able to come up with some incredible deductive reasoning that our tiny brains can’t comprehend or OR you are pulling a big one out of our collective sorry as…s (Which I think you are, because I hate to think of you guys as being so naive in your conclusions to the point of looking stupid)

    Of course you are insulting the intelligence of Bing developers and its entire team (despite your remarks to the contrary). It is extremely unprofessional considering the way you arrived at your conclusions and the timing you chose to bring this up. It reflects poorly on you (and your teams) analytical skills and ethics. If this is in fact a spin (as many seem to think) then it reflects poorly on your professionalism. Either way you just lost a lot more respect due to this story.

    Episodes like these (and Net neutrality, Android etc) just goes to show that in business there is no higher angel. Google has now much greater influence on every one than Microsoft, simply due to their reach and kind of interaction with general public. That is a huge responsibility on Google to maintain principles of neutrality and fairness and equal opportunity, the same that we so highly expect of Microsoft. You are now a grown up company and you can’t keep doing the CRY BABY routine on Microsoft.

    If any one needs to take blame on this then it is Google for spoiling the image of Bing and Microsoft with nothing but misguided (deliberately so) comments.

  182. I wouldn’t worry about who is copying who. It’s whoever is setting the trend, innovation, and is the leader in search that is more important. People will use what is better, and word of mouth and marketing will help along the way. Right now Blekko has come out of the search gates with a innovative method for better search. But, a one trick pony will only take you so far. I believe the future of search will be 180 degrees back to DMOZ method, or paid inclusion with human editors.

  183. G’Day Matt,
    Irrespective of one copying the other or not I saw the main issue as improving the search results.
    Why not trial a system that involves the searchers as well. Maybe an interactive user rating and based on the result of a random block sampling the site could then be referred to a human before being labled as (ad..)Spam.
    Sites genuinely attempting to provide usable information and services should still be entitled to monetize the site using the ad systems with related advertising.
    The whole affiliate marketing system should not be labled as spam. Sellers & Resellers should not be disadvantaged by any change to the system, after all anyone can setup a brick and morter and profit from their efforts.
    Political Sites – Well politics is politcs enough said.
    This is still going to be a very complex problem to solve however it is done.
    Gold we Glean
    Reg

  184. Wow, those snapshots are very interesting. Looks like Bing really does copy Google’s search results…

  185. It’s so obviouse what they’ve done, I don’t know why there trying to make up all the lame reasons.

    I had noticed myself that a lot of their results were very similar now, where as MSN was always way off.

  186. Sounds like sour grapes to me…

    I think you may have shot yourselves in the foot by giving such prominent exposure to a competitor that is now producing better results than yourselves. Anyone trying Bing for the first time may not come back!

    Let’s face it… 10 years on and you’ve ended up down a cul-de-sac of a top ten consisting of wikipedia results and spammy duplicate content sites.

  187. Google considers the use of clickstream data that contains Google Search URLs plagiarism and doesn’t want another search engine to use this data. Bing claims that Google doesn’t own the clickstream data and therefore doesn’t have a say in how this gets used.

    Should the courts decide what constitutes fair use of clickstream data and who “owns” these data? When the dust settles and the unfriendly back and forth between Google and Microsoft engineers comes to end, I’d rather see the smart engineers on both sides put their heads together and come up with a technical solution for fair use of clickstream data.

    One possible solution is applying the Robots Exclusion Protocol to clickstream data to put the Web site owner in control whether or not information from that site should be considered.

  188. I have read Yusuf Mehdi’s post and even though I love Google and use it 99.99% of my search time, I honestly believe that he has a point and believe the “honeypot” attack claim :)

    Also, the experiment done here is biased and shows tests that worked for Google’s claims only. To me this looks like a propaganda ;)

    Considering the click stream data Bing is getting, these data are owned by the users and if Bing is getting them with the users’ consent, it is seriously none of Google’s business.

    However, the only problem, as I said before, is that they are not being explicit with the users about this.

    Thanks for shedding light on this though, regardless of the opinions about it.

  189. Matt,

    You pretty much “pwn’d” this one, and you did it in a clear, concise and professional manner. I don’t think it is so much that they did it, but that they so adamantly denied it that is the whale here. If they would have just said… “well yea” this would never have been very entertaining to any of us.
    Thanks for your cool hand on this one. kudos

  190. I watched the debate yesterday and I found it really interesting. The competition has a lot of “guts”, that’s ok and pretty normal since they want to take on a search engine as Google .. Blekko obviously has a long way to go since their relevancy in the searches is weak even for the search queries they suggest in their own tutorial on how to “slice” the web :)) Like, I’ve searched “Comedy /cats” thinking I’m gonna get some funny clips with cats and the first result was some online shop were I can actually buy cats, cat food and stuff like that.. now, blekko’s ceo said that funny exemple when he would search “how to make a brick wall”.. but, how does that stand out when his own search engine will give such irrelevant results for search queries he actually suggests searching for :)
    Belkko is also way too techy for the average users.. displaying all that “SEO” information for someone . Also, the command line concept (slicing) can be considered way too techy also.. I’m just thinking about how many people on Google know and use the simple command line parameters (like AND, or at least the commas). I’m pretty sure a small percent of the searchers do that. So, Blekko might be good for a niche search engine (ie. for webmasters), but I can’t see it getting used by the average users.
    As for Bing, what they did is similar with stealing someone’s else content. They stealed the results of Google’s algorithms saving processing power and ideas. If big guys as MSN do that.. it will only encourage others to do that as well.

  191. @Kevin,

    I think that’s a little harsh an assessment to make. Sure, there is an issue with getting similar data from a query but that is in some part down to the number of sites on the internet. there is only so much data in the world and no system to sort it all and present a top 10 of varied data is going to be perfect. With regard to Wikipedia, it is a central resource of a lot of unique information so it is always going to place high.

    That said, information on the internet and not hidden behind security is by default in the public domain. If Bing is using data from Google then it should be referenced as a quote would be in any document. Vice-Versa Google should be more open about the sources of their analytical data, especially if the argument is over data collect from user clicks which nearly all online data handling includes.

    Rich

  192. Comment: Matt certainly knows more about this than I do. I have been using Google for years and the search results don’t seem to as ‘clean’ as they used to be. By ‘clean’ I am referring to sponsored sites that appear first and don’t have much free information unless you pay. This is especially true of people searches. Also putting ” before and after a name seems to irritate the search engine leaving you a note to remove quotes. Forget an image search. You may get that person, if your lucky otherwise you end up with unrelated photos and even different first/last names.

    And another thing after several years of blogging I had to delete my google blog and start a new one. The HTML screwed up my format and absolutely no help from Google Support. No one from Google addressed numerous quires regarding the spacing problem between text. Very, very poor support for blog users.

  193. Well it seems that there are no new ideas these days….and just like Duplicate content: he who gets there first gets the credit! Thats why Google owns what? 70% of the online search market!

  194. wilner

    Another issue to discuss, is data content copying. Google and Bing both make their money scanning/crawling the web sucking up the content that other people have worked hard to create, and monetize for their (Google and Bing) gain. And they use the business model pioneered by Overture (not Google) to do so, and the display design (links) and some of the methodology originally pioneered by the likes of AltaVista, and its predecessors. Talk about copying . . .

  195. Thanks for posting the video debate. As a not teckie it was interesting to hear so many good points brought up and debated by all three. Lots to learn from each other.

    The thing that surprised me most is that there are only 3 companies in search now, and I do believe microsoft/bing is doing a service by competing. I also signed up to blekko and will do more to support them with my business in the future. I still don’t really get / tags though, so hopefully they do videos like webmasters tools do to educate us (thanks google).

    My final point is as a consumer. I started 2 companies last year with websites. One in a small niche that I know well and we got listed high enough fairly quickly. The second is in a bigger market that I don’t know as well. It’s understandable that we’re not as highly ranked yet, but what really is annoying is how many spam sighte are ranked above us. We are a real company making a real product and doing honest blogging and linking. How is it that google can’t figure out that the crap ranked above us is just that. One of the only saving graces is that some of our other competition is worse at SEO than us.. but that doesn’t say much about the search companies.

    Keep competing, but thanks for all the help so far.

  196. GinaB

    While I still use and love Google, I am increasingly disappointed in the results that I get from Google. Chiefly, because there is too much junk in Google’s results. Google is a victim of its own success: so many companies what top placement on the first links page, that they are fighting to get PageRank to rank their sites high. This drives up keyword prices which is good for Google’s profits, but it also produces links to worthless information. Google needs to find a way to cull the illegitimate stuff from the legitimate material; removing duplicates is a step in that direction but it won’t get completely rid of all junk sites, even if Google removes sites like eHow.com and other content farms. I have been using Bing and DuckDuckGo more and more and Google less and less. Their smaller market shares mean that fewer people are gaming their results, since there is less or no monetary incentive for people to want top placement. For the searches I do, I find DDG (DuckDuckGo) an awesome site; it usually finds what I want on the first try and the first few links. Bing is not bad, and like DDG, I usually find what I want quicker than Google. Perhaps those “information overload” commercials were right after all . . .

  197. rush

    no one cares. deal with it.

  198. justin

    I know it is unprofessional for a google employee to say this but since i’m not a google employee:

    Microsoft is trying to make copying all of their competitors a legit. business practice. That is the ultimate conclusion one must reach following their logic. They can’t copy one, so they copy all and write up a complicated technical explanation of it.

    In my view, any technical explanation of Bing’s search is nothing more than Microsoft trolling so the discussion won’t center around the real issue: Microsoft has no ideas, so they steal.

  199. Loco Vox

    Google’s reaction to this affaire is silly and childish. Almost like the company is being run by kids playing Monopoly.

    This clearly comes from toolbars, suggested sites or whatever other mean. Privacy converns are valid. However, in order to criticize, you need to see to yourself first.

    Google has its own problems with privacy and use of users information, and to my perception and IMO, Microsoft collects and use lot less user information than Google.

    If you want Microsoft to publish what they do with users “queries’ and “clicks”, start by publishing what Google does with every information you gather from users. From Google Maps on cellphones, to latitude, to toolbars, to queries, to cookies, to email, to everything.

    Go white and clear with exactly how you use the data you collect from us. Then ask Microsoft about what they do with what they collect.

    IMO your screenshots are directed to the media to create noise. They truly do not prove anything but that Bing is using different sources of information including users clicks and queries to rank and index websites.

    They are not copying anything from you. You just found a way to trick their algorithm.

  200. Casey M.

    Matt Cutts hi its Casey M. from MSU! How are ya? Haven:t seen you since say; 1990 I figure. I’m living in Cincy, just purchased a new home and am working in astronomy and applied theories with the Spitzer space telescope in conjuction with Jet Propulsion Laboratories in Pasadena CA. and volenteering at SETI in Mountain View, CA. 3 months out of the year. So what are you doing these days? Say hi to Ross for me and Blake too, think I saw his name pop up here somewhere hehe. Anyways I agree with Mr. Cutts 110% about the above matters, it is truely a “MATT”er of a fact. LOL! Take care Matt God Bless!

    Respectfully,

    Casey M.

  201. If somebody imitating means that’s the best way of appreciation they are giving. I’m a big fan of Google but not believing Bing is copying the Google search result. There might be some similarities in the search algorithm that’s why it brings the same result for “some random search queries”. Personally I used to wonder how a search for my village is appearing in Google, Bing and Yahoo. I always found Bing and Yahoo are more closer in search algorithm than their similarities with Google. The results showing in Bing is more sensible as it’s showing the website of village first and blogs that used to update regularly. But in Google, it’s more oriented to the keyword popularity. Even a blog with only one post with too many keywords for the village is showing at the top after the website!

  202. Tom

    I just lost any respect I had for Bing. They were caught red-handed, which is bad enough. But then they blatantly lied to everybody about the facts. Do they really think that we are that stupid?

    Anybody with half a brain can easily view the evidence presented above and conclude that Bing was copying Google. There is absolutely no other plausible explanation for Bing returning the same results for those highly obscure search terms.

    As a result, I believe Bing is highly unethical…not for using Google’s data in their algorithm, but for having the audacity to LIE about it. I could care less if they use Google’s data as a signal in their algo…but at least be honest about it. Somebody at Bing needs to get fired over this…they completely destroyed any trust I might have had in that company.

  203. Hi Matt, Honestly while I understand that this is being looked into, I really think it’s a mistake for anyone at google to even pay attention to the problem in the sense that you are just putting more attention to Bing and making them more and more important. I tried to use Bing and frankly it’s not near as good as google. By about 95% the more Google argues about this the more advertising google is giving Bing.
    And the leader is always going to be copied, take the iphone and other smartphones on the market. They’re copying iphone GUI as well.
    voila, there’s my 2 cents :-)

  204. It’s obvious that microsoft and bing are trying to get some visitors from google.
    Google’s algorithm its unique, and microsoft should make a unique algorithm also.
    If the don’t do their own job, there will never be any competition between the engines, and without competition the progress of search engines will not proceed as quickly as it have to.

  205. Steven

    Does everyone really think it’s impossible for the worlds top two search engines to find the one page on the internet that matches a unique query?

    I guess if I build my own search engine, create a page with a unique 25 character key, and have my search engine cache the page before Google and Bing it means Google and Bing are both hacking into my servers and scraping my content.

    Is this what the engineers at Google spend their time doing? Query random things and see how small the search results need to be before Bing and Google show the same results? Or do they just assume Google is so great that there is no way Bing could possibly find those websites?

    Just to clarify I’m a big Google fan and use Google for 99% of all my searches. It’s just this seems a bit childish to me because isn’t the end result delivering the most relevant content to the users? If I was an employee at Microsoft’s search I would see this as Google openly acknowledging that they feel threatened my Bing.

  206. Christian

    Also, I’m sorry, but I hate stupidity. If you begin manually removing sites from spam reports, what stops me from reporting all my competitors for spam, or creating google chrome accounts like crazy blocking my competitors, know google would look at this info. I would not do this, but I know people will. If site selection becomes manual, this opens the door to manipulation, even greater possiblities than there is now. The possibility of external influence over a site, such as from competitors, cannot have the door opened. If they don’t automate, they will never be the most preferred search engine. With people choosing sites, the big guys with money could also corrupt their judgment. Algorhythms don’t care about their money, they will return the best results. There I have spoken my peace once again.

  207. Pawel Osiak

    I like that part in the debate about spam that is online. Many people and businesses is waiting for big clean up in search results.

    Nowadays when you create decent website it takes so much time to climb up in the search results that is really disencouraging for creating good content. Very often people us farm links like in JP example to be seen online. From my own experience I know that it took Google 3 years to notice my page with following their guidelines which was not easy.

    When it is about copying results or not, I think it is a good impulse for google to work harder on returned results and maybe as it was said ‘ as a leader they should provide some standards’.

  208. m_live

    The author of this article know more now about this fiasco than when the article was written. Atleast Bind did not steal the data and Bing is using it like how Google does. It is just another signal. who knows what Google’s signals are. Dare to list ?

  209. Jon

    What’s funny is that on the few searches I did, bing still shows the erroneous results as the top result. You’d think they would have removed these by now, especially because they are showing in the #1 spot.

  210. And, Matts that why you people launched that fight for SPAM algorithmic. Really appreciate.

  211. That is interesting however I always feel competition is good. Don’t let a bad apple rot such a great service. Statistics show we all love Googles software :)

  212. I still think Google is miles ahead of Blekko and Bing. I mean, I searched for some keywords related to my website, and Blekko’s results are incredibly inaccurate. Google will do a great job by deindexing those exact-match domains made for adsense and removing the spammy search results which are actually the internal search results from other websites.

  213. It was really shocking for me when I frst read the news that Microsoft’s Bing is copying Google’s search results. Microsoft is considered so big Internet Giant like Google and if such reputed big names will do such theft activities then how can one blame the smaller peoples.

    really disappointing by Bing

    ATUL

  214. Ashish

    Well I searched some realistic keywords in both google and bing. Results are totally different in both search engines. I think Bing is also working hard on search quality and their algo might be clashing with google’s algo. But I am really disappointed with matt related to this issue because the examples of copy results are for those queries which are not useful normal users. Matt can you give us a real solid proof which can change our views for microsoft.

  215. Guy Peri

    I just had to tell you that bing does a lot more then stealing your results…
    I advertise on adwords, and lately I found that some of my keyword I target is in the first page on bing, with the link to my advertise page (which only available on adwords).
    Also the result is the adwords ad, and not an indexed version of my website…

  216. Agree to that! I had also different results. so its hard to believe Bing is copying Google’s search results. I’ve been confusing now.

  217. Microsoft has always been an amoral and sometimes immoral company and probably always will be.

    Unfortunately, people are starting to use Bing. I try to tell them they are being foolish, but they just don’t see it.

  218. At first I thought: “well maybe, there’s a perfect algorythm, and both Google and Bing are so close to it, their results are very similar” Which at the end of day, may well happen one day.

    But now, I’m not so sure. It wouldn’t suprise me if they were serving Google results verbatum.

  219. First off – The guy above me actually used a keyword-stuffed name??? Seriously? On Matt Cutt’s blog?

    More on point: I think the main point here is EXPECTATIONS. People have wildly high expectations of Google search results. It reminds me of the early days of AOL when their service expectations were really really high because they were an early leader in their field. Will the same thing happen to Google?

  220. Josh

    Is it that bad if results are copied, think of it as flattery, it is a good idea to compare notes as it makes it more efficient.

    Regardless, algorithms can only go so far… judging a site’s quality by backlinks is extremely biased against new sites which may be much higher quality than older shallow sites. Analysing content is too easily abused by onpage spam (though arguably links can be just as easily abused)

    What’s needed is machine learning so an AI can actually read the pages and make judgement calls on the quality

  221. hi…

    I am waiting to read about the post about you copying the website preview option and image tiles in the image search…

    Really waiting.. as u copied Microsoft in that..

  222. The results seem almost exactly like another kind of test … when people are testing whether nofollow links pass anchor text. The tests basically pointed some very obscure (nonexisting) anchor text at a URL with a nofollow link, in an attempt to discover whether or not that anchor text was being passed to the URL even though it was nofollow.

    The result was that the anchor text did get passed with nofollow, and the resulting page was in google’s search results (even though the term wasn’t even on the page being linked).

    Today people swear that nofollow doesn’t pass anchor text. But when it came to explaining why the nofollow passed the anchor text on an obscure term, the answer most seemed to come up with was that nofollow anchor text counts just a very very little, but when you have very very little to go on — that’s what Google went on. It could very well be that the anchor text on gibberish from a nofollow link has 1/1000th the effect of a regular follow link — but when all Google has to go on is that 1/1000th signal, it uses it.

    Take that exact same scenario and apply it to this one, where Bing is taking clues from Google clicks on gibberish. That gibberish signal from Google very well may be 1/1000th the weight of other factors — but when that 1/1000th is all that Bing has to go on, it uses it.

    The only way I’ll be convinced that Bing is knee deep in a pile of impropriety, I’d want to see how this trend works vs any real term someone would type into a search engine. Just like the nofollow anchor text that Google practically ignores on real searches, it could very well end up being a very small piece of Bing’s algorithm and on a real search, other factors do weigh in 999 times more than on a gibberish search.

  223. James Tyler

    Matt, has it ever occurred to you that Google might have copied Bing, rather than the other way around?

    I think you should stop crying about microsoft and do a better job in your own place. Google is a great search engine but you are beginning to lose credibility with your frequent algorithm changes which ruin businesses at the drop of a hat. There seems to be less and less stability in your results while Bing is only improving with time. Remember not long ago Yahoo was a major leader in search engine market share, now take a look at them. Your post comes off as a desperate attempt to discredit the improvements, drive and ingenuity of microsoft as they are on course to overtake Google in the near future.

  224. hi,

    i think for now most of the people using bing do so inadvertently (either IE search bar, facebook or some tool bar..) and even though the guys from bing are obviously trying to get some of the global search traffic they should start relying on quality results…

    but as far as blekko’s human/editor’s interfering with serp – i’m not really sure that is such a great idea. because truth be told if there is human intervention there will always be subjectivity. what if the editor likes dunkin donuts and i like starbucks…

    in my opinion spam is like crime or the fine print – it will always exit and get more and more sophisticated, the question is who’s doing the better job countering – for now google

  225. HA – well I think that answers that question… Enough said.

  226. Wow, I didn’t know there was such an overwhelming crew of Google haters, haha. I think you guys do a great job and it’s pretty obvious that Microsoft/Bing wants a piece of your pie, too. I can’t blame them, but I would figure they would be able to choose a more ethical route.

  227. Is there going to be any legal action take against Bing ? I’ve seen another search engine copying BING :))

  228. I think this is the most feisty I’ve seen Mr. Cutts. I feel that if the major engines got together to form accreditation or certification programs for reputable and knowledgeable SEO companies that would be great first step in cleaning up the web. Just my .02 I think Blekko came out a winner in this discussion.

  229. I think if you asked a regular person about these screenshots, Microsoft’s “We do not copy Google’s results” statement wouldn’t ring completely true.

    As a regular person I’m completely lost with the odd queries in your screenshots. If Googlebot can find this, bing can also stumble over it. If googlebot found it before bing, then bing could stumble over it in obscured Google search results, where the syndication rights or whatever it takes are unclear.

  230. Looks like Microsoft have been caught out.
    Surely some sort of legal action could be followed through in this case?
    Doesn’t Goolge protect its assets like most companies do?

  231. I don’t think if this can really be the case. I have tried different keywords in both the search engines while planning my SEO and has found several different results. And the position of my website was not same not even a single time while doing the SEO.

  232. Kristina

    Hey Matt, just about all of my 100% original, most viewed, most published ezine articles are now being outranked by low quality autoblogs or MFA sites that just copied the articles 1:1. So I’m sorry if I don’t have sympathy for what you and Google are going through.

  233. Great post Matt! There will always be spam on the internet, and you just can´t protect yourself enough against it. Your plans are great, and I know that you guys will implement a lot again from what you presented here in the video.

    Awesome pieces of content, and fresh documents should always rank higher than spun and bad articles. It also counts for blogs and other pages.

    I appreciate the information provided within this video!

    Best wishes,

    Gerald

  234. I bet the guy at Bing spat his coffee across the board room table after seeing this one. To deny such activity so strongly leaves them nowhere to go after this.

  235. Well I searched some realistic keywords in both google and bing. Results are totally different in both search engines. I think Bing is also working hard on search quality and their algo might be clashing with google’s algo. But I am really disappointed with matt related to this issue because the examples of copy results are for those queries which are not useful normal users. Matt can you give us a real solid proof which can change our views for microsoft.

  236. Brandon

    Nobody likes to have their hard work copied. It can certainly steal your thunder. That’s why so many publishers are baffled right now by the new algorithm. It seems to allow people to steal content and then outrank the original source, even when the original has been indexed already. I’ve seen publishers complaining about this all over the web. It seems this would only encourage the copy-and-paste (steal) strategy for web publishing. Do you know if this is being addressed in any future algorithm tweaks? Thanks Matt. And thanks for doing your best to keep us all informed.

  237. Finally, it is almost pointless going to Bing to check for alternative results. I used to think that Bing gave the same value to spam links that Google does now it seems they are just copying Google. I just dont get why Bing would do this they will never be a viable or credible altnernative to Google if they simply copy their results.

  238. These screenshot comparisons are hysterical!
    Of course, I don’t trust Bing, so I may be prejudiced.

  239. Love your post!
    Your a pretty calm dude, and passionate about your work, and yes it’s frustrating when someone lies to your face and doesn’t get it (especially smart folks) :)
    Good job Matt!
    I signed up to read your blog after hearing you speak.

  240. So new SEO optimisation consists in clicking on his own website in Google, using IE, to improve ranking in Bing (and Yahoo). Welcome to clicrank. It’s very good the debate.

  241. It seems that I rank in the exact position for up to 5 websites in Google and Bing. I personally prefer to use Google but I can’t deny the crowd that uses alternatives, I would like to see where this will take us down the road as far as SEO is considered.

  242. T.kik

    Compared to google stealing adress and review data from our and other business directories, and reuse them compiled in google local/places, i’d say you have lost any right to complain about behaviours of other competitors.
    You complain about ms processing clickstreams collected by ie, and at the same time you disclose ANY info about what you do with the clickstream of chrome.
    Beeing the owner of a rss searchengine and a well established business directory myself, i have learned the hard way how you treat competitors. Unbelievable how dare you whine about this.

  243. Louis

    “If clicks on Google really account for only 1/1000th (or some other trivial fraction) of Microsoft’s relevancy, why not just stop using those clicks and reduce the negative coverage and perception of this? ”

    Yep

  244. I hate to say this on your Blog, Matt. Only because it seems like ‘ass kissin’ but, you really held your cool and seemed to answer questions very openly. The guys from Microsoft, seemed slippery and dodgy. The guy from Blekko lost all credibility by wearing a Blekko shirt. I compare that (his shirt) to using your business name in comments.. Cheesy.

  245. Dan

    Matt

    Not being funny but if your search engine only returns one results, why wouldn’t Bing, not sure that is an accurate test? On the flip side, not a big Microsoft fan so you are probably correct. As an aside, is is nice to see how popular Chrome is amongst the masses these days and how fewer people are using Explorer, that is what happens in business if you produce a good product against a rubbish product!

  246. Matt,
    This is a nice article. I’ve got to watch this video when time permits.
    Didn’t they say that copying is a form of flattery?
    Cheers,

  247. Soumil P

    Matt,
    You can see Harry Back-pedaling….
    Harry is soo sidestepping…. he main issue…. and then he gets personal….
    No cool Harry. Not Cool at all..
    -S

  248. Well any person without knowledge of seo as well as one would think that if you know you are copying but rather based

  249. I don’t think there’s much debate here. The evidence is clear enough.

  250. Matt, I think for some, it’s all to do with corporate preferences. On one of the search engine forums my partner used to comment on, one of the moderators had this to say on the subject of Google/Bing following the latter’s launch: “But overall this confirms my sense that Google, having abandoned the business model that got it to the top, viz. fast no frills search, in favour of added features and buying up other companies, is becoming increasingly fat and complacent, and along the way their search results are becoming less relevant.” My partner had such a run-in with this guy that he was banned from the forum. Bing was truly awful just after the launch. It’s much better now, perhaps because they have copied Google’s model. Period. Full stop.

  251. I watched the debate. The Bing bloke should be behind a computer somewhere in a Microsoft office, not in front of a camera. No disrespect intended, but he isn’t the best PR man, they needed a sharp shooter to cut down the mighty Matt Cutts.

    I’m glad that you left the Blekko bloke alone though, they are true innovators. Cue Google bid for Blekko (joke, they could still go the way of Cuil).

  252. True or not, with some good will, everything is going to be short out. The screenshot are a good proof. Too bad, Matt had to public post them. It would be better to been short out under the table…

  253. So maybe Bing did copy Google’s results, but I hope they don’t copy Google’s new search results. The results I keep getting now that Panda is completely rolled out aren’t much better then Bing’s own search results.

  254. So… this is the reason that in my blog such weird comments appear with URLs from Google, Bing, and Yahoo ? I started to mark them as Spam, since I didn’t know what to think…

  255. Hi,

    Bing is following Google to display search results, I know it before, but if you check facebook you can find that Facebook is having Google “Gmail DATA”.

    Plz don’t tell me that I have shared my Gmail Friend list with Facebook.

  256. Eddie

    Microsoft’s Bing is on a slippery slope, if they’re in fact using Google’s data because they would essentially be handing over the keys to their engine. Matt described a way to manipulate Bing’s results from the end user’s stand point but that’s not what I’m talking about. Google could simply feed Bing garbage.

    On a side note, I just realized how annoying it is to type Bing using Firefox. Apperantly, Bing is less a word then ting, ling, and ding, according to FF’s spell check. Competition can be great but can also become a annoyance for the end user.

  257. Microsoft create their fortune copying everithing. The same thing happen with the following cases:
    1.- ICQ > MSN
    2.- WordPerfect (and many others) > Word
    3.- Lotus 123 > Excel
    4.- Mac OS > Windows
    5.- QDOS > DOS
    6.- PlayStation > Xbox
    7.- Google > Bing
    8.- Novell > Windows NT
    9.- Unix > Xenix
    10.-RocketMail > Hotmail

    And Many Others. And they will still do the same. Microsoft beats the original creators but this will not happen again versus Google. Microsoft in my opinion is Falling.

  258. GK

    That was like a street fight.+1 from me!

  259. Scott

    Well played Matt, unfortunate that Harry couldn’t directly answer any of your questions on the topic and rather replied with a bit of cheek trying to make Google look like the idiots here.

    Quite obvious that they’ve been caught in the act.

  260. I happened to search on Bing and was curious why the result was the same as Google – actually it was the same as Google’s serp two days ago, not today.

    That’s why I search “bing copy google serp” and found this interesting post. I was surprised at the screenshots.

    I went further and checked a bit more keywords, and found this is not true on all cases. For example, the SERPs of the mentioned keyword “ygyuuttuu hjhhiihhhu” are much different. And in keyword “bing copy google serp”, Matt’s post ranks nowhere in Bing but at #3 in Google.

    It’s nice to be different, though smaller.

  261. One thing for sure I wish that google was still in China. Atleast there would be more fair advantage. Not sure how bing works out there algorithm but baidu charges website owners a small fortune to get on the first page.

  262. Ahamed

    well, my support goes to google.com

    blekko, you seem effective now, because, alot of seo-ers are not targeting to be on blekko.
    lets say in future if you are the search engine leader, and you have openly said it is human influenced results. that said, alot of seo-ers will soon target blekko and there will be automated bots with many proxies in them and then categorizing all their competitors to be spam and coming on top of the results of blekko. this is basic idea of how your system can fail. but definitely there are loopholes and that will be misused.

    google is now doing a better search that they are given relevance as much need as possible. however, one thing which irritates me, i own an ecommerce and definitely i want to rank high.
    and for a keyword like ipad stylus or what. we are pretty sure, the search is intended for buying. since it is a product. but then, sites like WIKIPEDIA and CNET or TECHCRUNCH which are articles outrank many ecommerce site. do searchers looking for items really want to be on wikipedia, cnet and techcrunch? you may say google product search is your answer. but then again, people don’t see google product as a market place. they need to know about a company before they make a purchase. and one direct click to their site and they can view the products and they will be way more happier than looking at 1000s of ipad stylus from 1000s of different commerce store.

    so, i feel for a product term, ipad stylus, ecommerce should be given more weight.
    and for a term like ipad stylus reviews, articles sites can be given more weight.

  263. Google is the gold standard of search. I don’t know why people bother with Bing. I have always been happy with the speed and accuracy of my Google search results. I don’t know what kind of technology Bing uses but I never felt the need to try the second tier search engines

  264. I can’t tell you how many people make lots of money just copying the designs of clothes that they see. Knocking products off is big business and it is more than possible that Bing is doing that with Google. Who else would the “borrow” ideas” from? Certainly not Infoseek. Google does such a good job of what they do that you have to expect others to copy them as much as possible without stepping over the line, or maybe crossing the line of of illegal infringement.

  265. Bing have been caught with their hands in Google’s cookie jar and they can’t even admit it! Shocking behaviour, really surprised at the screenshots haha!

    Oh dear…

Leave a Comment

Your email address will not be published. Required fields are marked *

*

If you have a question about your site specifically or a general question about search, your best bet is to post in our Webmaster Help Forum linked from http://google.com/webmasters

If you comment, please use your personal name, not your business name. Business names can sound salesy or spammy, and I would like to try people leaving their actual name instead.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

css.php