Bigdaddy on the move

Executive summary: if you want to play with a Bigdaddy data center, hit instead of

Longer explanation: I spent this morning writing posts about url canonicalization, inurl:, and 302 redirects so that everyone would be well-versed to evaluate Bigdaddy. In the post about 302 redirects, I even mentioned a search [sf giants] that Bigdaddy does better on than the current Google system. Then I got this comment by Michael Weir:

fascinating post – thx Matt! Although, at the time I post this bigdaddy was returing as #1 when querying “sf giants”. I don’t think bigdaddy likes you. 🙂

I think my mental thought was “Huh. That’s not good.” I walked down and talked to the Bigdaddy folks, and sure enough: has been taken down for testing. Note that if you go to, you’ll still get a Google search box–it will just be regular Google, not Bigdaddy.

So it looks like is the best IP address to use when testing Bigdaddy. That data center should be showing Bigdaddy results more reliably. Also, it looks like [sf giants] is a fine query to see if you’re hitting Bigdaddy. 🙂 If you get at #1, you’re searching Bigdaddy. If you get at #1 and an uncrawled url at #3, you’re hitting the older Google infrastructure.

39 Responses to Bigdaddy on the move (Leave a comment)

  1. One question, with the dissatisfied submissions still stand for both data centers? I just sent several and would hate to have to do it all over again.

  2. that do not believe you nevertheless probably?

  3. I did a search for my and its not showing up in this new update. Is there a reason for this? It shows up in the search for my main keyword on page three or four.. so I dont know whats going on but id like to fix it if there is some problem.


  4. damnit… I just finished a big long rant in the other post…

    Oh, same applies, my fictitious scenario still produces really crappy results… from an end-user’s perspective…

  5. Matt, thanks for loads of great info today, it’s 2am in Spain now and I’m totally wide awake after reading your posts!

    I have a question regarding BigDaddy which is not something I can put as a spam report. I noticed that the number of indexed pages has reduced dramatically when doing the site: command. Is this because the index is incomplete or is that it?

    I have made a right hash of 301s over the past few months so I was expecting the number of pages to reduce, but not by so much!

    Also, do conical…whatever… issues with supplementals need to be reported too?

    Thanks. Really must get some sleep now.

  6. Matt,

    This may be off subject but I had a question about the Google toolbar and PR. I have heard that the toolbar is not a completely correct ranking. I know it is not very high, but sometimes I show a 2 PR on my index page and sometimes I show a 0. My question is how do I get a ranking on the other pages. I am new at this and have built my first website. I’m basically feeling my way through as I go. Have I done something wrong? The other pages have more articles and information on them but none of the others show a ranking. I am still adding content to rank higher. I have seen this on other websites also. Sorry for the bother but any advice will be greatly appreciated.

  7. Bigdaddy is not smaller, Justin, so I wouldn’t expect the number of indexed pages to go down by much. It could have been caused by putting 301s in place..

  8. Big Daddy is Awesome. Congats!

  9. I love the results on! Obviously, my site is doing much better at number 3 for “sarasota real estate”. 😉 In my opinion, 8 out of the first 10 results are very good websites that any searcher would find useful. Will bigdaddy move over into the other data centers?

  10. From what I have seen from the searches I did there seems to be a lot more non-quality sites. Hopefully they will disappear soon. Sites I also would like to be gone are site that only have Google Ads and nothing else but that’s another story 🙂

  11. If I am understanding this correctly, I dont think it is working. When I check my site www and non www I show an increase of 3000 pages for the non www and yet the www still has most of the same pages cached as non www. Is this suppose to work this way or wasnt BigDaddy suppose to fix this canonicalization issue?

  12. I’ve recently started to use the Apache Multiviews command to allow me to omit file extensions (.html, .htm, .asp, etc.) on my site, as the W3C and others recommend. (Not that I understand what I’m doing when I cut and paste stuff into .htaccess, but anyway…) It certainly does make the URLs look a lot cleaner and more user friendly.

    Matt, if you have any thoughts on this practice in re: canonicalization I’d be glad to hear them (i.e. = I haven’t seen it mentioned anywhere yet.

    Many thanks for all the interesting material you post – I hope Google appreciates the service you’re providing.

  13. Marc, I do expect Bigdaddy to be brought up at other data centers eventually.

    George, fill in a spam report on the sites that you think are poor-quality or spam. Earlier is better than later when giving feedback. 🙂

    Wayne, you may want to fill out a report too, but on the dissatisfied link and talk about how we should combine your www and non-www hostnames.

    RJO, I personally lean against omitting file extensions. But that’s just because I as a user prefer to know whether I’m about to click on html vs. a cgi script vs. a PDF, etc.

  14. Gheesh, how much coffee is Matt drinking these days? 😉

  15. Re: spam

    What about sites that were previously reported, but still there? Do we re-submit those?

  16. ok, thats a bit offtopic but here it goes…

    I am based in Germany and when i enter i get to a german based Google site, however i do look for english phrases, are those that i see exactly the same a person from the USA would see or is “my” result because of the german language preference different, meaning is the whole BigDaddy ranking just for USA based people?


  17. RJO, I personally lean against omitting file extensions. But that’s just because I as a user prefer to know whether I’m about to click on html vs. a cgi script vs. a PDF, etc.

    I can certainly understand that preference. My sites are pretty simple static pages with almost no scripts – I’m mainly trying to chop five characters off the length of the URL and making them look less crufty, following Tim B-L’s old advice. But, I never delete the extension from URLs that point to .pdf files, and in fact always indicate in the page text when a link points to a .pdf: that’s always good usability. (Nothing I hate more than clicking a link and then groaning when I see my old slow eMac cranking up Acrobat Reader….)

    If my site disappears from Google after I delete all the “.html”s I’ll let you know. 😉

  18. Gheesh, how much coffee is Matt drinking these days?

    None, but as soon as US Congress is done testing every athlete in the world for steroids, they may want to have our boy pee in a bottle and check for methamphetamines. ;D

  19. Hello Matt! I tell you from Spain…When do we will see the changes of BigDaddy in Spain? Thanks.

  20. Is this new infrastructure ‘Bigdaddy’ a covering data from ?

    I dont mean this as a jab, I hope it does as my site will gain from it.


  21. Justin, just under 50% of my site has dropped off when using in bigdaddy but I see this as a good thing.

    Looking at the results, the pages that have been dropped are the “provide feedback” pages that really only differ in title information about what you are providing feedback on. While these are valuable pages when you are on the site they shouldn’t be the users first jumping off point from search results as they don’t know what they are providing feedback on. So to me this is a good drop, perhaps you are experiencing something similar?

    (I also used sitemaps to drop the priority of the “provide feedback” pages down very low as some of them were ranking higher than the actual pages in the regular Google results! Don’t know if that has any relevance.)

    So far the results are looking much much cleaner…

  22. mmm – why is the cached version, of my site, older on ‘big daddy’ than on the previous Google datacentre – I’ve gone back from Dec 25th to Dec 22 – and less clean pages indexed.

  23. Interesting… our RSS feeds are showing up in the normal SERPs in BigDaddy, but as “Unrecognized” file format (ie followed by “View as HTML”). I presume Google are planning to add a link to their feed reader for these – any info on what versions of RSS they’ll support?

  24. Matt,

    My homepage is located on a sub-domain ( but google has always shown as the homepage. I have a PHP Location: redirect to from Should I help google out and change all the links to and just let them redirect, or just leave it as is?

  25. @joseph

    I think you should change bigdaddy-url from hl=de to hl=en to see the correct serps in Germany for english phrases. Matt is it correct?

  26. Makes No Difference

    Matt Cutts,

    I posted this before, and I did tell you that if you keep pretending to be a god you will soon realize that you are only one report away from CNN or Fox news channel to having your stock go down to almost nothing per share. You are still moving forward by removing people’s sites from your search engine. You must either be full of false hope, or just feel like Google is a god. In either case, you are so off base.

    People who have the word of the exact product they are selling on their site in hidden text in an effort to keep the site aestheticlly pleasing are being removed from your search engine. You want to force everyone on the planet to buy sponsored listings. That is you plan. There is nothing more to it. Well, it is time I fill out a report with some news media partners of mine about how much a click costs. Do most people know how much a click on a sponsored result costs? As you well know Matt, they most certainly do not! However, after my report, most people will know how much a click costs, and then we shall see how many people actually click on a sponsored listing. Do you predict a reduction of 50%? I predict more! In addition, do you predict the federal government will force you to be honest about your advertsing and post your per-click fees right next to your ads? I do:-)

  27. Is BigDaddy still live on those datacenters? They appear to currently be serving “regular” results.

  28. What? Is bigdaddy .. I don’t even have a word for it… But, as you move around the pages in the listings for a search the SERPS *changes* (!)

    I’m just investigating a search for a site with some problems and the first time, the problems started at page 3 – after looking a few pages forth i returned to page three, and then the problem strated at page four. Now it’s at page five – no, damn… page 14!

    And before the SERPS only went to page 11 before supplementals kicked in. Matt, what is this? It seems to be alive?

  29. BigDaddy Check- It seems another way to confirm you are using BigDaddy at is to have easily visable Google Search Preferences set.

    For example, by default on I receive 30 search results. On BigDaddy I get the default 10 results, which seems to confirm I have not been redirected back to

  30. Hi Matt,
    Thanks for the interesting news about BigDaddy. I have been playing around with “him” a little bit and as far as I can see my site(s) are better ranked than in the “old” Google SERP’s. The results are not consistent (yet?) because on one search you can show up on first page and when you hit the search button again you might be on second or even third page. Not sure if it is supposed to be like this but that’s what I see.
    I have sent spam reports to Google using the Google Ads Report a Violation. Not sure if that’s the way to go but 99.9% of the sites have Google ads on them. I can’t tell you what site it is (No URL’s here, you can email me if you want a list of them!) but if I go and search for one of my URL’s I find hundreds of pages that use my URL and text from my site (and of course rel=nofollow) just to get hits using my popular site(s). A lot of them are crappy scraper sites and they live off of Google Ads stealing traffic from other more quality sites. I am hoping that Google look at the spam reports and remove the sites or at least stop them from making money and they’ll eventually disappear.

  31. Google BigDaddy is an SEO honeypot. Don’t get caught!

  32. Hi Matt,

    Thanks for the information on BigDaddy. I’ve two comments / questions:

    1. Yesterday I was trying out and found with a site search that my site ( seemed to have been crawled a lot more thoroughly on BigDaddy than on usual Google – it was showing up about 30 or 40 pages, instead of the usual 1. I’m not worried about the number of pages on normal Google, because the site has only been up a short while, and we’ve not attracted many links yet – I was pleasantly surprised that BigDaddy had indexed it so well. Today I clicked back on to and I discovered that was now returning just one page (same as normal Google). Is that IP address still good for BigDaddy? If so, why would I see this yo-yo-ing of numbers of pages indexed? As I say, I’m not worried about how the pages are ranking just yet, but I’m intrigued as to whether they’re getting onto the indexes at all 🙂

    2. Not exactly about BigDaddy, but it’s come up in relation is the issue of Duplicate Content. For usability reasons we have put up a series of information guides, both in a ‘generic’ section ( and a location-specific section ( There is some specialisation of the “Nottingham” guides, but the majority of the content is the same. In future we may extend the site to cover other locations, and I would expect that, again, the bulk of their information guides would be duplicates, with some specialisation. My question is – will this hurt us in the Search Engines, and if so, what is the recommended solution?


  33. In addition, do you predict the federal government will force you to be honest about your advertsing and post your per-click fees right next to your ads? I do:-)

    And I predict it’ll be a cold day in Hell before you realize the nationalistic ignorance of this statement.

    You seem to forget that a law like this would never fly because there are a whole bunch of other countries outside of the US (forgive me if you’re not American, but I haven’t ever seen a non-American utter a statement like this) that would have to adopt the law as well.

    And since does reach those countries and will be shown in those places where geotargeting of an IP does not resolve it to a country (e.g. AOL), it isn’t going to happen.

  34. Hi Matt,
    I’m noticing a huge drop in the number of pages indexed on BigDaddy vs. the other datacenters as well. The other datacenters are showing 16,300 pages indexed while BigDaddy is only showing 515 pages. There have been some 301 redirects set up recently but not enough to warrant a drop like this. Meanwhile our rankings are really good on BigDaddy so it leaves me confused when you say that the index is not smaller.

  35. It looks like we have a major rollback in progress, including

    What was once old, is new again!

  36. Matt, when is Big Daddy going to go live?


  37. Is there a tool or a list somewhere so that we can test different data center results for the same search. For example, what does a data center in Paris return and what does one in Boston return?

  38. Hey Matt,
    I have a discovered some confusing searches. Now, forgive me if this was posted elsewhere as I’m kinda new to posting in forumns. I’ve had some questions/issues with having pages dropped. I had started using Sitemaps and noticed 1 day we had an SSL issue. I had some 401/407 errors. The issue was resolved that day but then noticed shortly thereafter that we lost around 22 pages that had previously been ranked/listed well. i.e. Those pages listed with the 401/407 error.

    1 of the pages that we had been dropped for suddenly appeared (1 month later) in a natural Google search today. Great! In trying to research our site status on Google I stumbled upon a post about “Big Daddy” and it’s testing IP addresses when it was still being tested. and I entered them into my browser and brought up Google. I did a search for some pages that were dropped and there they were! Go back to the natural Google search and Poof! not there!

    A couple terms we have had at #1 & #2 for years now suddenly is gone. Yet, I see them under the IP addresses listed above. Has the old content and Big Daddy not caught up yet? What gives? For the longest time I’ve had around 9800 pages listed but now down to 7150? Big Daddys IP from above listes us still at 9800. Big Daddy is Live now I believe so which results are current? Any help/info would be Absolutely appreciated

  39. I couldnt load, is it something to do with my network connection or has the IP now changed?