Catching up

I’m still way behind on my email and blogreading, but I’ll go ahead and mention 2-3 things that I’ve come up to speed on.

I was happy to see that by the time I heard of some issues, they were already resolved. On June 16th, Matt Mullenweg posted that he’d been banned from Google. Happily, Matt kept updating the status as he learned more. It turns out that someone had uncovered Matt’s password by scouring the source code for a new project Matt was working on. The bad guy flipped on a privacy feature on Matt’s blog that added a “noindex” meta tag. And we know what the noindex tag does. When Matt figured this out, he removed the noindex tag and he’s back in Google now. In general, if your server is down for a few days and Googlebot can’t crawl your pages, those pages can drop out of our index. But when the pages are alive again, Google will often find the pages quickly and you should usually return to where you were before.

Ruslan Abuzant noticed what looked like a fragment of a server status page. He posted over at Digital Point Forums, and people there debated if the fragment was real or not. Yes, it was real. No, I’m not going to comment on what any of it means. 🙂 Folks have taken steps to keep it from happening in the future, but personally, I think that we need to start including some extra settings for fun. I’d say that we should add

–initial_time_travel_wormhole=”Wednesday, December 31 1969 11:11 pm”
–use_googlepray=false
–docid_size=more-than-four-bytes
–SETI_alien_communication_port=31337
–skynet_sentience=0.33
–plane_load=snakes
–pigeonrank_seed=42
–use_mentalplex=true
unicorn_versus_werewolf=its-on-now

Let’s see, what else. When I saw the obligatory “Google found data we didn’t want indexed” article that I missed while I was gone, I almost didn’t bother to ask around. Barry covered this story pretty well when he noted that Googlebot doesn’t go around guessing passwords. I assumed that someone left the information lying out somehow, or that there was a hyperlink out on the web with a username and password embedded in it. When I had a chance to talk to a colleague back at Google, I got a little more info though. He said he didn’t mind if I reprinted what he found:

The URL was on a server that this school district thought was password-protected. Before they took down the server, I was able to retrieve the live URL. I was getting a username/password login page with a regular Firefox user agent, but I got a server error when I changed the UA as Googlebot. I changed back to Firefox and was able to retrieve the username/password page again. It seems their document system was cloaking to Gbot, likely unbeknownst to the people who are writing us now and requesting the removal.

That’s my best guess for how the information got into Google. Of course it’s a moot point now because the urls are no longer in Google. But that’s what prompted me to write a short/sweet “How to herd Googlebot” post. If you administer a web server which has information that you don’t want to be public, it’s easier to exclude content in advance than to try to remove it from search engines later.

Again, I’m still catching up, but I’m planning to discuss at least a couple more things that I missed while I was on vacation.

65 Responses to Catching up (Leave a comment)

  1. A couple things come to mind about your responses:
    1) If a site gets banned, there is always the old reinclusion request. I have been successful on both attempts. I never figured out exactly what went wrong, but I got back in. I wonder if there is ever a circumstance these days where there is a reply back to a reinclusion request when circumstances require the webmaster be notified, such as he is accidently using the noindex metatag?
    2) I have found google indexed content in the past that I didn’t want indexed. I removed it! 🙂

  2. –pigeonrank_seed=42

    Instant flashback to http://www.google.com/technology/pigeonrank.html lol I always thought there was something to that April Fools bit… LOL

  3. Are you allowed to comment on This…. 😕

    blog.outer-court.com/forum/43895.html

    and of course ADAM responded to the “Spammer gets Billions of pages on Google’ controvery, but is there anything else that could be shared of a more technical nature 😕

    merged.ca/monetize/flat/how-to-get-billions-of-pages-indexed-by-Google.html

    BTW: 🙂
    Miss kentucky/USA arrived yesterday in LA for the start of Miss Universe
    We got a chance to see her

  4. Search Engines WeB, the “5B” spammer is one of the issues I intend to talk about; I’m still digging into a couple side-topics about that. The suggestion for [therapy products] is interesting and I asked someone about that, but it’s also just a suggestion for a query refinement, so that one bothered me less.

  5. I still would like to know what happend on june 27.

    Can you Matt, please explain why thousands
    innocent sites (it is not 5 or 10) has been affected by june 27 and thats are usually sites older than 4-5 years…that never ever before has been affected by any google changes, didnt change anything and still dissapeared on 27 june.
    I think that it is at least questions of morally responsibility to explain us why.

    thanks

  6. When are we going to address the most important Google-related issue of all?

    No, it’s not spam.
    No, it’s not search engine quality.
    No, it’s not size of the index and storage capacity.

    IT’S THE OTHER FREAKIN’ ADAM.

    I got here first. Make him change his name, Matt. He’s your lackey now. So I say we call him…Joey Joe-Joe…Junior…Shabadu. Yeah, that’ll do. 🙂

  7. How about if we make him go by Adam Lasnik? 🙂

    Sharon, I talked about that topic with Bontar and Arubicus starting here:
    http://www.mattcutts.com/blog/reminder-check-your-sites/#comment-44861
    That would be the better place to talk about it compared to here.

  8. Hi Matt,

    One of my sites was down for several hours (actually the SQL server was down and it was producing an sql server error page), because of my bad luck it was cached during that time by Google. Next thing I know is that I lose almost all pages and the remaining pages turn supplemental. Googlebot used to visit daily before that and crawled quite a few pages, now I am lucky if it visits every couple of days and if it crawls more than 3-5 pages.
    Should I just wait and expect things to return to what it was before?

  9. hi,
    I have a corporate web site which I have recently moved to wordpress (because old site used CMS which was a disaster to search engines).

    Now I have moved web site, included it into sitemaps program and while I have created valid sitemap I get ERROR1 (dns timed out), I have checked and rechecked my URLs and everything is OK, DNS was not touched for years…
    Sitemap site tells me that last sucessfull googlebot visit was 15th june

    I was thinking that something happened while I was changing sites and that it will be all resolved within few days, but ERROR is still there after 15 days and google cache returns old URLs.

    my question would be;
    a) how can I more precisely learn what this ERROR means (I do not understand it, everything works fine for me) and how to resolve it
    b) maybe it could be smart to put some form on sitemaps site where you can signal to googlebot that old site is flushed and replaced with something completely different
    c) maybe people from sitemap could put some “contact us” form for queries like this

    any suggestion would be helpful

  10. Would be curious to know what you say on this story — New York Times and Cloaking:
    http://blog.outer-court.com/archive/2006-06-19-n18.html

  11. –use_googlepray=false
    –SETI_alien_communication_port=31337
    –skynet_sentience=0.33
    –pigeonrank_seed=42
    –use_mentalplex=true

    LOL 😀

    Matt, can you comment at least on the experimental spam mention ? It will help us a lot if you could point at least a hint.

  12. Anybody know why a site will not appear using the site: command, yet it has PageRank and at least the main page – and often several internal pages – is/are in the cache?

  13. Cristian, sorry that I can’t talk about any of the settings. Philipp, I haven’t had time to come fully up to speed on NYT yet. Jim, usually you just need to wait for a few days for them to show up again where they were. I’d check to make sure that you’re showing the original pages you were showing, just to be safe. If there’s still any SQL errors, we might not have enough on-page content to be ranking you where you used to be.

  14. William Donelson

    Our sites have not been crawled by GoogleBot for four months now for some reason… And we are seeing DMOZ titles listed in Google rather than the actual title pages, e.g. for “taj mahal” for our Explore the Taj Mahal site.

  15. Christian, i’m not going to speculate on what the experimental spam thing is, (although I’d guess it has to do with the recent trustrank paper)

    what I find more interesting is
    sections_to_retrieve=body+url+compactanchors

    You’ll notice it says nothing about title or meta tags ..

  16. You’ll notice it says nothing about title or meta tags ..

    Meta info can just as be included in that “body” mention. For that piece of code, body could mean all the HTML in the page.

  17. it “could” but that would sorta violate an engineering thought process, as body already means something. Engineers like to call things by what they are, and they will argue semantics.

    anyway, just my 2 cents.

  18. I guess your version is more accurate yeah.

  19. William Donelson, is it possible that you’re looking at old bot IP ranges? Looks like for your home page we last fetched it on July 6th, which is pretty recent.

  20. Dave (Original)

    Philipp Lenssen, while I would like to see Google mark these pages as needing subscription in the SERPs, there is no cloaking going on.

  21. Hi, Matt:) I have the same problem like William Donelson – my home page is showing up with title from DMOZ (very old). Please, tell me what can I do & is something wriong withh my site?

  22. William Donelson

    [quote]William Donelson, is it possible that you’re looking at old bot IP ranges? Looks like for your home page we last fetched it on July 6th, which is pretty recen[/quote]Thanks Matt, but my point was we don’t see any Googlebot accesses in our site logs…

  23. William Donelson

    And why are we seeing the 5 year-old DMOZ title for our websites?

  24. And why are we seeing the 5 year-old DMOZ title for our websites?

    It’s because Google uses DMOZ’s title and description of your website William. You posted this issue in other places right ? I think I answered you in some other places too.

  25. Dave (Original)

    Don’t worry. DMOZ only takes about 2-5 years to get an edit done to an existing site 🙂

  26. Aww Matt, surely the very least you could do is tell us what –use-borg means :):D

    (by the way, a preview button would be nice, and your comments doesn’t work with HTTP referer disabled, which some people do for privacy).

  27. Matt,

    This is somewhat off topic and I may have missed it, but are you going to be speaking at SES next month? If not, will you be there “mingling?”

  28. I’m still trying to figure out why, on some DC’s, the site: command shows pages for my site that have NEVER existed before… they are even cached!

  29. William Donelson

    Cristian, I am asking Matt about unchangeable, unfair and WRONG DMOZ titles, NOT YOU.

  30. > William Donelson writes:
    >
    > Cristian, thanks for your reply on Matt Cutt’s website, but NO THANKS. I would like to make a point to him, and you are raining on my parade. Do you always make it a point to meddle in other’s business? I very much resent your interference.
    >
    > For 5 years, Google showed OUR title; I do not like and CANNOT CHANGE the DMOZ title.
    >
    > I would like an answer or acknowledgement from Matt, NOT FROM YOU.
    >
    > Sincerely
    > William Donelson

    As I told you in my reply to your e-mail, Matt is not responsible for the wellfare of anyone’s website.

    People like you deserve no real help from anyone.

  31. Hi Matt

    Well, I’ll have to repeat my self on this one, but my site went out of the serps again. Now, I know that this isn’t such a big event for Google, but that was the single one I had.

    Curious, as far as BigDaddy is concerned, my site was up on the first page for quite a few keywords, that until last week when it dropped to the last page of the search.

    Not sure if you’ll have the time to look over, but I thought to write about it here, it’s more like a consolation…

    Thanks for your time and keep up the good work,

    Chris

  32. William Donelson

    Cristian, are you speaking for Matt now? Is he aware you have his power of attourney? Or are you just a meddler?

  33. ** what I find more interesting is
    sections_to_retrieve=body+url+compactanchors **

    I assume that will be:
    body: the title and snippet shown in the SEPS.
    url: the URL shown in green in the SERPs
    compactanchors: the links to “cached” and “similarpages” in the SERPs.

    The retrieve is the retrieve from Google’s database to display the entry in the SERPs. It is not talking about any sort of GoogleBot retrieve action from your sites.

  34. Matt,

    What does it mean when you go to look at a cached page, and the google cached page is blank but all the info is still at the top?? I have been seeing this rather frequently since the Big Daddy update with a bunch of our pages. Any light you can shed on this would be helpful.

    Thanks

  35. Dave (Original)

    William, I doubt Matt will answer specific questions on specific sites. You will likely find that the DMOZ title will drop soon. I have seen it happen many times.

  36. William Donelson,

    I don’t know who you are, and I don’t know all that much about Cristian either. But from what I’ve seen of Cristian, he’s got others’ best interests at heart, including yours…initially.

    But having seen your responses to his, and the way you’re talking to him now, I have three questions for you:

    1) Who are you?

    2) Why should Matt or anyone else answer any of your questions if you’re going to treat them like crap?

    3) Why should Google or anyone else give a rat’s ass whether or not you like your DMOZ title or description and whether it should be used anywhere? (Note: I’m not condoning anything DMOZ does…I’m just pointing out that if I were Google, I wouldn’t exactly be rushing to help you right now.)

    All Cristian was doing was trying to help and you trashed him because “he’s not Matt”. So what if he’s not Matt? There’s only one of him, you know. Take advice from wherever you can get it and say thanks, even if you don’t like the advice.

    Not cool, dude.

  37. Chris Said,
    July 10, 2006 @ 12:56 pm

    Hi Matt

    Well, I’ll have to repeat my self on this one, but my site went out of the serps again. Now, I know that this isn’t such a big event for Google, but that was the single one I had.

    Curious, as far as BigDaddy is concerned, my site was up on the first page for quite a few keywords, that until last week when it dropped to the last page of the search.

    Not sure if you’ll have the time to look over, but I thought to write about it here, it’s more like a consolation…

    Thanks for your time and keep up the good work,

    Chris

    Looks like Pornstar SEO Week on Matt Cutts’ Blog continues. But it’s all from the same network as BrandiBelleLive.com (The Bang Bros Network). It’s a good thing too, because this is becoming quite the circus act (especially the way Brandi juggles balls. 😀 )

    For those of you who would like to see proof, no problemo. BrandiBelleLive.com links to BrandiBelle.com in numerous places. Click one of the BrandiBelle links and scroll down to the bottom (where the footer links are.)

    Now, on to our boy Chris’ site. Click in, and then click the “Bang Bros Network” picture link at the top of the page (the 5 x 2 picture grid in the top left). Scroll down to the bottom of the page and look at the footer links.

    Anyone see a pattern developing here? 🙂

    The only thing that I haven’t figured out yet is if Brandi Belle and Chris are promoting doorway sites (which I’m leaning toward personally) or they’re two independent affiliates of the same program. Either way, they’re tied into this Bang Bros Network thing, and are showing quite the set of stones posting on Matt’s blog.

    (Side note: I’m also giving the benefit of the doubt and assuming they’re two separate people, although they may well be one and the same.)

    All joking aside, Matt: I’m not trying to tell you how to run your blog or anything. I’m only asking this question in the interest of not seeing it shut down because a couple of purveyors of adult material happened to use your site to link to theirs. But wouldn’t the links to stuff like the Bang Bros stuff be a violation of your hosting ToS and potentially get your site cancelled? I think the vast majority of us would hate to see that happen.

  38. William Donelson

    I have been discussing Google’s use of DMOZ titles with others, and because the DMOZ title is WRONG for our site, they agree that Google is, in fact MISLEADING its viewers by using it. I never submitted the DMOZ-shown title to DMOZ; it is an uncorrectable mistake by DMOZ.

    We produce ONLINE virtual tours, but the DMOZ title that Google uses makes our virtual tour site look like an airline or ticket-sales agency or non-virtual tour guide operator.

    This means that those who want to “virtually” visit DO NOT see that that is what we offer.

    Does anyone know a way I can get our site De-Listed from DMOZ, or some such? We really want an Accurate Google SERPs title for our site…

    HELP!

  39. [quote]The Adam That Doesn’t Belong To Matt Said,
    July 10, 2006 @ 11:27 pm

    Chris Said,
    July 10, 2006 @ 12:56 pm

    Hi Matt

    Well, I’ll have to repeat my self on this one, but my site went out of the serps again. Now, I know that this isn’t such a big event for Google, but that was the single one I had.

    Curious, as far as BigDaddy is concerned, my site was up on the first page for quite a few keywords, that until last week when it dropped to the last page of the search.

    Not sure if you’ll have the time to look over, but I thought to write about it here, it’s more like a consolation…

    Thanks for your time and keep up the good work,

    Chris

    Looks like Pornstar SEO Week on Matt Cutts’ Blog continues. But it’s all from the same network as BrandiBelleLive.com (The Bang Bros Network). It’s a good thing too, because this is becoming quite the circus act (especially the way Brandi juggles balls. )

    For those of you who would like to see proof, no problemo. BrandiBelleLive.com links to BrandiBelle.com in numerous places. Click one of the BrandiBelle links and scroll down to the bottom (where the footer links are.)

    Now, on to our boy Chris’ site. Click in, and then click the “Bang Bros Network” picture link at the top of the page (the 5 x 2 picture grid in the top left). Scroll down to the bottom of the page and look at the footer links.

    Anyone see a pattern developing here?

    The only thing that I haven’t figured out yet is if Brandi Belle and Chris are promoting doorway sites (which I’m leaning toward personally) or they’re two independent affiliates of the same program. Either way, they’re tied into this Bang Bros Network thing, and are showing quite the set of stones posting on Matt’s blog.

    (Side note: I’m also giving the benefit of the doubt and assuming they’re two separate people, although they may well be one and the same.)

    All joking aside, Matt: I’m not trying to tell you how to run your blog or anything. I’m only asking this question in the interest of not seeing it shut down because a couple of purveyors of adult material happened to use your site to link to theirs. But wouldn’t the links to stuff like the Bang Bros stuff be a violation of your hosting ToS and potentially get your site cancelled? I think the vast majority of us would hate to see that happen. [/quote]

    Hi

    I’m afraid I really don’t see your point here. the true is that I’m promoting that network you’re talking about, but I don’t have any connection with the other guy.
    As for the links: I inserted many comments here, most of them without my website in it. I placed my site inthere just because I wanted Matt to take a look at it, he is usually posting here if something’s wrong with it.
    Also, all links from here are nofollowed (I think). Also, inserting comments to promote your site means spam, and I DON’T DO THAT.
    Also, clicking the image you’re into will lead you to the affiliate site, on wich I don’t have any power in modifying or else. Also, the link to that site is nofollowed.
    My site isn’t a doorway, it’s a site where people can find more info on the sites they’re going to.

    Thank you,

    Chris

  40. William Donelson,

    You’ve probably already got this link somewhere but:

    http://www.resource-zone.com/

    It’s the DMOZ forum. I don’t know where specifically you can get your title changed (if at all…I’ve never had a reason to use it), but it’s worth a shot.

    Chris:

    While the links are nofollowed, there’s still the issue of Matt potentially trying to keep this a PG-13 blog (as it appears he’s doing). In other words, my comments had non-search-engine reasons. It appears that you didn’t notice or realize that actual human beings click on those links and may discover something they never wanted to (not that a little adult entertainment bothers me personally, but I suspect out of the thousands that read and post to this thing, there are some who would be offended).

    The nofollow attribute isn’t relevant here simply because my greater concern was of people visiting from Matt’s blog with no prior warning. You may have wanted Matt to have a look, but in the process other curious types will have a look too.

    As far as whether or not you’re related or connected to the other person, that’s gonna be tough to pass off if it is true. You’re both promoting the same thing using the same affiliate marketing tricks. (By the way, part of your answer may lie in that….for more information, read some of the other posts here. If you’re looking hard enough, you’ll find it.)

  41. The Adam That Doesn’t Belong To Matt:

    Concerning other people that might hit my site in the process: you are right, Matt, please delete that damn link. I’m verry sorry about that, but I do not know any other way to hmm, show my site in case someone might take a look and tell me what’s wrong with it.

    Second, what marketing tricks are you talking about? I may be stupid, but I really don’t understand that remark, could you please elaborate a bit? Also, if this isn’t the place for it, please contact me over email, romaniucl at gmail.com, I’d be verry happy to understand and remove such tricks.

    Note: I DID read all posts here.

    Thank you,
    Chris

  42. William Donelson

    Adam that doesn’t…

    Thanks for that forum reference. I am trying to withdraw from the ancient-history DMOZ listings, but given that it’s lost in the past, I don’t know whether that’s possible.

    I guess it’s like AOL — you need the attourney general of New York to get any attention…

  43. The Adam That Doesn’t Belong To Matt….

    I don’t mean to insult you but I think I saw your name in this forum too much.. and now when you acted as a cop I think somebody needs to remind you that this blog belongs to Matt Cutts… If you love blogging I guess the best thing for you to do is to open your own blog…if you are trying to be Matt’s favorite and earn some points…go ahead. It just came to a point when I don’t read your posts anymore. By the way you have a nice website I guess that if you tunnel some of the effort you invest here it can be even better.

  44. Did no one click on the werewolf vs. unicorn page?
    http://topatoco.com/itson.htm
    Lisa Barone, you don’t think that’s funny? I dare you to find a funnier T-shirt.. 😉

  45. Nice find Matt, too bad I’m so far away to get it shipped…

  46. What’s so funny about it ?

  47. William Donelson

    Well, we have finally spoken with a DMOZ editor, and he has point-blank refused to put “virtual tour” in our site title there, in spite of the list of 50+ other sites that DO have that in DMOZ, ….

    AND, he has refused point-blank to de-list our site from DMOZ. No officer of the company agreed to their terms, so the contract is void, but even that does not impress them.

    So, we are considering (a) refusing incoming links from DMOZ, or (b) even pulling the entire site offline, getting a new domain, and starting over.

    (Comments please)

    Hardly seems fair, does it? But its “their clubhouse” and you are not allowed out of their prison, especially if you ask nicely.

    Grrr…

  48. William. Please go and post you thoughts and ask comments in another blog. Like in your own. Or In Digitalpoint. There are a lot of DMOZ editors there than can maybe remove your website from DMOZ, if that’s what you are after.

    Matt can’t help you man.

  49. William Donelson

    Cristian,

    The point is: Google is using DMOZ titles, which is causing us harm. So much harm that we may take our sites offline, and Start Over in the whole SEO endeavour.

    Matt should know that for Google to use DMOZ titles is a capitulation of authority, a cheat, and a failure of good practice.

    If Google were not using the bad, 5 year-old DMOZ titles, then I would not be posting here at all !!!!

  50. Dave (Original)

    William, is this title showing in the organic SERPs? If so, how long has it bee showing.

    BTW, no surprise about DMOZ holding a site hostage

  51. William: I’m sorry that didn’t work for you. I’m not surprised in the slightest, but I do apologize if you feel you wasted your time. Unfortunately, that’s DMOZ. As flippant a remark as that sounds, it’s the sad reality of the situation.

    I’m not totally sure William’s asking for Matt’s help in getting his title specifically changed. I think William’s asking for Matt’s help in eliminating DMOZ listings from SERPs in general based on the general inflexibility and despotic nature of the DMOZ editors.

    Assuming that’s the case, I’d tend to agree with the guy. They’re basically playing by their own set of rules, which is their right to do as the owners/operators of the site. We all do that. I don’t have a problem with that in and of itself.

    What I have a problem with is that they haven’t lost perceived relevance while their actual relevance has gone down. Their search function doesn’t work, it takes at least 5 levels to find something relevant, and the editors seem to have formed a “mob mentality” as most volunteer-run sites do. If the directly is useless to the general public, then why should it be given any more weight by big G or anyone else?

    Kind of interesting that some of the DMOZ cats appear in Google’s Supplemental index now. Just today, a client and I were looking up some things and I hit upon this purely by accident. Here’s an example (although not the one that caused it, it seems to be fairly illuminating):

    http://www.google.com/search?q=Toronto+business+consulting+site:directory.google.com&num=100&hl=en&lr=&rls=GGLG,GGLG:2006-19,GGLG:en&as_qdr=all&filter=0

    I wonder aloud what caused that.

  52. William Donelson

    Adam that doesn’t belong to Matt –

    Thanks. You are EXACTLY right. I cannot believe that Google, a ga-zillion dollar company, takes the word of a bunch of DMOZ amateurs!

    I forum-spoke with around 12 of them last night, and they were a pretty confused bunch. I had to re-explain the situation to them at least 25 times.

    This morning, some were suggesting that I change the GRAPHIC on my site (not the page title) to something else, and some were disagreeing with that. No one is in charge, there’s no appeals process, and you’re lucky if they don’t screw you up completely.

    On the bottom line, I will probably use .htaccess to “deny from dmoz.org (IPs)” to kill the links on their site. I cannot believe it, but some of the DMOZ editors said that even if my link was dead on their site, THEY’D KEEP the listing anyway, just to annoy us!

    So, Google should (in a million years) use ANYTHING from a bunch of rank amateurs like DMOZ.

    Why do they place semi-infinite trust in Unknown Amateurs ?

  53. Well, alright, DMOZ might be a “miserable failure”.
    http://www.google.nl/search?hl=nl&q=miserable+failure&btnG=Zoeken&meta=
    😉

    How about some real stuff now, like search engines policy on domains issue?
    Anyone… ?

  54. William Donelson

    Just announced today — AVOID DMOZ TITLES

    http://sitemaps.blogspot.com/2006/07/more-control-over-page-snippets.html

    THANKS Google Guys !

  55. While I dig the tag and the idea behind it, it does lead to one question:

    What would happen to the directory.google.com listings which had this tag on their sites? Would they still use the DMOZ titles and descriptions or would they use the on-the-page titles and descriptions?

    Great idea, though. Friends don’t let friends use DMOZ.

  56. Cristian and William,
    Thanks for the links and a BIG THANKS to Google for this option. I hope the other SE’s honor this as well. To me it was a HUGE competitive disadvantage to see obsolete site titles and descriptions (From DMOZ) appear on our listing next to what appeared to be more relevant and up to date titles and descriptions of other sites appearing in SERPs.

  57. The Adam That Doesn’t Belong To Matt,

    I don’t understand your comment about the Google directory. It might be that I don’t understand the supplemental index or how you can tell what’s what.

    OT I really like this blog. I find it interesting that folks use it as a forum but I guess I’m contributing to it.

  58. I’m not really clear why I got this sudden motivation to, but I looked at the output, and if any of http://ofb.net/~wtanaka/node/221 is true, most of the error doesn’t even seem that interesting.

  59. Gareth Alexander

    With Google offering images, groups etc, it seemd that they can sort by almost everything now, why don’t they allow you to sort by the type of information you are looking for.

    E.g. I was looking for troubleshooting information about a printer yesterday
    and had to refine my search 5 times before I got any relevant information.

    1st refinement added -prices
    2nd refinement added -supplies
    3rd refinement added -compare
    4th refinement added -toner
    5th refinement added -‘imaging cartridge’

    Even then the sponsored links all included these items.

    All these things could be filtered by removing shops from my search target.

    If I could do an information only search or a shops only search when
    necessary things would be so much easier.

    Is this something that is in the pipeline or maybe you could give me an idea of the reasons behind why it isn’t.

  60. I had seen that piece of code that Google algo spitted out, it was interesting piece of information, but too little information to draw conclusions. people will be studying it to find some leverage they can get out of it, but i doubt that they will find anything useful.

  61. Google have such more tools for an internet client browser that you can find allmost everything. They release recently the Google Search Book a powerfull search engine for searching books on the internet. Also they launch a search engine offline. It’s name it’s Google Base and much more companies the appeal on this search engine offline to increase their bussiness. So don’t be surprised if Google will attack Microsoft and will create an online operating system(a joke of course):).

  62. Google engineer Matt Cutts with some nice detailed instructions on the process for pleading guilty, kissing up with Google, and maybe getting your website restored to the Google index. The old email address is apparently out of business or will be soon, and you need to go through the online form.

    Unfortunately, he doesn’t address the collateral damage caused by some of their more aggressive filtering practices, or the possibility that following Google’s webmaster guidelines on redirects can do more harm than good.

  63. Google are punishing cat bloggers if they blog about their cat. If you have a dedicated site to cats, or even a passing comment (such as I have) you run the list of being dropped from thier search engines!

  64. Hey Matt:

    Did Steve Jobs personally send you your $100 Apple Credit for paying full price for the iPhone? 🙂

css.php