Matt Cutts: Gadgets, Google, and SEO

382 Comments »

1. Mike Said,

   September 18, 2005 @ 11:16 am

   How can you tell *for sure* if your site has a spam penalty?

   What are the different types/severities of spam penalties?

2. Chris_R Said,

   September 18, 2005 @ 11:21 am

   >>I would request reinclusion for one domain at a time. It looks bad if you had 20+ sites all thrown out at once, and you send a reinclusion request for 20 domains in one email.

   What no GoogleReinclusionMaps - where I can submit a xml file with all my misdeads?

   I think this is good stuff. Besides C Classes and Whois questions - this is probably the question I hear most often.

   I love gadgets - and don’t think you should stop posting on them. I just personally wasn’t interested in the three you made so far.

   If you could combine:

   http://www.ambientdevices.com/cat/orb/orborder.html

   with their SDK and Google’s API to do some cool stuff - people would read. You guys probably have the biggest concentration of Geeks per square ft in the US - we want to see some stuff other than converting addresses to GPS numbers. While that is cool - for some people - I am not going to bother signing up for an API key for that.

3. Trisha Said,

   September 18, 2005 @ 12:16 pm

   What if you really don’t know what it is that you have done wrong? In my case I’ve got two sites which lost almost all their Google referrals since Bourbon. I’ve taken down anything I can imagine that might have been a problem (none of it done to spam Google). What will happen if I fill out a reinclusion request but there is still something on the site that Google doesn’t like? And will the penalty ever go away without filling out a reinclusion request?

4. Harith Said,

   September 18, 2005 @ 12:26 pm

   Matt

   Great much needed info which hundreds of webmasters shall be very pleased to read.

   For the first time I read such clear guidelines for submitting a reinclusion request.

   May I suggest adding such info to the webmaster guidelines on Google site.

   http://www.google.com/webmasters/guidelines.html

   Keep those SEO posts coming

5. Mike Said,

   September 18, 2005 @ 12:27 pm

   Uh, what happened to my comments? I’d love to know the answers to my questions.

6. lots0 Said,

   September 18, 2005 @ 12:53 pm

   Are these procedures for only domain wide penalities or are there specific page (URL) or specific keyword penalities that can be addressed thru this method you lay out Matt?

7. aaron wall Said,

   September 18, 2005 @ 1:52 pm

   >Don’t bother mentioning that you spend money on AdWords or you’re an AdSense publisher. The person who will look at your reinclusion request doesn’t care if you have a business relationship with Google.

   Not once, but twice, different webmasters I know and trust well have told me their organic Google problems have been solved through their AdWords ad reps.

   Both times there was an obvious penalty on the site and the problems were corrected within a few days. These were high spending accounts though.

8. John Said,

   September 18, 2005 @ 2:01 pm

   Matt

   This is a great tip. We really needed some clarity on this (Particularly give the story in search by John Battelle about bigfeet.com). Please keep this kind of feedback coming.

   regard

9. Michael Still Said,

   September 18, 2005 @ 2:21 pm

   I get a lot of comment spam on my sites. Is there a way to report these link spammers to Google so they can be considered for a spam penalty?

10. IanC Said,

    September 18, 2005 @ 3:48 pm

    As someone who is currently waiting for the site to be re-included after a manual exclusion by google around 18 August, I’ve so far had mixed results following the above procedure. The sequence / speed (for those who have mentioned lack of response from google ) of response from google went as follows;

    22 Aug sent re-inclusion request by filling in form.

    2 Sept - Google confirms that our site has been indeed blocked. Advised to read google guidelines and respond after making suitable amendments.

    2 Sept. As we’d already started to review the html coding etc and cleaned up the site, we sent back an email informing google that we had corrected potential problem areas. We’d succumbed to adding a third party written script(not a link exchange scheme) to build links with related sites a few months back. This and all the links were deleted from the server. The html coding was also checked and cleaned up.

    2 Sept We received an email response thanking us for trying to meet the guidlines and that our site had been passed to the engineers for review. If its accepted then it would appear in the next few months.

    13 Sept.. For our own peace of mind sent another email asking whether the particular third party developed link script we’d used was indeed the problem.

    13 Sept ( few hours later) received emial saying that we just wait for review.

    18 Sept ……. still waiting for re-inclusion….

    It was rather interesting to learn about a penalty based upon the seriousness of the irregularity. But this does raise a question: are we ‘ being done’ for a minor road traffic offence or stealing the Queen’s jewels?

    

    regards
    IanC

11. kirk Said,

    September 18, 2005 @ 3:49 pm

    So if my site has dropped out of the SERPs because I did a 301 redirect to a new domain name (rebranding) does that mean I need to file a reinclusion request? Or have I been assessed some other kind of penalty that a reinclusion request can’t fix?

12. Gerald Said,

    September 18, 2005 @ 5:15 pm

    and what to do when you have a side where you do not know why it happend. where perhaps some small things happend in the past or something you have missed?

    if you have a site in the top positions in an competitive sector you have to do some little seo efforts in order to not lose positions. as everybody around you is doing it. they all buy links, they all exchange links. sometimes really heavy, with not so nice methods.

    in the past we had a problem with some automatic linktrade programs especially here in germany, everybody seemed to jump on the waggon. the site of my customer went down from 1 to 8. i refused to do it like all the others. my customer and i were not happy. my customer lost real money. it took several month until google reacted. this site was the one i made the smallest seo efforts from all in order to do not risk anything. i did it as there had been guestbook spamming in the past (before my time) etc and i did not want to accumulate to many bad points for this site.

    what happend, one week after switching to a new server and 2 days after insertion of adsense the index page went off. and later most of the other pages went out of the index. but not all. believe me, if i would know the reason i would have eliminated them. it’s not so simple. i can’t pull down the whole building and set it up once more in order to find the one point that ultimately killed it. there are many other possibilities like duplicate content ( + www and not www domains ), link bombing, url hijacking and perhaps mistakes made by myself which happened accidentally or mistakes or misinterpretation by the google software. i do not expect any reaction, but as you spoke about this topic i must complain a little bit . it bothered me for such a long time now. i have seen hundreds of sites using all tricks that seos can invent and in 99,9% of this cases i could tell you why this sites dropped from the index. but not in this special case. i would like to close with the famous words from Giovanni Trappatoni: “Ich habe fertig”.

13. rochelle Said,

    September 18, 2005 @ 7:24 pm

    Hey Matt,

    It’s nice to see you communicating with the masses. I really appreciate the openess that you are trying to start. Even though, we always think you have hidden messages.

    I’ve met you a few times and I can testify that you are a straight-up kind of guy. Thanks.

14. Chris Said,

    September 18, 2005 @ 7:38 pm

    Hi Matt,

    One thing I’m curious about is how do you know if your site has a penalty?

    For example, I used to run Amazon Web Services affiliate script on one of my sites - I thought it would be useful for my visitors and potentially profitable (I only used the parts of amazon relating to my site).

    I think this may have resulted in a penalty on the site as a whole (the script produced hundreds of product pages - far more than the rest of the site’s content). However, I don’t know if this is true at all or if Google has some other reason why that site has dropped in rankings. I don’t use any SEO techniques so I can’t think what it could be.

    I can see that if your site has completely disappeared then you know that you have a penalty, but do you actually give penalties that just lower your rankings a bit (and if so how do you know what caused them?).

    Would the re-inclusion request link you mention be the way to find out about any penalties you think you may have?

    Thanks!

    Chris

15. Your fan Said,

    September 18, 2005 @ 8:35 pm

    Hi Matt, what if I can still find my site in G index by queryin “www.mysite.com”, but unable to see any page in serp? My site used to rank very high in some keywords but about one year ago, an idiot downloaded/cloned my whole website. The DMCA helped me out and my site ranked very well some months but it now goes no where. I do not use any trick mentioned in your article and some of my competitors use hidden text and they still rank very high:(

16. Matt Said,

    September 18, 2005 @ 11:11 pm

    Trisha, good question. If there’s an algorithmic reason why your site isn’t doing well, you can definitely still come back if you change the underlying cause. If a site has been manually reviewed and has been penalized, those penalties do time out eventually, but the time-out period can be very long. It doesn’t hurt your site to do a reinclusion request if you’re not sure what’s wrong or if you’ve checked carefully and can’t find anything wrong.

    lots0, the reinclusion request process can apply for any type of spam penalty.

    Michael Still, let me make that the subject of a future blog post. The short answer is to do a Google search for [report spam] and look for the Google web form for reporting spam.

    kirk, 301s from an old site to a new site shouldn’t cause any problem, so I wouldn’t use a reinclusion request for that.

    Gerald, we definitely want to find the ways that discourage site owners from doing things like hidden text, hidden links, and the sort of stuff you mention in your comment. We’re always working on ways to help straightforward, hard-working sites while keeping sites that violate our quality guidelines from showing up higher than they should. I think Google’s results are definitely less spammy than (say) 18 months ago, and that many spammers have to work harder to show up highly, but the job isn’t finished yet.

17. rob Said,

    September 19, 2005 @ 12:49 am

    Matt

    I appreciate stuff like this too.

    A while back ( 3 months) I used the form you referenced to request a little feedback. A site I’d worked with had tanked in May of this year and basically couldn’t rank for anything other than its sitename and 2 other keywords. Prior to this, it did resonably ok ( a few top tens, a few top 20’s etc) on most of its targets.

    The people at Google were kind enough to tell me that I didn’t have a penalty and that I should continue to get some more quality links.

    I was naturally baffled why I found that I couldn’t rank for jack, all of a sudden, and am still of the view that the domain must have some kind of penalty.

    My question is this. Would the engineers have access to penalty info that the ‘1st line support’ responders wouldn’t? Is it the case that one person could be saying that no penalty exists, when another might know otherwise.

    Cheers for any feedback you can provide.

    Rob

18. Kim Stian Ervik Said,

    September 19, 2005 @ 1:27 am

    Thanks for providing such valuable feedback, Matt. It is really helpfull.

    Kim

19. Nigel Said,

    September 19, 2005 @ 2:04 am

    Another suggestion,

    You have skyscraper adsense advert, it seems to be inside an iframe, sometimes it only displays 2 or 3, instead of the maximum 4 or 5 adverts. Why don’t you shrink the iframe to fit the adverts?

    i.e. why not create a new advert type: collapsible skyscraper, which is 160 pixels wide, but the height is only big enough to fit the adverts in it. If there’s no adverts, instead of public service ads, you collapse it to zero.

20. Gerald Said,

    September 19, 2005 @ 2:43 am

    matt, i must admire that google made a good job in fighting spam. much better than before, and much better than some competitors. but sometimes it lasts. have you ever considered to work together with SEOs in order to fight spam even faster and more effective? (i think so, but is it opportune) or does it happen already? if i were you i would go into this direction. i had several discussions with other SEOs, imho really good ones, and i and most of them had the opinion that they would include something like a SEO task force at google in order to prevent spam. may be this blog is part of such a strategy and the first step into this direction.

21. Stephen Newton Said,

    September 19, 2005 @ 6:51 am

    My site at http://www.pr-consultant.co.uk includes a blog, which is indexed by Google. But the main pages (including the index page) are not included. I’ve had tried to tell Google the main pages are more important using sitemaps, but this has not helped.

    I also wonder if Matt could share his thoughts on Sitemaps with us. Are they worth the effort?

22. Rob G Said,

    September 19, 2005 @ 7:37 am

    Matt,

    Thanks for providing some transparency here with regards to webmasters desire for inclusion in the Google results.

    Your first statement makes an assumption that Google only bans and penalizes webmasters who have “SEO’d” their sites, where I am finding that many bans I have encountered are a results of ignorance regarding redirection best practices.

    The problems I encounter involve simple domain redirects to the homepage of the client’s site. i CAN ASSURE

23. Rob G Said,

    September 19, 2005 @ 7:37 am

    Matt,

    Thanks for providing some transparency here with regards to webmasters desire for inclusion in the Google results.

    Your first statement makes an assumption that Google only bans and penalizes webmasters who have “SEO’d” their sites, where I am finding that many bans I have encountered are a results of ignorance regarding redirection best practices.

    The problems I encounter involve simple domain redirects to the homepage of the client’s site. i CAN ASSURE

24. stuntdubl Said,

    September 19, 2005 @ 7:39 am

    Glad ya liked the post. *cough* shameless rep request *cough*.

    Reinclusion is a great topic. I think most of us *do* like gadgets, we just like the other SEO information MUCH better.

25. Adam Said,

    September 19, 2005 @ 8:42 am

    “Why should a Webmaster have to make a choice between Aesthetics and SERPs!”

    Erm, how about because the SERPs are provided by the search engines who don’t answer to webmasters. The very nature of the relationship between webmasters and a search engine is such that you have absolutely no place questioning the methods of a company providing you with free - and potentially very lucrative - marketing, when all you need to do in return is be bothered to meet a few paltry guidelines.

    Being listed by Google, or any other search engine, is a PREVELIGE, not a RIGHT.

26. Andi Said,

    September 19, 2005 @ 8:58 am

    My reinclusion requests have been frustrating and maddening. I have decided to deny googlebot access to my banned site for at least several more months, these issues should be clearer then. Every page prominently bears the words “Not indexed on Google.”

    Until that time (and perhaps forevermore) I am promoting your competition.

27. Port of Dreams Said,

    September 19, 2005 @ 9:03 am

    Is it true Google is going to expand the use of email’s to notify site owners when their site has been banned for shady practices?

28. Andi Said,

    September 19, 2005 @ 9:07 am

    Oh, I might add (and you will find this if you search) for years I have been defending Google from its detractors. I built my site as an adjunct to search and feel very betrayed.

    I have become one of those detractors.

29. Bill Hartzer Said,

    September 19, 2005 @ 9:25 am

    Expired domains are banned from the index automatically.

    Any thoughts or suggestion about what to say to Google when making a reinclusion request for a domain that’s been banned because it was expired? Obviously you want to get the domain back in the index–but cannot remove anything spammy that was done by the previous owner–the site is owned by someone new.

30. Eric Said,

    September 19, 2005 @ 10:04 am

    Matt,
    I followed the procedure that you listed. I received an automated response from Google right away listing links to Google’s webmaster guidelines. Do I need to reply to this email for my request to go any further? If I do not reply, will my request still be reviewed?

    We have been dead in the water with Google since Feb. 2nd, 2005. How long can penalties last?

    Thanks for the help.

31. PhilC Said,

    September 19, 2005 @ 10:10 am

    Matt Cutts wrote: “We’re always working on ways to help straightforward, hard-working sites while keeping sites that violate our quality guidelines from showing up higher than they should.”

    You will forgive me if I take that with a pinch of salt, won’t you. If you really had any desire to “help straightforward, hard-working sites”, you would answer people’s genuine concerns. For instance, it’s been pointed out more than once in this blog, including in this thread, that javascript redirects are perfectly valid, and even essential, in some circumstances, and you’ve been asked to confirm that Google won’t hit sites that make valid use of them, but you’ve failed to reply. That’s just one example where you’ve been asked to “help straightforward, hard-working sites”, but you are reluctant to do so. Why can’t you answer people’s genuine concers? I’m sorry, but I don’t believe for a moment that Google is interested in helping any site other than their own.

32. Joe Hunkins Said,

    September 19, 2005 @ 10:46 am

    Matt -

    This is *really* a great effort by you to clarify issues that come up at WMW and SE Watch and elsewhere. Super helpful stuff and very much appreciated.

    I am concerned though that there may be a difference between what you and the engineers see as “penalty” and what the support staff is told to call a “penalty” and that this is leading to a confusion for many sites.

    Based on WMW and our experience large, content rich sites can stop ranking yet support says “you have no penalty”. Clearly there is some form of penalty on sites that suddenly face big traffic drops and filters. I bet you’d be surprised at how many webmasters *really have no idea* why they’d been hit, rather they’ve been using the same code/ideas for years and suddenly are penalized in an effort to combat the spam.

    I’d suggest a paid review process through Google Answers where the results are *shared with the community* this way you can control what is said about the penalties/algo, those who can afford it will pay to be reviewed, and those that cannot afford it will benefit from the case studies.
    I think it’s easier to get white hat behavior in an information rich environment (like this blog) than an info poor environment (webmaster support system which is not very knowlegeable or responsive.)

33. Mariano Said,

    September 19, 2005 @ 12:34 pm

    Hi Matt,
    I was reluctant to send a reinclusion request until your post. Thanks. However, When do you know you’ve been penalised? I mean, If I look real hard for my site, I finally find it, but I don’t know if it was penalised or it’s getting the position it deserves.

34. crazyj Said,

    September 19, 2005 @ 2:05 pm

    [quote]You can select “I’m a webmaster inquiring about my website” and then select “Why my site disappeared from the search results or dropped in ranking.” Click Continue, and on the page that shows up, make sure to type “Reinclusion Request” in the Subject: line of the resulting form. Upper- or lower-case doesn’t matter, but make sure you use the words “reinclusion request” in the subject line so it gets routed to the right place.[/quote]

    Doesn’t Google have any HCI guys who can make that process easier for people? Specifically, if the message requires “reinclusion request” in the subject line then why doesn’t the web app do it automatically?

35. Matt Said,

    September 19, 2005 @ 3:33 pm

    Gerald, that’s a good idea for a future post.

    Andi, I would try a reinclusion request if I were you. I think it would help.

    PhilC, I responded to your original comment over in the JavaScript post comments.

36. David Dyer-Bennet Said,

    September 19, 2005 @ 4:41 pm

    I’m pleased to hear Google is experimenting with actually communicating with webmasters. One of the big problems I’ve seen in the search-optimization community is a huge amount of snake oil and a huge amount of Fear, Uncertainty, and Doubt being spread for commercial advantage by lots of people. And a lot of superstition. These things are natural in the absence of real information.

    Google clearly has some big legal issues; lots of sites consider their search engine ranking to be one of the most important business “properties”, and will resort to legal action if they think it can get them something. It will be an extremely delicate balancing act for Google to dance through, to communicate usefully without engendering even more suits. Good luck!

37. Jad Bitar Said,

    September 19, 2005 @ 5:07 pm

    Hello,

    This is not exactly about reinclusion but very close, so thank you for bearing with me.

    We launched a blog 2 weeks ago and all articles have been listed in Google’s blogsearch except for 2. The articles written before and after are all listed as well as the ones published in between. The strange thing is that the two articles in question have the same subject, a before and after appreciation of the ebook and Rich Jerk program. I am completely stumped as to why these articles are not listing and any help would be appreciated. Thanks in advance and congratulations on the quality of this blog.

38. PhilC Said,

    September 19, 2005 @ 6:33 pm

    >> PhilC, I responded to your original comment over in the JavaScript post comments.

    Thank you, Matt - almost a month after the concern was posted, but better late than never. Not responding to genuine concerns like that is unnecessary, imo, and can foster negativity. Anyway, to move on…

    It’s interesting for me that you should post about reinclusion requests, because I’m a bit stuck with one right now (the first time I’ve ever needed to even think about reinclusion), and I honestly don’t know the way forward.

    I have a site that predates Google, and has had many very good rankings in Google from the start. More than 2 months ago it was penalised. The penalty was such that the thousands of pages that were indexed steadily decreased until there were none in the normal index, and then many thousands showed in the Supplemental index.

    The site contained a few things that were very light grey, one comparitively small part that was dark grey or even black, and all but one of its pages auto-redirected to pull the framed site together with the correct page in the main frame. I have no idea which aspect it was that caused the penalty.

    I cleaned the site up totally, except for the necessary auto-redirections, and used the form to ask what the cause of the penalty was. I received an auto-response that didn’t help at all.

    So I submitted a Sitemap to see if I could get any pages indexed. After a month, all the Sitemapped URLs (only ~100) were spidered, but they never appeared in the index as far as I could tell, presumably because the site has a penalty. The thousands of pages in Supplemental weren’t decreasing even though spidering them had been disallowed in the robots.txt file a month earlier. I wanted rid of everything (clean slate), so a few days ago I requested their removal using another Google form. The pages were from one area (sub-directory) of the site, and had only been created to allow search engines to reach the tens of thousands of pages that they couldn’t otherwise reach, as they are all behind forms. The pages were all removed the following day. That will keep them out of the index for 6 months, but it doesn’t matter for the site.

    However, Google had already crawled and indexed a handful of pages by then, and the last time I checked they were in the normal index. But they are gone now, even though their removal wasn’t requested. I don’t know if my removal request will prevent those pages, and others, from being indexed.

    So I’m stuck. I still don’t know precisely what caused the penalty - it could have been the auto-redirects, which still exist and may still cause penalty problems for the site, or it could have been that small black area. I don’t think that anything else in the site would have caused such a heavy-handed penalty. Considering that I asked for the removal of an area of the site, is a reinclusion request likely to be any good at all?

39. alex Said,

    September 19, 2005 @ 10:47 pm

    My site has been penalized since may 2005, and at that time google support sent me a message that the site was not penalized. The reality is that the site didin,t rank for anything anymore!

    So it clearly had a penalty going on…

    I just finished rewriting the whole site and asked google to remove the penalty, the only message I received is to please have a string look at the google guidelines. But since i’ve been working on correcting DUP issues since the last months, I must say the reply leaves a bad taste in my mouth! I need more detail! Which part of the guidelines still represents a problem for my site!

    I’m frustrated because they say google emails people with javascript redirect and even hidden text, and for my site which is an honest review site, I cannot have a clue about what’s the problem!!!

    I wish google rep told me clearly something like, check out this, and this and this.

    I’m getting desperate about ever getting back in google, because even with removing all the duplicate content problems, I only receive generic answers

    Alex

40. Christine Said,

    September 20, 2005 @ 2:13 am

    How do I get images and news related to my website included into the Google’s database so that they appear in the Image search and News search.

41. Harith Said,

    September 20, 2005 @ 2:15 am

    Hi Matt

    I see much interest to this post, lets call it “The Father of All Matt´s Posts”

    Following your advice, I filed a reinclusion request. Reason is that my site has lost a big deal of its Google´s referrals.

    Before filing the request, I went through my site to clean anything which might have triggered a “red flag”.

    I could find and have deleted a 4 years old page with “100% Frame” . Within the frame is a remotly hosted template for e-books search on ClickBank. I pay for that customized template. I regarded the page as “value-added content or service” to the visitors of my e-books library (a section on my site) because ClickBank has no search facility at all. However, as I said I have deleted that page just to be on the safe side.

    My question is; does Google mark any 100% Frame as offensive?

    And I have another question for you

    Talking about “value-added content or service” in connection with online-affiliates who are somewhat equivalent to off-line resellers. I understand that you require more of them than whats required in real life of their fellow off-line resellers. And I´m not talking here about those affiliate sites which are simply just duplicates of other sites “Thin Affiliate”. I´m mostly interested about Google´s positions concerning hard working affiliates

    Therefore I wish to ask; how do you define “value-added content or service”? and is it a must to have a kind of prise-comparison on an affiliate site to be reagrded as “value-added content or service”. I.e wouldn´t you regard the presence of related articles, forums, carefully sellected outbond link pages as “value-added content or service”?

    I´m asking such question because I read on several webmaster related forums that people started talking about Google waging a ware against the Affiliate Marketing industry!!

    Thoughts?

42. ModemMike Said,

    September 20, 2005 @ 4:54 am

    Matt, no disrespect intended here but there is a definite problem with 301 redirects, I myself have felt the sting of being banished from all results after a 301 redirect. I placed a 301 redirect on a high ranking site following the webmaster guidelines hoping for the best but fearing the worst. I have now been absent from the results for over 3 months wondering what I did wrong. I have sent 3 emails to the help address and have only gotten back canned responses that I have not been penalized and that I need to be patient, call me inpatient but doesn’t 3 months seem a little excessive? From what I have read elsewhere a 301 redirect is essentially starting over but that is not indicated in the webmaster guidelines in even the lightless way. For further details (if you are interested) check out “The 301 Club” over at WMW.

    BTW, I’m a major gadgeter as well, any plans to review or blog home theater pc’s?

43. Oriol Said,

    September 20, 2005 @ 6:32 am

    Dear,
    The problem in spain is that are some guys with lots of spam wich google doesn’t matter because it is in spanish or may be other reasons.
    Many seos have done and seen them sites out while they stand in the same best positions for the time of times.
    just make a search in “posicionamiento en buscadores” and look all webs in position 900 and under….
    thanks!

44. Lagniappe Said,

    September 20, 2005 @ 3:27 pm

    I have a very similar problem. I have a new site, domain purchased 3/21/05, site has been up and running for 4 months. We’ve submitted sitemaps since the program inception, finally had significant visits from Googlebot in July, but still no inclusion in the SERPS. Not a single solitary page, not one.

    I’ve been over the site again and again and we aren’t doing any hidden text, hidden links, no javascript redirects, no doorway pages, no keyword spamming. Nothing. We’ve built a very clean site. If there is some obscure reason why we’re being excluded, I’d never know. How would we ever know if our site was banned or our domain expired before we even owned it?

45. Andi Said,

    September 20, 2005 @ 3:53 pm

    >>Matt Said,
    >>Andi, I would try a reinclusion request if I were you. I think it would help.

    No, I think I’ll wait until late November and see how this discussion develops. This will be costly perhaps in fame and fortune but my pride remains intact. Being unceremoniuosly dumped without a word after four years in the index is not something I take lightly. I will not beg. Gbot will remain disallowed from Andilinks for another month at least and Google searchers will be denied my work.

46. Janine Davidge Said,

    September 21, 2005 @ 9:46 am

    Hi Matt,

    I would like to read more about the JavaScipt post but can’t find it. Could we please have a link to this post?

    RE: PhilC, I responded to your original comment over in the JavaScript post comments.

    And thank you for putting out the effort and time to help webmaster’s better understand Google’s intent in these maters. I’d imagine Google may think they’ve opened a can of worms by increasing dialog with webmasters and SEO specialists, but we do find it extremely helpful and very much appreciate getting answers to our many questions.

47. Janine Said,

    September 21, 2005 @ 11:06 am

    And can we edit our comments here, please? Spelling/typing errors left uncorrected are embarrassing :-p

48. Lawrence Said,

    September 21, 2005 @ 11:24 am

    Does this work for 302 hijacked sites? For those of you who are not aware, check this out:
    http://google302.ibotech.net/resources.html

49. Trisha Said,

    September 21, 2005 @ 12:34 pm

    Thanks for the reply Matt! I sent in one reinclusion request yesterday - and have my fingers crossed that it will take care of the site’s problem!

50. George Said,

    September 22, 2005 @ 3:37 am

    ModemMike wrote:
    >there is a definite problem with 301 redirects, I myself have felt the sting of being banished from all results after a 301 redirect.

    Same here. Yesterday I’ve 301′ed about a hundred pages to a new directory on the same domain and the whole (quite reputable) site has been banned in less than 4 hrs. I immediately put everything back where it was before; in total, the “misdoing” lasted for about 10 hrs. Guess I should send the “Reinclusion Request” anyway?

    ModemMike, thanks for the link to 301 club - a must read.

51. Andi Said,

    September 22, 2005 @ 5:06 am

    Thank you Matt, you are a gentleman and the best thing out of Google since the very beginning.

52. PhilC Said,

    September 22, 2005 @ 5:16 am

    The Javascript thread was last month, Janine:-

    http://www.mattcutts.com/blog/seo-mistakes-sneaky-javascript/

    Another thought on this thread…

    It is easy to know when a site has been banned, and when a ‘Reinclusion Request’ might help. But knowing whether or not a site has been penalised (but not banned) isn’t always as clear, because a sudden and significant drop in rankings (an apparent penalty) may be attributable to a significant algo change. The Reinclusion Request method is straight forward, but is there a similar Subject line phrase that can be used to find out if a penalty exists, or to request removing a penalty after cleaning up?

53. Burned B. Google Said,

    September 22, 2005 @ 7:08 am

    The reinclusion request system works infrequently, because:

    1. Most requests go unread.
    2. Of those few reinclusion requests which do get read, very few are responded to.
    3. Of those few reinclusion requests which are responded to, almost all are responded to with completely irrelevant form letters.

    My advice:

    If Google has penalized or banned your site for no explainable reason - register a new domain name and 301 the old domain to the new domain.

    It’s a whole lot faster and more certain than waiting for Google to find and fix their mistake.

54. Fred Said,

    September 22, 2005 @ 12:34 pm

    I strongly suspect my site was booted from Google, because I was using Rankingpower. I guess this would qualify as a doorway page program or scraper program. I was also employing rpower’s “cloaked_url” feature - shame on me! I got rid of all those pages, did an extreme makeover of my homepage, added real original content and asked Google for reinclusion. I was completely honest about my wrongdoing and promised not to do it again. I last received a message from Google Aug 26 that my case has been referred to their engineering dept for further investigation.

55. bentar Said,

    September 22, 2005 @ 8:19 pm

    I m new to read all your post Matt, and I like it much just wondering to know one of google men come and here he is lol

    I had a experience (sorry my english) once my client’s site was duplicate in meta tags that happen in 2 years a go

    and the ironic is that site exist now on Google SERPS but the terrible think that my client’s site was banned from that time, what happen.

    I just wondering to know what happen actually here.

    ok guys thanks in advance for the change to drop my post here.

    I tired some how with all the SEO stuff and just started the blog in my hobbies , any body could join my blog here

56. Antti Said,

    September 23, 2005 @ 4:00 am

    What should I say in the reinclusion request if I have no idea why my site was penalized? It’s still in the index but dropped from page #1 to around page #10 on all keywords. The url is http://www.ringtones.fi/ and it was my real, proper, honest try at doing everything by the book. Ps. ticket #34400643

57. arubicus Said,

    September 23, 2005 @ 10:32 am

    We have taken a hard penalty in Google and remain that way since May. Our site publishes articles from many sources (some unique and some not) and are all submitted to us by webmaster, experts, publishers, and notable authors if they serve our visitors interests. Some articles we retain the rights to even though we may allow them to be re-published in certain cases us of course being the originating publisher. We always believed that by doing this we would receive a “page only” penalty on any re-published articles (not originating with us) and uniques would be treated as, well, unique. This seemed to always be the case for the last 5 years.

    We also have a directory of online shopping sites that also could consider our site as “scraperish”. The directory was and continues to be hand-built and all inclusions have to be submitted to us and screened. For the last 5 years we never really ranked or care to rank for that section of our site. It was mainly to satisfy the divide of ecommerce and information within our readers not to gain SE traffic.

    After a huge site cleanup of old 301 redirects in which Google was not handling them very well serving up “Google generated” dupes. After which we reported these 301 errors with examples and filed for reinclusion a couple of times. Not a word in response for either and crawls of our site has fixed many of these errors but still no improvement. Still nothing seems to stick.

    Our next step is to rewrite the complete site (dire need of it may I add) and browse through our database of some 8000+ articles, pages, sites looking for any reason for having a site-wide penalty (even looking for links to bad neighborhoods). This will take about 6 months to do. The only things we can think of (since we don’t use any black hat SEO) is:

    1.) We use titles and descriptions in our sub sections to introduce contents of our articles which is the same as the title and description on the top of our articles and related articles as well as the meta title and descritpion.

    2.) Site wide penalty for using articles that get submitted elsewhere (again you would think this would be a “page-only” penalty)

    3.) PHP database driven template - without query strings - hasn’t caused a problem the 2-3 years we have used it.

    4.) Crappy html code - will be corrected soon going to a CSS design.

    Other than that I have no CLUE. We have lost over 6 figures of income since then and we have been VERY patient holding off our redesign for 7 months now but it has now come down to having to do it anyway and micro-searching for any cause. I just would hate to spend countless of man hours picking through everything where time could be spent by our employees actually running our company. Granted if it is something that can be fixed.

    We would also hate to have to pull any re-published articles since that would GREATLY take away from our visitors. That is something at we would not want to do because we and also our visitors consider them to be of great value. It also is the foundation of what our company is and does. We also cannot “test” edit many articles that we have not retained rights to do so. Besides MSN, Yahoo, other search engines, sites, online and offline publications featuring us seem to have no trouble with us using them.

    So here we sit in a major bind when it comes to Google without as much as a clue to where the problem lies. I guess we will just have to sit in the dark trying to find a needle in the well known haystack with our eyes gouged out. But we do stick by our guns when it comes to hope, patience, professionalism, and faith in Google.

58. Michael Pedone Said,

    September 23, 2005 @ 11:49 am

    Matt:

    Would Google ever consider having a web page were you can enter your domain name and have it analyized on the fly if there are any problems (spam) with it so that the webmaster can instantly know what to fix?

    Our site which used to rank very well on Google has disappeared and we have combed the site over and over again removing anything that could be even remotley as spam (which was a good thing) however since there was never any spam done intentonally, it would some of us sleep better at night if we knew that it’s a Google thing (page jacking etc) and not anything on-site that is casuing the problem.

59. arubicus Said,

    September 23, 2005 @ 12:02 pm

    Wouldn’t that be an awesome tool. But it may hurt the medical industry by reducing stress related ulcers!

60. Fred Said,

    September 24, 2005 @ 5:13 pm

    If I am awaiting reinclusion to Google for a banned site, Is it probably not a good idea to try to get it accepted into dmoz.org? My logic here is I figure getting accepted into dmoz might help speed up the reinclusion process if Google is taking their time.

61. Eric Said,

    September 25, 2005 @ 5:30 pm

    Matt,
    As Arubicus has posted above:

    “We use titles and descriptions in our sub sections to introduce contents of our articles which is the same as the title and description on the top of our articles and related articles as well as the meta title and descritpion.”

    I am monitoring many sites which use the same setup as above and seem to have a penalty or filter. Is this practice considered bad? Could this trigger a duplicate content penalty or filter? Would appreciate your comments.

    Thanks.

62. Alexo Said,

    September 26, 2005 @ 1:04 pm

    Hello Matt

    first of all thanks for providing some transparency here

    can anybody explain how it’s possible (or may be it’s G new algo), that page with original and unique article is ranked lower (searching “Artilce title”), than sites which use the ur RSS feed on their sites?

    so searching “Uniqal article title or any phrase” in google (with quotes) u can got N results.

    ur site is the last
    before u are sites, which use RSS feeds of ur site (where is this artilce short description) with link to ur site.

    is this side effects of new algo. or this is result of hijacking or G’s penalty?

    i don’t know what to do?

63. Kelly Jones Said,

    September 27, 2005 @ 8:10 pm

    Last Thursday 99% of our search results disappeared from Google — our site is no where to be found except one search term, our oldest and best by chance. However, all the rest of our terms/sites/results are gone. We have two domains with identical content for copyright purposes. I was avoiding 301′ing one to the other to avoid the problems. However, it appears Google simply removed the primary domain and let the secondary domain which is VERY poorly indexed. Our site traffic disappeared overnight. Ugh!. I’ve 301′d the secondary domain to the primary but now I’m wondering if I shouldn’t have done the reverse.

    Thoughts?

64. Kelly Jones Said,

    September 28, 2005 @ 12:04 am

    Correction to that last entry — I’ve just found my results in Google. Sorry for the last post — I didn’t think to look for pages which were formerly on the first page of Google results down at the #370th position. I have no idea why Google would rank me on the first page one day (and for the past many years) and then move me to positions 370+. This is weird and is killing my business.

    Matt, I’ve done what you said but this seems different than being banned. It would be great if someone could explain it. So far I’ve only received canned responses from Google and am waiting for a real person now.

    KJ

65. MMMPPH Said,

    September 28, 2005 @ 11:52 am

    Matt -

    Good stuff and probably most appropriate for those who have severe stuff going on, got the manual ban, or maybe tripped over the line on the REALLY nasty black-hat SEO. However, I’ve run into quite a few that seem to have merely tripped over an update where something triggered something that didn’t get them in trouble before, or maybe fell over some “points against” thing. I saw quite a bit of this in May.

    For those sites, it seemed as though they might have been violating new spammy criteria (e.g. too many H tags and bolding every instance of keyword, etc), but nothing like intentionally creating doorways or other attempts to play games. In many cases, it was an over-zealous attempt to compete with the big dominant boys.

    In each case, the rankings dropped out of site, but the site was still indexed, though many had convereted to URL only links. Hence, I concluded that there was no outright ban, but probably an algo penalty. I would imagine that Google probably runs into thousands or more of these per day. The goal then was to trip the algo back the other way. I concluded that this was going to take some time no matter what was done and since it had not been an outright evil doing, maybe it was best to NOT contact Google, but just do what needed to be done and let the algo sort things out.

    I don’t know if Google keeps track of requests, comments, issues, etc according to domain name, essentially a file history of problems by domain. But I reasoned that if they did, making a reinclusion request could certainly get you put into that “bad-boy” list. Obviously it would get you noticed, and if something is amiss, getting noticed isn’t always a good thing. At least you had a chance of not being on a bad boy list if you just let the bots do a reassessment.

    In all cases, the sites suddenly came back a month after completing the changes, and none required contacting Google. The way I look at it, a domain just might have a certain number of chances with Google and if you do the reinclusion request, you are probably using up one of those chances. ???

66. Angela Rami Said,

    September 29, 2005 @ 3:45 am

    Google ignoring requests for certain domains?

    Please keep comments on topic, this is about reinclusion requests. We have requested for a reinclusion twice now, in 2 months. Kindly explaining (like already pointed out by Matt above) what 1) we believe that happened and 2) explaining we don’t want this kind of things to happen again (accidentally or not). Both times no response (except for the standard ‘go and read the guidelines’ response).

    It might be that Google is filtering requests on domain name, or maybe even IP-addresses? If this is the case it really would be pointless trying for a reinclusion again. As previously comments by a few, at some point, it feels like you have to beg for reinclusion. Some even block Googlebot. Actually, we are blocking Googlebot now for pages we feel are helpful for our visitors, but probably are — since we are banned — against the google guidelines. Maybe Google-engineers ignore your robots.txt-file, and just keeps you banned.

    If there are many people/organisations working on a website, and 1 screwes up (according to Google’s guidelines), everybody is screwed unfortunately. We were using feeds, yes, but my advice is to never use other peoples content, even if they reward you for it, one way or the other.

    We know many people are using google just to type in ‘your-brand’ and even ‘your-domain-name.com’, to find your website (instead of typing it in the address bar). We have been banned now for two months, so that’s really a loss for us, even if it’s only a handful of visitors.

67. Julian Moss Said,

    October 1, 2005 @ 2:22 am

    Thank you, thank you, thank you Matt for this article. After following your advice, my fledgeling new site The PC Guru has finally appeared in the Google index with a Page Rank of 1, after months of trying. No other pages are there yet, apart from a cached entry for the domain’s previous owners, but it’s a start.

    Thanks again. I would probably never have resolved this issue if I hadn’t seen your article.

68. John Rooke Said,

    October 4, 2005 @ 4:46 am

    I have accidentally submitted the homepage of our orlandovillas.com (non http://www.) site for removal using the automated removal tool. The current status is showing as ‘complete’.

    Overnight, last night, the non-www homepage entry was removed from the Google index but also the www homepage was removed too (we did not want this one removed and did not request it’s removal).

    I have been reading on various forums that there may be a bug / feature of the url removal tool that if you request the non-www homepage to be removed, it also removes the www homepage at the same time.

    We’re not convinced the feature / bug exists as the www homepage returns a 200 status and thus the url removal tool should not have removed it, but would like some reassurance about it.

    We’ve just filed a re-inclusion request on this basis.

    How do we get our www homepage back?

69. Anne O. Nymous Said,

    October 5, 2005 @ 3:49 am

    The worst things about being banned and having to ask for reinclusion are
    - that Google is not acting transparently about its criteria
    - doesn’t inform webmasters about a possible ban, e.g. via an automated mail or form, where you could get a hint, what actually is considered as spam on your site
    - isn’t really able to tell spam from useful info e.g. internally useful comments in the source, hidden text for navigational or informational purposes etc.
    - worst point last: affiliate programs being able to nearly spam the first 100 results of every search that relates to a sellable product or service

    I’m watching sites in a highly competetive market spamming with the simplest methods, gaining top rankings, even months after i reported about them. They may drop for one or two weeks, just to regain the same positions as before. This is hilarious.

    Another point i critizize is using DMOZ only as descriptions in search results, instead of additional information, so not only this description does show up, but part of what people are intended to get from the actual content or meta information. DMOZ is far too slow, if you change your site frequently.

    E.g.: A site i work for is listed in the wrong DMOZ category for years now. I therefore even suspect, that one of their competitors is the editor, as several requests to change the category appropriatly have been ignored. DMOZ therefore is not a reliable source anymore, since the word about its importance for SEO has been widely spread and SEOs are taking over categories.

    Trying to bring this different perspectives together:

    Google has to realize, that having achieved a near monopoly about search results, it has become a supplier of the basic material of the web (like air, water, energy, food, excuse my english), many people’s lives depend upon.

    Banning sites for practices that are considered at the edge of spamming, without even warning or asking webmasters about it, is a misuse of that monopoly, as Google is not using its spam policy consistently and transparently.

    If you have to ask for a reinclusion, and even have to guess, if you’ve been banned at all or what you might have done that could be considered as spam, is a misuse of that monopoly. Especially, if the top results for the relevant search terms are so obviously spammed as they are today.

70. Ted Goldsmith Said,

    October 5, 2005 @ 7:43 am

    We operate an established web directory that has been fairly successful (up to 50,000 visitors per day). This year after six years of operation we were delisted by Google. We made some changes to make the site more search engine friendly such as intentionally delisting (via robots.txt or noindex tags) pages that had little persistent value (news, etc.) and implementing a sitemap. We also attempted to fix anything that might be perceived as “abuse” (such as removing keyword tags) and followed the reinclusion procedure. This resulted in the same form email (”we have passed your message on to our engineering team..”) that others have reported. A registered letter to Google management produced no response, not even a “thanks for your comment”, which is unusual behavior for a major company.

    If we are doing something wrong we would obviously appreciate any clue as to what that thing might be. Google says they are implementing a program to advise sites of their sins. How big a deal could it be to tell us?

    Obviously, it has occurred to us that Google might be blocking access to our directory merely because it is a directory or for some other “editorial” or “business” reason. Their web site clearly states that a site can be delisted for secret reasons and that they can decide that something is “abuse” even if they have never disclosed anywhere that they consider that particular practice to be “abuse”. However, if Google has an editorial policy shouldn’t they so state? Wouldn’t it be even better if they told us what it was?

    Has anyone else had experience with a long established site being suddenly delisted for unknown reasons? Does anyone have a clue as to how to resolve this problem? Any help appreciated.

71. Kelly Jones Said,

    October 5, 2005 @ 7:26 pm

    Yes Ted, we’re in the same boat you are. We haven’t been banned but moving thousands of our listings from the first page of Google to positions #350 and above has killed our business this month. So we’re not banned but we’re being punished without any idea why and no idea how to fix it. We haven’t spammed anything, haven’t hired any SEO companies, etc., etc. Google has not responded to any of our requests so right now we’re kinda screwed. It would be nice if Google gave us some form of hint as to what’s up but their silence is making fixing anything impossible.

72. Andi Said,

    October 5, 2005 @ 8:08 pm

    After having suffered a 54 day ban and then being reincluded without ever being informed as to whether I was at fault for any specific thing or whether anything I did during the ban helped or hurt I have to wonder whether Google has any clue at all how this is eroding good will among the people it will probably need in the future.

    Google does not appear to have peaked, but no company stays on top forever. I don’t think that five, ten or twenty years from now I will have forgotten the hell I that lived through during August and September 2005, or that may await in 2006, etc.

    I am a very forgiving person and I very much want to view Google as the wonderful and friendly company I’ve known all these years. Just a little
    more official communication would certainly go a long way toward keeping me (and all these others) on your side as I very much want to be.

    The blogs are a step in the right direction but I still feel there is an adversarial relationship where there once was a sense of common purpose: helping people find stuff.

    The silence leaves the impression that there may be chaos behind the green curtain, please disabuse me of this notion.

73. Ted Goldsmith Said,

    October 6, 2005 @ 7:46 am

    Kelly and Andi — I agree. I am a fan of Google and even have some stock (a great investment so far). However, a healthy company does not blow off serious inquiries from suppliers and web site owners are basically suppliers for Google. There seem to be problems in Mountain View although anybody Microsoft hates can’t be all bad.

    I have also noticed that the quality of Google searches has been declining and there are ever more garbage sites popping up in high ranking results. Google seems to be gradually losing the spam war. I suspect they are putting most of their effort into diversifying into email, maps, video, etc.

    I also think Google, as industry leader, should be doing a lot more to cooperate with site owners to improve search. There is a title tag and a description tag but why aren’t there a whole bunch of tags (geographic location, category of site, etc.) to allow site owners to communicate more information to the search engine. If Google adopted such a tag it would be an industry standard the next day.

    My understanding is that Google admits to manually banning (delisting) individual sites but they claim that their ranking algorithm applies equally to everybody. This presumably means that the ranking algorithm does not have a place where it says “check if this site is on the bad sites list” or a place where it says “check if this site is on the good sites list” as it is computing rank.

74. Kelly Jones Said,

    October 7, 2005 @ 4:39 pm

    Received the following from Google today (finally):
    “Thank you for your reply. Your site is not currently penalized, and
    multiple pages are included in our search results.”

    Nice stock email. They obviously didn’t do more than ten seconds worth of research to determine that we weren’t banned. However, when your traffic from Google goes from 1,000 page views/day down to 50 (and those are from adwords PPC), something is very wrong over at Google and it’s too bad they’re not more forthcoming about it.

75. Alexo Said,

    October 7, 2005 @ 6:33 pm

    Hello There

    2 Kelly
    >>>“Thank you for your reply. Your site is not currently penalized, and
    multiple pages are included in our search results.”

    hmm .. u got such email from G ?
    amazing …

    >>>So we’re not banned but we’re being punished without any idea why and no idea how to fix it. We haven’t spammed anything, haven’t hired any SEO companies, etc., etc. Google has not responded to any of our requests so right now we’re kinda screwed. It would be nice if Google gave us some form of hint as to what’s up but their silence is making fixing anything impossible.

    the same is here and no idea, what can be the reason of this

76. Kelly Jones Said,

    October 7, 2005 @ 8:13 pm

    Ted — I have to agree with you. Over the last year we’ve watched the quality of the search results in Google decline rapidly due to spammers. It seems every time I do a search the top results which come back are old domains which have been bought for spamming purposes. What’s sad is that in order to fix this, they’ve thrown the baby out with the bathwater. Personally, I think it’s time everyone woke up and started using a different engine as Google has been resting on their (organic) laurels for a looooong time. Maybe a bit of bad press would sharpen their PR skills a bit — we can all see they need it.

77. John Rooke Said,

    October 10, 2005 @ 12:40 pm

    We’ve had a back and forth discussion with Google over the last few days and the upshot is they are insisting there is no way to re-set a url removal, even though they admit it is not clear that removing the non-www homepage also removes the www homepage.

    I’ve even read of a user at webmasterworld who managed to get their accidental removal reversed within a few days of asking so we know it’s possible.

    We’re at a loss what else to do as we know they must have a database somewhere with the www homepage url and a NOINDEX and the start date somewhere (as otherwise how will they know the 6 months are up and to put the page back in the results). Surely all they have to do is re-set this for our homepage and let Googlebot pick up and index the page organically.

    So, we end up with a homepage which is not indexed, coming up to the busiest time of our year, through no fault of our own. How fair or reasonable is that.

    John

78. Matt Said,

    October 10, 2005 @ 1:46 pm

    John Rooke, I’ll try to do a post about that. Doing a reinclusion before six months involves going in and editing the url removal database, which we’re usually loathe to do. Is the domain that you’re talking about the site that you used in your comment?

79. thomas Said,

    October 11, 2005 @ 3:36 am

    maybe gonna loose my job, but still trying. hi Matt. I just jumped in your post while searching for the solution to my big problem. I’m rewriting our website totally in css, but what’s really making me go mad is that we can’t figure out in google… has surely much to do with the content of your post, but even if we followed all procedures nothing changed. If I will not succeed not only to perfectly write our code in pure css but even to index the company website I will probably be home before christmas.. but what are we doing wrong?
    Putting into google : http://www.xxxxx.it

    google finds the page but without related link and without cache copy… and it’s the true result.

    Puntting into google: http://www.xxxx.it/k/

    google finds a cached copy (12th july) that takes to our homepage. for this research i can find even links, etc etc etc..

    What have I got to do to make google discover that the first result should lead to our domain with related links.. and that the second query should be removed from index??

    HEEEEELPPP!!!

80. John Rooke Said,

    October 11, 2005 @ 10:25 am

    Matt

    Yes it is. Our requests seem to have got through to the right person at Google in the end as our www homepage is back in the index as of this morning. We’re absolutely delighted.

    I think we will be steering well away from the url removal tool in the future as we do not want to go through what we have gone through in the last week again.

    Thanks Google…all is now forgiven.

    John

81. Nancy Said,

    October 12, 2005 @ 9:39 am

    Matt, I love your blog, and have learned so much from this Reinclusion thread - I followed it closely to submit a client site. I think I just got the final death notice. I have been anxiously awaiting the “big reply” from Google - the all-important 2nd reply - after sending in the Recinclusion Request, then replying to the “canned reply”. I had big, exciting hopes. Another blog posting about a succesful reinclusion from “blogpingreview” made me really confident. Well, I got the email early this morning. Here’s what it said:

    “Thank you for your reply. Unfortunately, we are unable to send personal responses to all of the requests we receive to review individual website content. Websites can fall out of our index for many reasons, including penalization.

    Certain actions such as buying or selling links to increase a site’s PageRank value or cloaking - writing text in such a way that it can be seen by search engines but not by users - can result in penalization. Please review our quality guidelines at http://www.google.com/webmasters/guidelines.html for more information. If you identify problems with your website and make the changes necessary to comply with these guidelines, please do not hesitate to contact us.”

    NOOOOOOO! Wait! Don’t go! What happened to the one where you say “we’re passing your problem on to our engineering department”. Are you eluding to something that might be a problem? Is this a hint for me to read between the lines and find something that still needs fixing? Is this just a canned reply? Are we through?

    Here’s the thing - I don’t think we actually did anything wrong on the site. Well - maybe one possible thing - so I pleaded total guilt to it and asked for reinclusion. We got dropped early this year. Totally - Googlebot comes - we aren’t indexed. It’s a very nice, old, family friendly retail business. The owner originally bought and used 3 URL’s for the store 6 years ago. Well you just can’t do that. It’s considered duplicate content. I slapped his hands this summer, and took the site over - again. The 2 old url’s (which hadn’t been used in 2 years) are redirecting to the correct one. Is the redirect perhaps incorrect?

    I wouldn’t know how to make something cloak if my life depended on it. I know what it is, but wouldn’t have a clue how to do it. We’ve never bought or sold any links for anything. We hardly have any on - at this point we’re afraid of most everybody’s links. The only really horrible thing that’s ever happened is back in 2002 we were duped by a company called Traffic Power. We were too dumb to know much about SEO companies back then. Once I saw what they were up to (everything in the manual of what not to do) - I was horrified and dumped them as quickly as I could. We lost plenty of money on that scam. I thought I cleaned up all of the nasty mess they made. Could something from that long ago still haunt us? I’m at a total loss here. We’re about ready to shut the company down and start over from scratch. Any advice? Any tools out there that I could use to check for problems I don’t know about? Is there a single thread of hope we’ll ever be in Google?

82. Matt Said,

    October 12, 2005 @ 11:59 am

    John Rooke, I asked someone to go look into your question on the url removal database.

83. Matt Said,

    October 12, 2005 @ 12:02 pm

    Nancy, did your site have linkpool pages in the past?

84. John Rooke Said,

    October 12, 2005 @ 2:04 pm

    Matt

    Thanks very much. It’s very much appreciated that you took the time out to help us as we were beginning to despair.

    Is there a possibility of a few extra things being added to the url removal console in the future to help with this kind of situation. The kind of things I am thinking about are:

    1) When a site is showing as ‘pending’, have the ability to remove the site from the pending queue e.g. if you have submitted a page in error and then realised your mistake, you could stop the page being removed before it’s too late.

    2) Allow a site owner to stop the 6 month page removal i.e. reverse the removal. I can’t see any harm in allowing this and it would stop what must be quite a few email requests from webmasters to Google. This would negate the need for 1) above.

    3) Make it more obvious in the url removal console that removing nonwww also removes www as it does not say that anywhere I have looked.

    Having been though what we have in the last week, these suggestions would have been a lifeline for us and enabled us to solve the problem (which was of our own making) ourselves.

    John

85. Kelly Jones Said,

    October 12, 2005 @ 6:05 pm

    Matt — it’s obvious that Google has implemented some new filter. For obvious reasons, Google may not want to openly discuss this filter. However, for many of us, we’ve obviously tripped something in the filter and are suffering greatly for it. The cost to our business in the last month alone is over $10,000 in lost revenue. Unfortunately, we have no way of knowing what is wrong so that we can fix it. I know that Google doesn’t owe any of us anything but being silently punished without any knowledge as to how to resolve the situation doesn’t make sense. Worse yet, the communications from Google state that we’ve neither been penalized nor blocked. Maybe this is an argument of semantics then and that somewhere inside Google moving a site from page one in thousands of results to positions 350+ is called something else. Is it “a flogging?”

    We’re honestly really baffled here. Our site has been in it’s present format without change (other than content) for years. About the only thing we’ve done is to add printable pages for travelers (as we’re in that industry). So I’m stuck fumbling around in the dark trying to figure out how to be un-penalized from a company which openly tells me they haven’t penalized me when they obviously have.

    Matt — how on earth do we resolve such situations?

    KJ

86. Matt Said,

    October 12, 2005 @ 7:31 pm

    Good points, John. I’ll pass that feedback on. KJ, you didn’t give any specifics about your site, so it’s hard for me to know more. We do roll out changes continually though.

87. Kelly Jones Said,

    October 12, 2005 @ 10:11 pm

    Hi Matt:

    Two things:
    1. Because your site will be indexed by Google among other search engines — I can’t possibly reveal what our domain is. The loss of positioning in Google is bad enough but for our entire industry to know about it and openly discuss it would truly devestate our company. Yes, it really is that bad. I’ll gladly reveal anything you want via private email but since I don’t know your email address I would need you to email me first.

    2. I’ve run into something strange on Google. We created a new web design for a client (redesign). Generally our web designs perform very well in Google but something is wrong with this client site. I first noticed the problem when the site performed very poorly in Google. While diagnosing the problem, I did a backlink check and noticed Google only listed two of their own pages (home and another one) and one other site. Knowing this couldn’t be true, I did a search for just their domain as a phrase and found 147 sites which actually link back to them. I check and most of them are valid links (not scrapers grabbing their adwords links) from sites like their local chamber. However, this sites aren’t listed in the backlinks. I’ve emailed support about it and received the stock reply. Honestly, this one has me truly baffled.

    KJ

88. Nancy Said,

    October 13, 2005 @ 8:14 am

    Matt, I’m pretty sure that’s what Traffic Power was doing back in 2002. A bunch of funky pages with nonsense writing, thousands of links to everywhere, etc. The first time I found one that came up during a search in Ask, I flipped out and pulled the plug on it immediately. (They wouldn’t refund our money - the smooth talking sales guy we had spent hours with on the phone before we signed up with them had left the company - all we could do was send a letter to the Attorney General in Nevada.) We were only involved with them for 3 months. They left a really bad taste for the SEO industry - there are probably some good companies out there, but I would never know who to trust. Hence the reason I am limping along on my own. Naive, honest, and failing.

    The site was getting indexed and had an okay page rank until early this year. At this point, I would be thrilled with a PR 1, and having our returns even on page 100 - anywhere in Google! Could those linkpool pages still be haunting us in 2005?

89. Nancy Said,

    October 13, 2005 @ 8:40 am

    Hold - the - train!! I should have started working this morning before seeking advice. I just pulled up our website and lo and behold! We have a beautiful, tiny green bar! It’s the cutest little thing I’ve ever seen. This is one of the happiest days in my life! Two whole pages from our site come up when I type in site:www.oursite.com!!! It’s just a tiny start, but we’re back! Thank you Matt for your great article and advice. Good grief, I can’t believe I’m going to party down tonight because of a tiny green PR 2 Google bar. So, all of you out there complaining about a low Google PR, don’t forget - you could have a plain old gray bar. My glass is now more than half full. Green is beautiful…

90. thomas Said,

    October 14, 2005 @ 2:06 am

    Matt.. desperate desperate desperate.. may someone of you guys (who certainly have more and deeper knowledge than me) give some advice for what to do with my domain over google.. who catched the wrong entry page? I’ve recently posted a request in comments but got (sob sob) ignored. while yahoo and msn seems to appreciate all i’ve been doing to clean code and do the best for indexing I get with google who gives strangely good page rankings within the domain in sub-category and category pages.. but still ignore (even after sitemaps) to clean the cache results for the domain…

    1 - if I put on google: http://www.DOMAIN.it

    google finds the page but without related links and without cache copy. and this IS the real homepage…

    2 - if I put on google: http://www.DOMAIN.it/k/

    ..which is a variable….used for creating for users and spiders (so no tricks…) pages… google finds the page, and gives a cache copy back to 12th july… even links and so on…

    whom can I ask, who may give a little support to understand how to get out from this mess?

    best aLL

    Thomas

91. Brian M Said,

    October 14, 2005 @ 11:00 am

    Hi Matt,

    Before I submit a re-inclusion request, is there any way to find duplicate URLs that point to the same web address and cause a duplicate content penalty?

    A few years ago, my predecessors purchased numerous URLs and pointed them all at my company’s home page, but those employees have all left, and nobody knows what all the URLs are anymore.

    I have re-directed all the URLs (that I have uncovered) to a landing page in order to let Googlebot update its cache and remove all of the duplicate content, and this seems to be working (only two more URLs remain to be updated). I plan on submitting a re-inclusion request once I see that those remaining URLs are updated, but I am afraid that there may be even more URLs out there.

    Thanks in advance for your help and advice. Great blog!

    P.S. googlemap.xml has been a fantastic help in eliminating the internal duplicate content caused by some links in the site having CapitalLetters, while other links were in lowercase (the site is on a Windows server, so it didn’t matter to the site’s original programmers).

92. Will Spencer Said,

    October 16, 2005 @ 1:19 am

    Brian:

    I use http://www.copyscape.com for that purpose.

93. Dave Shields Said,

    October 17, 2005 @ 1:33 pm

    For years I was the webmaster for a Brantford florist’s web site which enjoyed #1 ranking on the big 3. Early on I wanted to take the same website to other communities (cities), as the results achieved and the cost were far lower than methods used by the major city to city florist referrers. This year I put the plan into motion in August. Each site is the same except for prices, addresses, names, and telephone numbers.

    Didn’t even think to check for Google guidelines, as it was very obvious that the majors had nearly identical sites in different cities. Plus all my sites have been squeaky clean for years.

    In mid-August we sold our first site. Marketing then suggested we put up a few more sites so that potential out-of-town clients could see exactly what they woould be buying. All extra sites had dummy data for names, addresses, phone numbers etc.

    The starting off rankings in Google were much less than I am am used to. Research lead me to the aversion Gooogle has to duplicate content. As a means to make pages less similar, I created artifical names, address, and non-working telephone numbers. In a matter of days, all the sites except 2 diasappeared, and the remaining 2 went to 100+.

    NEVER include a phony address or telephone number on your website!!
    All home pages have unseen links to password protected areas where clients can add comments to their pages and change prices.

    As this is written, 3 sites have rankings betwee 41 and 48, 2 are way over 300, and 9 have been indexed, but cannot be found in Google even when viewing EVERY entry Google returns.

    My first question is how can I fix this mess? Secondly how can I persuade Google that similar content on websites for unique cities are not spam, as the content is no good to anyone outside that city. Lastly, the big guys in the flower delivery business are providing similar content in different cities. I am looking for a level plaing field for the smaller mom & pop flower shops.

    All comments are appreciated. Excel worksheets are available at
    aquilifer-web-design.com/xls/

94. Kelly Jones Said,

    October 17, 2005 @ 3:47 pm

    After having received the “nothing is wrong with your listings” email from Google I sent them proof showing that our listings/site have been severely penalized. Today I received the following from Google.

    “Thank you for your reply. Please be assured that your site is not
    currently banned or penalized by Google.”

    — Deep within their own denial? Refusal to help people? What? It’s so frustrating knowing that something has gone seriously wrong even though we’ve done nothing malicious or underhanded.

    Matt — you stated the following:
    “Remember, we need to know 1) that the spam has been corrected or removed and 2) that it isn’t going to happen again.”

    Obviously we’re not talking about a site which has been actually removed from the index but the result is basically the same. So given your advice, how can we possibly admit to having done anything wrong when we don’t know what it is? Furthermore, how on earth can we correct it?

    This is like being spanked without being told what you’ve done wrong and when you ask, the parent tells you everything is fine and that you were never spanked. How weird is that and how on earth do we fix it?

    KJ

95. Brian M Said,

    October 18, 2005 @ 8:01 am

    Thank you very much, Will Spencer, for that link to http://www.copyscape.com. I have used that many times (I have been struggling with this problem for almost 4 months). Unfortunately, it never once found the duplicate URLs that pointed to the same place.

    Today, Google’s cache for two old URLs shows exact duplicates of our home page. Google also stopped updating the cache of our current home page (once again), and it has gone back to showing a description taken from the DMOZ instead of from the META description. If I didn’t know that these URLs existed, I would never be able to find them, so I still need a tool that will help locate duplicate URLs. Or, please tell me where I can sign up for webmaster notification when a problem is detected.

    Also, I am afraid to use Google’s URL removal tool on these URLs, because the cache for these is identical to the actual home page, and I do not want the real site to be removed for 6 months.

    If webmasters could create a googleremoval.xml file to remove old URLs, it would speed up the entire process and clean out the supplemental index as well. The same rules for googlemap.xml would apply (place it in the root directory, etc.).

    Matt? Help!

96. mobile-heaven.com Said,

    October 19, 2005 @ 2:48 am

    hey Matt n everyone is there a place where banned sites are listed or where sandboxed sites are my site has been live 6 months http://www.mobile-heaven.com, can be found in google but if you type http://www.thesite.com but no cached pages or description any reasoning why??? thanks m8y

97. canboy Said,

    October 19, 2005 @ 10:47 am

    Serves all you whiners right for trying to fool Google. G will become smarter, so why not focus on quality content instead of tricks to make G think the site is good.

98. canboy Said,

    October 19, 2005 @ 10:50 am

    mobile-heaven.com is a site that’s mostly affiliate links. G is cracking down on this! You have little original content, so you won’t rank.

99. Jonny Said,

    October 19, 2005 @ 11:02 am

    Please don’t believe all these fucking lies. They are excluding thousand of sites to make sure that you will buy adwords to be sure to have your site there. There are no sites without problems, all the sites have problems and all the idiot webmaster think they have been excluded for these problems… they want your money because when the sell their bonds every 50$dollars of adwords they earns 50.000$ by investors… Please, google, try to be a serious company!

100. Kelly Jones Said,

     October 20, 2005 @ 1:22 am

     Sorry Jonny — I can’t agree with that assessment. We’ve been an adwords users since the beginning and love the program as do all our clients. Great system and great compliment to our organic listings. Or, our former organic listings.

     And Canboy, we didn’t try to fool Google. Our site is ten years old and the current design and content is four years old. Something tripped Google’s filters and wham, we’re at positions 350+. It’s the denials from Google that anything is actually wrong that’s bothersome.

     KJ

101. Nick Said,

     October 20, 2005 @ 2:48 am

     Thanks for this URL http://www.google.com/support/bin/request.py it’s really helps me for one of my friend’s website. Because his website PR suddenly has washed out. I will pass this URL to him.

102. Aaron Pratt Said,

     October 20, 2005 @ 7:27 am

     What is considered an automated link program?

     Wow, I just started visiting the “search engine forums” out there again and found a link to this blog by Matt, all I can say is excellent!!!

     I know you most likely will not have time to answer this but I am going to put it in here anyhow and even give the URI. I also am going to come out of the closet to find out what is right and wrong because I believe I have good, honest websites that are of value to indexes AND humans.

     Quick Story:

     I made a website for a neighbor who makes great gift baskets for dogs, which I will transfer to her once it succeeds.

     Site came alive on 9/4/2004 and was sandboxed for a year and now is about #150 for it’s needed search term (but just wont move any higher).

     The site has a blog (on a subdomain), and a directory (on a subdomain) that uses “free php directory script”, my idea was that eventually I can break away the “pet directory” and it can earn me some income (adsense) to support what I do for free. On occasion I do allow people who have related sites inclusion into this directory. Is this considered this considered an automatic link farm? Please help me out here! I feel that offering a focused (on topic) directory is a good alternative to “link trading” with lame Viagra sites that spam me every day.

     AND I am sorry for making fun of Matt in the “search engine” forums, but those geeky Halloween pics kick major ass dude!! HA!! It’s good to know you guys have a sense of humor and are real, real geeks!

     Warning: As I have noticed with DMOZ, you can start off with good intentions (as they did with their inclusion “status check” forum) but if it is not done correct you will soon have an angry mob weilding pitch forks and axes at your door.

     -Aaron

103. David Nelmes Said,

     October 20, 2005 @ 2:08 pm

     With this last update Jagger1 on Oct 16th, my website lost all top 10 positions and is still listed with google, but mostly after 300 to 500 or more listings. i just submitted a re-inclusion report and i will report back here from time to time to document how this is going, but I can guarantee you that our website did not perform anything intentionally to spam Google.

     On another note, in the midst of this 80% loss in traffic, my recent revisions to the site have worked well and three days after having turned into a virtual ghost-site, I now have a PR6 for the home page….but with almost no Google traffic at this time.

     The good from this is that I did not realize how dependant i had been on Google for traffic and I have reinitiated my efforts to find as many alternative methods of traffic from other indexes and link exchanges. i have no plan to eliminate Google from my overall plan, but i think it’s dangerous that a single action by one company has the power to eliminate my capacity to run my online business. i think it would be prudent for everyone to embrace the google traffic, but find as much non-google traffic as possible as well…and not other traffic that itself is dependant upon google.

     I’ll be back.

104. Motion Hosting Said,

     October 20, 2005 @ 2:30 pm

     Well, my site disappeared from search results about a week ago and although it is still in the google cache and is still displayed its search positions have dramatically changed. I contacted google on confirmation of what happened to my site, but still nothing…

105. Phoenix Said,

     October 20, 2005 @ 2:34 pm

     Hi Matt,

     My website doesn’t use any hidden text, hidden links, or cloaking. However I lost my pagerank 6 six months ago. I can’t find it even if I seek my domain name ?

     I send reinclusion request at support/bin/request.py like you suggest but I have always pagerank 0.

     Can you just watch quickly my website and tell me if there is a problem for Google? Something I don’t see ?

     Thanks in advance for your help and advice.

106. Brian Said,

     October 20, 2005 @ 5:38 pm

     My url has been used by I suppose a competitor to spam blogs. I believe google just penalized me for it. Mydomain.com is page rank 3 my index.php same page right is page rank 4. I have way way more backlinks to mydomain.com. I have been receiving angry emails from blog owners claiming I left them a comment with link to mydomain.com. I have not left a single comment link a.k.a. comment spam. Google I beleive is punishing me for it. Apparently you can ruin your competitors site rankings and page rank. Just spam there url to blogs. Any thoughts on this. Any help on how to get google to investigate.

107. R.Anand Said,

     October 21, 2005 @ 12:30 am

     Dear Matt,
     I am from India and I have a problem simmilar to KJ. My site http://www.arounddelhi.com had more than 60 keywords ranking in the top 10 and suddenly we are left with almost nothing but the domain name keywords only at the top. We have made no significant changes in the web site only adding more travel pages and recently added adsense. I know i am not banned as no of pages in google remain same and some keywords are no. 1 still. Also the sites which are now ranked much higher than us, some of them are not even operational and some give totally irrelevant results. Show me a way out.

108. Paul Said,

     October 21, 2005 @ 6:12 am

     This is thanks for great recent improvements. Many of the spam sites that were in the top search results above mine have now disappeared. My site is loaded with original, daily updated content. Recent changes at Google seem to be suceeding.

109. Bob Larson Said,

     October 21, 2005 @ 10:25 am

     Matt,

     I do not envy your position. Doing PR for a company trying to balance two mutually exclusive activities; relevant results & clicks on paid advertisements.

     You may find interest in this.

     One of my sites has some affiliate links but provides more quality information than most of the G listed competitors. It generates a thousand uniques per day, yet none from G natural. 16 months old, PR5, tens of thousands of G indexed pages, 20K GBot hits per mo.and not one keyword ranked. Not even exact unique strings.

     Now here is the good part. A search for the unique phrase URL alone gives us #1. Add any other relevant word to the URL and our page is eclipsed by the scrapers. 40,000 results with our unique name and we are not one of them. wow.

     But alas, I have lost all my animosity towards Google many, many months ago.

     My best wishes to you Matt. You are trying to make a difference, albeit with your hands tied.

110. Aaron Pratt Said,

     October 21, 2005 @ 11:46 am

     (I linked my personal blog in the URI to use as an example if you like)

     My website (Wordpress blog with comment and other fields removed) on “rainwater harvesting” dropped from page #1 in google to page #3, it also has a directory that uses the “free php script” which makes me wonder about how safe it is to do focused directories that are on subdomains? AND yes I add people to this directory who have sites related to “environmental issues” if they are kind enough to link to me it can get a few visits in times of trouble like this. The only site I link out too from the index.php are mine and 2 friends, I have completely given up on the idea of “link trading” which has the word SPAM written all over it, besides it just feels wrong now.

     I also lost 80% of traffic from Google as well but the site is now a pr5 and most of the internal pages are pr5, I wonder what “PR” means, it seems to have hosed all my results. Is PR a bad thing? I have heard that Amazon.com constantly changes it’s pages to get fresh results, I would consider that spamming yes? In order for a site to stay fresh do you need to change the names of the pages and have high dominating