Bacon polenta

(another post-without-particular-polishing)

You have to leave room in your life for serendipity sometimes. For example, without an accident, I never would have discovered the joy of bacon polenta. (All the people who say “Matt, I thought you were going low-carb?” can step off. I’m playing hockey today. I’ll skate it off.) The Google cafe may like to call it creamy polenta, but as a Southern boy, I just think of it as cheese grits. It’s even better with bacon. Mmmm. Bacon-y goodness.

Accidents in crawling/indexing/scoring happen too. Sometimes they’re happy: “If we turn this factor off, scoring gets better? Cool!”. Sometimes they’re unhappy: “What happened to this page?” One of my least favorite accidents is when someone reports a 301 or 302 problem. The heuristics we put into a place have greatly reduced complaints about “302 hijacking.” For the first time in ~1 month, I got an email about a “302 hijacking”. This case was especially interesting because I got an email from both sides: someone from the destination site wrote, and the source site also wrote to say “we didn’t mean for this to happen.” I take that as a kinda good sign; when I hear about it from both ends, 302 problems are hopefully much more rare. I passed the info on to the mailing list we have for that, and I’ve asked a colleague to email both sides when we get it debugged.

What do you do if you suspect a “302 hijacking” but don’t have my email address? There’s a convenient way that should get your report to the same engineering list, where it will get the same level of investigation. Go to http://www.google.com/support/bin/request.py and click “I’m a webmaster inquiring about my website” then select “Why my site disappeared from the search results or dropped in ranking” and click continue. In the webform that you get to, make sure you put “canonicalpage” in the Subject line, then put the details in the Message body. Someone will route that message to an engineering mailing list where we dissect claims of canonicalization problems (that is, picking the wrong url).

I also got one email today about a site being indexed under both www.domain.com and domain.com. The proper procedure (assuming that you want www.domain.com to show up) is to make domain.com do a permanent (301) redirect to www.domain.com. The person that wrote said that we hadn’t crawled domain.com recently to find the 301/permanent redirect. I’d be curious to hear feedback (in the same way as the paragraph above) to see how many other people are running into this issue.

64 Responses to Bacon polenta (Leave a comment)

  1. >The proper procedure (assuming that you want http://www.domain.com to show up) is to make domain.com do a permanent (301) redirect to http://www.domain.com.

    Hey Matt,

    You may want to throw up a few examples of how to do this (great advice btw), I know Marcia had a nice blog entry on how to do it on unix (didn’t want to link drop it here, but I can if it’s okay)…

    It’d be nice to see how to do it on IIS…and REALLY nice if you could show an example of it on Lotus Domino (yes, I have a client who has a site STILL hosted on this beast).

  2. Charlie Vonn

    Funnily enough, I JUST ran into a similar problem today. Using Apache, I had moved some file directories around using mod_redirect – but I did a [R] instead of [R=301] – our new pages were being “sort of” indexed by Google but had no cache version and wouldn’t come up in a search result unless I forced it to show, but the old outdated version wasn’t there. I banged my head against a wall until I discovered that [R] defaults to a “temporary redirect”… oh. I changed this in the httpd.conf and I HOPE Google will pick up on it and give me back my pagerank.

    This isn’t really the same thing as the real redirect problems (pagerank theft etc), and it’s completely my fault (or Apache’s for making [R] default to the lesser used temp. move, I guess I could complain), but it’s the sort of thing that can drive a newbie webmaster insane. Redirecting is ugly business, and I think Google is doing a good job fixing the theft issue as far as I can see.

  3. Engineer

    We set-up a 301 redirect in March.

    Three weeks ago we got slammed with the canonical issue with cache dates of the non-www version from January. Also, pages that have been taken down many months ago are showing up with cahce dates of January.

  4. Hi Matt,

    3 of my sites got into the double indexing (www and none) nightmare.
    Here is the code I used in the .htaccess to try to correct the problem.
    But Google is sometimes very long to take into account a 301.

    RewriteEngine on
    RewriteCond %{HTTP_HOST} !www.site.com$
    RewriteRule (.*) http://www.site.com/$1 [QSA,L,R=301]

  5. Brian

    Seems clear that for sites experiencing diminished rankings as part of Google’s quality improvement efforts, “301/302″ and “duplicate content” are the SEO industry’s strawmen-du-jour…

  6. Daniel

    Hey Matt,

    I’m new on the site. Can you post your thoughts on the new book – “Search Engine Marketing, Inc” by Mike Moran and Bill Hunt?

    Thanks.

  7. Interesting that you bring this up Matt. I just had a friend email me about the fact that they have a PR 4 for a domain without the www and a PR 2 for the domain with a www. When I do a site: check on the domain, I see pages for the www. and for without the www. Interestingly enough there are no IBL’s for without the www.

    I will have them do a 301 and get back to you on the results.

  8. Mmmm BACON Good. 302s Bad.

  9. Naw

    Matt, if an domain got penalty by duplicate filter, after fix non-www redirect, when www will back normally?

    BTW, thanks, your blog is very useful.

  10. Tell the health freaks to step back & relax… bacon is a vegetable.
    http://www.dieselsweeties.com/shirts/

  11. Hi Matt,

    We’ve had a lot of clients who seem to have been hit with the duplicate content issue on www and non-www versions of their sites. It’s good to see you talk about it here. Todd is right though, this is a larger problem for those on IIS. I have used Isapi Rewrite before but this is an extra cost for smaller webmasters and requires some liaising with the sys admin.

    And thanks for the tip on the polentas – not sure we have corn meal in Russia but after finding mascarpone anything is possible…

  12. Stephen

    Hi Matt

    Thanks for visiting this topic :)

    I think I was caught with an indexing attempt on my non-www earlier in the year and I have since done a 301 but my rankings have not improved. If G does find and index the non-www does/can this lead to a duplicate content penalty ?

    Just cant work out why things have not improved yet for the site in question. I think a lot of others are stuck in the same boat aswell.

    It is also worth noting that you have not got a 301 redirect from you non-www to your www. If you do a site:mattcutts.com -www search you will see that the index recognizes that the non-www exists – can this lead to problems also ?

    In addittion even when the 301 seems to be picked up – eg I do a search for a domain example.com and the cache shows http://www.example.com – you can still do a search for example.com -www and find that Google still knows about the non-www version.

    I am not sure really why a penalty results from this problem – some of the theories I came up with is that Google thinks the non-www is the canonical page (often the non-www can be PR0) – therefore rankings drop – or as mentioned above it leads to duplicate content penalties.

    He he – you may have also seen me posting at WMW about this as it is something that I have a bit of a obsession about – the thread at WMW is :-

    http://www.webmasterworld.com/forum78/9632.htm

    I would appreciate any light you could throw onto the situation.

    Thanks for your help.

    Cheers

    Stephen

  13. Matt, my website is an example of this – rockforums.net has been redirected for a while now using

    RewriteCond %{HTTP_HOST} .
    RewriteCond %{HTTP_HOST} !^www.rockforums.net
    RewriteRule (.*) http://www.rockforums.net/$1 [R=301,L]

    but rockforums.net hasn’t been spidered/updated yet

  14. Stephen

    Matt

    Just to expand on my point futher – this site is a perfect example of whether Google can determine that the non-www is the same as the www or not.

    Currently if you do a search on “mattcutts.com” it will show details for the http://www.mattcutts.com page – OK – perfect, however as explained in my first post – the index does know that the mattcutts.com exists by doing a site:mattcutts.com -www search. So even when it appears G is handling it correctly there seems to be some doubt.

    Also if the indexing of the non-www does cause a duplicate content penalty there are a lot of sites out there that have been wrongly penalised.

  15. gpmgroup

    Hi Matt – Great blog, :)

    The natural way to deal with [domain.info] and [www.domain.info] in IIS is to use multiple identities, as these allowing the varying IP, port or host header.

    In order to do a 301 redirect you have to configure the IIS web server with two websites. The first one pointing to the directory with the files and the second with a [permanent] redirect pointing to first website & add parameters to handle subdirectories & query strings.

    From a theoretical point of view should it be a permanent redirect or can a website have multiple identities? In the real world one may wish to brand with and without the www to different target audiences, in which case would a 301 redirect be an accurate reflection?

    If there is more than one identity pointing to a site would it not be more elegant for the engine to choose which to index and put a single entry in their index for the alternative one(s)? Or even better combine these entries in a singe line within the identity chosen to be fully indexed?

  16. lots0

    One question Matt;

    Why does google use a 302 to redirect

    http://google.com
    to
    http://www.google.com,

    instead of the 301 redirect you recommend?

  17. Thanks for bringing this issue up Matt. I think that it is an entry like most of us want to see more of.

    Anyway, I have a similar issue with a site. It is not a 302 hijacking, but it is for sure some kind of hijacking (maybe a meta-refresh hijacking?).
    I added my website to a certain directory, which uses meta-refresh to redirect to the destination url in a given entry. For about a week now, my site has the Pagerank of the directory entry and the url in the cache of my website is that of the directory entry url.
    However, it is still my website which shows in the SERPS, so I see it as a semi-hijacking with no major side-effects (so far).
    I was wondering what you can tell us about the meta-refresh hijacking.

    P.S. It is not the website that is linked from my name in this comment, though it is also a PR5 site.

  18. Stephen

    Hi – Matt – are we just giving feedback or do we expect feedback on our feedback.

    Cheers :)

  19. Out of curiousity – why does google use a 302 instead of 301 for their redirect?

  20. nutkeis

    Here’s a somewhat related dilemma. Let’s say I run an e-commerce platform that can be customized in look and feel for each affiliate who uses it – “private labeled”, as they say. Now, let’s say that this capability extends only to the action/form pages. In other words, the core revenue generating process consists of many, many duplicate pages (x number of actual pages * y number of affiliate tracking codes).

    No problems there, as I don’t care whether or not these internal pages are indexed in search engines. All search engine traffic comes by way of our actual site content anyway, and this is not duplicated.

    Until recently. Let’s say we have a few “major players” who want to customize the URL as well, so we create majorplayer1.mysite.com and point it to the root of our domain. In theory, all traffic to this new subdomain should be either from a form or link that contains the appropriate affiliate tracking code and points to the private lable action process, creating no more of a dupe content situation than we already have.

    The problem is that Google has now indexed all subpages of majorplayer1.mysite.com, not just majorplayer1.mysite.com/Form?AffID=majorplayer1. Since both the domain and subdomain resolve to the exact same codebase, all of the unique content on mysite.com is duplicated by majorplayer1.mysite.com.

    The question is, how do I get majorplayer1.mysite.com out of the index without affecting mysite.com? Robots.txt is shared, so that’s out. Dynamically including a noindex tag or 301 redirect on every page is cloogy, and I’m pretty sure that doing a 301 at the IIS level will have the same effect as simply not having the subdomain in the first place. Any thoughts?

    P.S. I lived just down the street from the GooglePlex for 5 years, and you guys never once invited me over for coffee. Sheesh. Some neighbors…

  21. I am not sure really why a penalty results from this problem – some of the theories I came up with is that Google thinks the non-www is the canonical page (often the non-www can be PR0) – therefore rankings drop – or as mentioned above it leads to duplicate content penalties.

    I agree. http://www.referatde.com

  22. Shri

    Matt,

    How about enhancing the SiteMap protocol to support redirection and removal? I know, it bypasses the http spec’ed redirection, but it does make things a bit more foolproof for the people who complain the most.

  23. Hi Matt, people say the weirdest things sometimes, I just had to read the source. But you’re right – “low carb” is nothing but hype born out of the fact that people can relate to simple things… and lazyness. Exercise is what will keep you in shape, not consuming less carbs.

    Anyway, here’s a few helpful links about 301 and IIS I posted back in April – scroll to msg #380:

    http://www.webmasterworld.com/forum30/28742-38-10.htm

    IIS is generally not the webmasters’ best friend when it comes to hardcore tech. There’s just too many things that looks and feels “about right” without being exactly right.

    Speaking of 301′s – it seems that now some issues are cropping up with 301′s. Even though some of the 301 issues might be intended, or “by design”, some other part of it must be bugs of some kind – it certainly seems like there’s more to it than what Brian noted above.

    Perhaps the mail you get isn’t the whole picture? Don’t get me wrong, it’s just that I can’t imagine that it would be.

    From where I sit it looks like quite a few webmasters would have to be desperate before telling you – not that they’re not squeaky clean (some are, some are not), it’s just not their job to clean up Google’s mess especially as the Google of today can afford to hire a few thousand people to look at the issues. In general, my own feeling is that some people have given up on you guys by now, so they try to move on in stead. The people left behind are the ones that don’t have an aggressive strategy (or have adopted one), other sources of income, a lot of sites, or any knowledge about SEO at all – IOW the wrong people.

    Feel free to delete this post if you don’t like it to be here, no problem. I can never find a mail form on these blog thingies, otherwise I could have put the latter part in an email and left the IIS advice here.

    Nice blog by the way – one of the few worth reading. I’ve read a few entries before and even though there’s no big interesting trade secrets being revealed (didn’t expect that anyway) it’s always interesting to get a glimpse of the view from inside the black box.

  24. Matt, I have a suggestion for you (or rather for Google), it’s not necessary to blog post this comment, I simply don’t have your email address so I’m using this form.

    Take a look here:
    http://www.khiewchanta.com/linkthumbs/thumbs.html
    (The zoomy link pagey thing I did for Appon.)

    Imagine a search GUI where the user searches for “thai recipes” you use your mighty bandwidth to pull the pages down on-the-fly in parallel across a cluster, render them in Firefox (at a low thumbnail resolution and at a medium resolution) and either display them in a grid or mixed into the search results. Like some of the engines do now, but with mouse-over zoom.

    Appons mouse over zoom is basic, but imagine a grid where when one image zooms up the other images in the row and columns zoom down to accomodate it. That could be done in DHTML fairly easily.

    Now imagine that when you zoom in, it also makes space underneath the expanding image for a relevent text extract from the page.

    So the user searches for results, is presented with a page of small thumbnails, he can mouse over those thumbnails, they will zoom up and display the medium resolution thumbnail with the search extract.

  25. Stephen

    Hi Matt

    I feel bad a bout dominating this subject here and at WMW – but what has happened to Mirago.co.uk seems to be the same as what has happened to me.

    Is this a Canonical url problem ?

    Cheers

    Stephen

  26. After all: gmail.com not indexed (main domain isn’t, subdomains are).
    gmail.google.com is.

  27. I think rockforums.net is a good example, it seems the solution for this problem is embryonic

  28. We followed your advice completely concerning non www pages, as well as dynamic links that should never have been indexed to include editing our htacess file, yet a few days following such actions our site’s main page, index.htm has ben removed from the Google index. Now we are really confused.

  29. Matt

    Nigel Johnstone, nice app! Fun to play with. :)

    Shri, someone said something similar at SES San Jose and (crikey) I haven’t had time to email him yet. In theory, that’s a good place to do it.

    Chris_R, I’ll ask around. It could have something to do with handling French IPs by sending them to google.fr, German IPs by sending them to google.de, etc.

    Stephen, a www vs. non-www version isn’t something that would trigger a duplicate content penalty, in my experience–it’s just too common. I can believe that our algorithms might not always handle this as well as they could, but it’s not the sort of thing that would be penalized. As we work down the list of canonicalization issues that people run into and cross them off the list, I wouldn’t be surprised if this issue + 301s taking longer than before is the next thing on the list.

  30. Stephen

    Oops – sorry Matt thinking and needed to type more – cant edit posts here like at WMW – he he

    Yes, your engineers always insist it is not a penalty that I have got when I email them. But have you read my first post of symptons at WMW in this thread:-

    http://www.webmasterworld.com/forum78/9632.htm

    It very much looks like Google is having problems reading my site – now I dont know if this is due to the non-www issue or something else. Looking at those symptons would you agree that G is having problems with my site :(.

    OK – I waffle a bit after the first post :) as my frustration levels increased.

    Oh and I am starting to waffle now. EFV site you probably know about – it appears that site had a non-www problem, was fixed soon after the problem started and rankings returned. So it does look like a downranking occurs from the outside looking in.

    Now – I have a site which appears to have had the problem and went longer without being fixed – eg 301 redirect – this site has not returned.

    Food for thought – as if this is not a penalty the sites appear to certainly have ranking/crawling issues.

  31. Pat

    If Google wants to give the mom-and-pop type sites a chance, I can’t believe they would require them to do all this 301 redirect stuff between www and non-www sites. I find it really frustrating to think I would have to do this. I am not that technical and would have a hard time figuring out how to do this. Everyone knows that doggiewoggie.com and http://www.doggiewoggie.com are the same. Shouldn’t Google know this too? I should not be penalized because I don’t know how to do a 301 redirect!

  32. I got pointed to this site, because I had a lot to this topic. First if you have been a victim to google 302 problem and maybe had some hijackers try to take your site, then you will have troubles for years, because all those 302 links which is cached on google are still online even if they are gone on the real site, but the caches can be from early 2004 and they are still in the index, so you are then filtered for dublicated content.

    Also when you then find out you also have a non www issue, then you make a 301 which is the right thing to do, but the problem is you are filtered on google so your redirecting wont take effect, I got a redirect active for 4 month now, even have links from another site to those pages that are listed with non www on google, but it dossent help. I also created a new site, did a 301 it only took 3-4 weeks then it was active on google, but still nothing on the site that is filtered.

    I think as long they dont update there supplemental results DB, we wont see any change, it just a theory.

    Its hard to see a site with 32.000 unique visits a day from google only and has been online in 4years never had any troubles with any update, go down because of the 302 bug and then all other just follows.

    Zeus

  33. That’s the best suggestion I’ve heard in quite a while Shri.

    With the authorization that is needed to do sitemaps, it should be enough credibility to accept that the 301 is legitimate and greatly reduce the time involved.

    I think there is a “301 fear” right now due to the time involved with them taking hold. Definitely right Matt that “301 time” will become an issue with webmasters next. Might as well head us off at the pass;)

  34. grnidone

    May I also suggest you roast a red bell pepper to stuff the polenta into. Add some fresh ground parmesean cheese on top and it is nothing short of delightful.

  35. nutkeis

    Pat raises an excellent point. Why in this age of nothing-but-web is Google treating www. the same as any other subdomain? The www may have outlived its usefulness, but it is pretty well entrenched in the minds of most users as being synonymous with the root domain.

    So why? Poll a thousand webmasters and you probably won’t find one who cares if www. is deemed canonical across the board in Google.

  36. HUD

    There are loads of legitimate websites being caught out by this 301 bug. Why not just accept that there is a problem and offer to manually check redirects for an admin fee? A kind of Google “domain moving” service.

  37. Zeus

    Im not sure what is wrong, but I do get more and more emails, about whats happening to there site for no reason, from big sites to small family sites, most of them I have to explain that google has troubles, with non www, googlebug 302, 301 redirects, to have dublicated filter, sooner omitted results, more supplemental results and wired results in the serps.

    Many have no clue what to do, I tell them they can only sit and wait, thats the best, because as you can see no other SE has those troubles and there is nothing wrong with our site.

    zeus

  38. ModemMike

    One of my least favorite accidents is when someone reports a 301 or 302 problem.

    Yes, I’m going there again :-(

    Has this topic/bug/feature been brought to the front burner yet and do we 301 victims/whiners have any hope for the next update?

    Yes, I’m being a little self centered here but there are many others like me that have lost hope and I’m reduced to soliciting comments so flame away but for those that are feeling the desperation I am any feedback from Mr. Cutts would be greatly appreciated.

    Matt, email me, I’ll give you 2 great examples.

  39. Stephen

    Hi Matt

    Have you seen this thread at threadwatch:-

    http://www.threadwatch.org/node/4103

    It is an intresting situation that has happened to Mirago – as lots of other webmasters seem to experience the exact same problem.

    Cant add much more to that has already been posted on that thread.

    I guess this is what you would call a canonical url problem – not sure.

    Thanks

    Stephen

  40. Stephen

    OK – I am going to shut up for a while and give you a chance to look into it.

    I hope this is good feedback rather than a waste of time though :)

  41. Hijacked

    Do not post this if it exposes a problem that unscrupulous people can take advantage of to penalize competitors or hijack sites.
    —-
    Is there such a thing as hijacking by mirror domains, or is the below a duplicate content penalty?

    A competitor pointed an old, unused domain to my IP address, creating a mirror of my content at their address. It was apparently like this for a while, until I found it and sent them a cease and desist. They took down this mirror by adjusting the DNS, and their domain no longer resolved to anything.

    I thought things were resolved, but then a couple weeks later Google updated, my rankings plummeted and referrals dropped to almost nothing. It’s now a couple weeks later, and things have not changed.

    I discovered that Google had indexed many pages of my site on this other domain, and apparently considered it the canonical domain. Perhaps this was because their domain was older, even though my domain has many links to it, and their domain only has a few. Their domain doesn’t rank now, probably because at subsequent spidering there was no content or site at all that domain. (I believe they would be ranking if the site was still up at their domain.)

    If you search for the backlinks to their domain, you get sites that actually link to my domain. If you do a unique search, theirs comes up first (as a supplemental listing) and you have to filter=0 to get mine. If you search for terms that I previously ranked highly on, neither domain is anywhere to be found, except if it is for one of my pages that is not indexed for their domain. In this case, pages from my domain often show up in the serps, just not nearly as high as before.

    Is it really as simple as this to knock competitors out?

    Is this a case of PR / link hijacking, or is it more likely a duplicate content penalty? Either way, will this automatically be fixed at the next PR update, since they have removed the mirror? Will my domain fully recover all links / PR? How long will it take? (Some people talk about these supplemental listings lasting for years.)

    Is it a problem that their domain is now pointing nowhere? Would it help if their domain did a 301 redirect to mine? (It may take months and lawyers to get them to do that.) Would a DMCA request to Google expedite a return?

    Thanks!

  42. M

    Hi Matt,

    Regardless of our having 301’d http://www.oursite.com/?referrer=google to http://www.oursite.com/ on September 24, 2005, and several crawls by Google since then, our 301 has yet to be picked-up.

    We’ve followed the steps you outlined regarding contacting Google at http://www.google.com/support/bin/request.py to no avail and no response.

    And now we’re wondering if the 301 on the following URL: http://google.com/url?sa=l&q=http://www.oursite.com?

    Server Response: http://google.com/url?sa=l&q=http://www.oursite.com?
    Status: HTTP/1.1 301 Moved Permanently
    Location: http://www.google.com/url?sa=l&q=http://www.oursite.com?
    Set-Cookie: PREF=ID=3a2384ef78ba77f6:TM=1128887291:LM=1128887291:S=11qPpCAtDgje3Z0e; expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.com Content-Type: text/html Server: GWS/2.1
    Content-Length: 206 Date: Sun, 09 Oct 2005 19:48:11 GMT

    Might be what’s causing the problem. Although we can’t get http://google.com/url?sa=l&q=http://www.oursite.com? to appear in the Google serps, they are viewable in Yahoo’s serps (sorry for the reference here :)

    I also apologize for not including the site name/Yahoo page as I’m none too certain if that’s allowed here and/or where this information appears.

    Any guidance you could provide would be greatly appreciated.

    Thanks so much for your time!
    M

  43. Stephen

    Hi Matt

    Me again. Sorry to be like a Bull in a China Shop – but have found some more ways to demonstrate the problems that canonical urls cause (assuming that this is the correct name for the problem). I will call it Canonical URLs in the post for point of demonstration.

    I will use the Mirago example as a site with Canonical url problems (as it has been mentioned already) and Amazon as a site without the problems.

    OK – You may have seen some of my posts at Webmasterworld where I suggest doing a “www.domain.com” search (as a phrase) to see if your site has problems – if you dont end up top then that rings alarm bells for me.

    Now I have just had a site (minor site) which is now top for that search (on most dcs) – and the rankings are better and Gbot is intrested in visiting again. Although some dcs have gone back the other way over the last few days :(.

    However, it is not just restricted to the homepage as per the following examples.

    A search for a url at Mirago (as a phrase):-

    http://www.google.com/search?num=100&hs=6xJ&hl=en&lr=&safe=off&c2coff=1&client=opera&rls=en&q=%22www.mirago.co.uk%2Fsearchbar%2F1293%2Fsearchbar.asp%22&btnG=Search

    This shows that even though that page is not supplemental it is ranked below supplemental pages that mention/link to the url – hmmmz – surely the best result for the that search would be the actual url – but this does show what is happening more (might not prove anything to you – but if it helps problem fix then great)

    More examples:-

    http://www.google.com/search?num=100&hl=en&hs=4Pz&lr=&c2coff=1&safe=off&client=opera&rls=en&q=%22www.mirago.co.uk/scripts/advanced.aspx%22

    http://www.google.com/search?q=%22www.mirago.co.uk/regions/default.asp%22&num=100&hl=en&hs=T6J&lr=&c2coff=1&safe=off&client=opera&rls=en&start=100&sa=N

    Well you get the idea – this probably is why so many webmasters are coming out with “this site out ranks my site for my homepage name, my content etc.”

    Once you get a canonical url problem it is like the worst of the worst penalaties :(

    OK – lets look at Amazon:-

    http://www.google.com/search?num=100&hl=en&hs=moe&lr=&c2coff=1&safe=off&client=opera&rls=en&sa=G&q=%22www.amazon.com%22

    A search for “www.amazon.com” shows the correct url in number 1 postion (in this case not Amazon.com – but the canonical page) – so therefore correct.

    http://www.google.com/search?num=100&hl=en&hs=aBK&lr=&c2coff=1&safe=off&client=opera&rls=en&q=%22www.amazon.com/exec/obidos/tg/browse/-/868706%22

    The Amazon page at number 1 for the search on the url – with link pages etc below.

    http://www.google.com/search?num=100&hl=en&hs=ZCK&lr=&c2coff=1&safe=off&client=opera&rls=en&q=%22www.amazon.com/exec/obidos/tg/browse/-/15015721%22

    and again etc etc

    Point being that a site that has Canonical url problems can not even rank for a search on its own urls – if it can not be top for its own urls then it probably is going to struggle being ranked for its content etc.

    I hope this is helpful in a pointer – there is a bit of movement on the DC at the moment so dont know if you are working on.

    OK – not sure what triggers this behaviour (penalty, indexing of non-www, supplementals) but I have done the 301 from non-www to www in March and still little improvement for my main site that I cam concerned about.

    You have my email if you want more examples. (I assume that you can access the emails registered here)

    Best

    Stephen.

  44. Blazes

    Hello!

    Thank you for the information 301 vs. 302. What about this scenario? Could you address this for other readers too because many sites are in this situation. My site: domain.com and http://www.domain.com both have 200 OK. My Google sitemap is “domain.com”. Google has indexed both versions. Is this Okay or should I have the 301 redirect done?

  45. I think one of my clients is getting 302 hijacked by business.com! They recently purchased a listing in the directory and business.com links to the site with a 302 redirect. Their listings in Google disappeared completely and business.com is now in the place they used to be!
    I submitted a complaint to Google and business.com.

  46. Hi from germany,

    my site was hijacked 10 days ago . Somebody placed a 302 redirect on his site – now my homepage is gone ! I thought this issue was fixed – perhaps Matt can explain if the problem is still present.

  47. Karl

    301s on some dynamically generated pages.

    This is a little technical….

    We have a primary site used to deal with property in the UK and Ireland. New functionality was added to the site to use Postcodes – that mean’t we had to remove our Irish customers to another site since they don’t have Postcodes like they do in the UK.

    I’ve put a 301 redirect on my page /propertysearch.asp?town=Dublin to the new site.

    However the vast bulk of the original site remains e.g. propertysearch.asp?town=London or ?town=Edinburgh.

    Does this selective 301 on some pages trigger some SEO-blackhat or indexing/ranking issues?

    Regards

    Karl

  48. Marc

    Hi,

    which websites concern the 301/302 problem?
    Only Websites with a Page Rank 7 and higher or everyone?

    Thanks and best regards

    Marc

  49. Marc, you should use 301 to redirect your websites and avoid 302 since some people use it to steal PR from other websites.

  50. Ralph

    Matt –

    What is the best way to make a permanent move from http://www.domain.com to new http://www.domain2.com (say a company does rebranding) without loosing SERPS in google? Yahoo and MSN seems to have resolved this but clearly many people are still having a problem with it in google..

  51. Dave

    Hey Matt, Since I am not too experienced at all this I always assumed it was by far best to strive getting both the www and non-www indexed as I believed it would have a double chance of being found that way in the SE and likely get more traffic too due to it being indexed twice, is that possible?. I also went the extra mile many times by getting one way links to both versions rather than concentrate on only one.

    From reading your excellent blog it seems many here say only get one indexed but I still wonder why it makes much difference? Sometimes when I check PR some of my sites have the same PR with and without and sometimes one may be a PR5 and the other perhaps a PR3 or 4. Why is that anyway assuning the content is the same and links are about equal? Just wondering. Please excuse my lack of knowledge on this.

  52. Hi Matt

    Forgive me if I’m way off topic here is but this is something totally new to me. Last night my domain returned a Cannot Find Server error. I assumed it was a “router” problem and went to bed.

    This morning things seemed back to normal until one of my pages on loading displayed a rather juicy hardcore porn teaser page. I experimented a bit (not with the images LOL) and discovered sometimes it would load right, but on refreshing the “other” page would come up, and sometimes when I clicked away from my site the porn page sustituted when I clicked back to my own page.

    So I typed “subdomain hijacking” into Google and came up with your blog ( amongst other things). I have to report the situation seems to have returned to normal now – thankfully. How is this done and is there anything you can do to protect your domains from it?

    Many thanks

    Patricia

  53. The proper procedure (assuming that you want http://www.domain.com to show up) is to make domain.com do a permanent (301) redirect to http://www.domain.com.

    How can i do that?

  54. Again my site was hijacked …. If customers search for my name (Honscha) they were sent to that page :
    http://69.41.173.145/ww/www.honscha.de/ instead of http://www.honscha.de . Any advise from you matt ?

  55. Actually, 301 redirects are also being used for fairly insidious purposes. That is directly because of monetization of Google Page Rank, and how people are selling links, domains, websites based just on page rank. A lot of people are doing 301 redirects and using the page rank of the redirectee site to sell links. There is really no good solution, unless Google wants to send bot from a previously unknown IP address, or change the pattern of its bot from googlebot to something else. The other one is obviously a nifty little tool, that one can use to check the PR. http://www.sem-faq.com/fake-pagerank-checker.php is an example of such a free tool…

  56. It would appear that I have inadvertantly ‘faked’ pagerank from one of my sites to another. An added feature of what I have done is that the www version of the site has the fake PR and the non-www does not.

    I have found information all over the place talking about creating fake pagerank, but other than when content is duplicated; there’s no discussion on fixing a mess similar to mine.

    I temporarily parked a new domain, not realizing that would start this mess. I changed the new domain to an “add-on”, but it seems to be forever tied to my primary (where the pagerank is being stolen from). There are no links between the two sites and there is no common content.

    A search in Google for the new domain name brings up the primary.

    A third site, also hosted on the same account as an add-on domain has essentually fallen out of the serps for keywords it had been doing quite well for. This may be only a coincidence, but that seems unlikely.

    I kept telling my webhost there was something wrong, they kept telling me to be patient; things would iron out as the new site was spidered. In the meantime, the last pagerank update transpired and has just made things worse.

    What, if anything can I do to fix this mess? or Can Google do something? If anyone has any suggestions they would be greatly appreciated.

  57. Hi,
    You can set your preferred doamin in google sitemap or do it by 301 redirection. for example I’ve redirected my domain, no-filter.com to http://www.no-filter.com . That means if you type no-filter.com your browser address bar you will end up with http://www.no-filter.com

  58. seo

    I’ve got many things from this article.I was just thiunking about an operational issue that sprung to mind since if this fell through the cracks, it would be a very subtle failure as basically everything would continue to work except for those folks actually doing this test.

  59. Chef Apparel

    All about Chef Apparel. To dispel the notion that a chef apron is similar to that of a housewife’s apron, you will find that these are considered an integral part of the chef apparel where their utilitarian look protects the wearer from the various stains and spills that can sometimes happen….
    Read on at: http://www.cooking-food-recipe.com/Chef-Apparel.php

  60. Great information for Chef Hat. It is possible that the earliest known chefs wore a chef hat not only to keep their hair out of the food but also to indicate their position in the kitchen. This was supposedly done by the number of pleats that could be found on the chef hat….

    Here is the rest of the post: http://www.cooking-food-recipe.com/Chef-Hat.php

  61. hi, how about enhancing the SiteMap protocol to support redirection and removal? I

  62. Great title, It’s so easy to find this kind of issue =P My site has been proxy hijacked, there is no doubt… =( I’ve sent all the reports to google but I don’t see any change… Why Google doesn’t fix this problem? I don’t understand you (I mean, Google)… Lots of sites are stealing my ranking in google.

  63. Mike

    Matt,

    I know this is an old topic, so I’m not sure when or if you’ll have a chance to read this post.

    I have discovered some major problems in Google with my site being switched out for others. Not willing to disclose much about it in a public blog/forum. The number of visitors coming to my site has not been this low since the 90′s.

    Unfortunately, what you wrote below doesn’t seem to be an available option for help with this issue any longer. Where is the preferred place to send information now, so that the right team sees it and can dissect what is taking place with the issue?

    “Go to http://www.google.com/support/bin/request.py and click “I’m a webmaster inquiring about my website” then select “Why my site disappeared from the search results or dropped in ranking” and click continue. In the webform that you get to, make sure you put “canonicalpage” in the Subject line, then put the details in the Message body. Someone will route that message to an engineering mailing list where we dissect claims of canonicalization problems (that is, picking the wrong url).”

    Thanks for your time,

    Mike

  64. I have had the same issues on previous sites using the 301 redirect, but now I am not having that problem anymore? Seems like there is more than meets the eye and luck can be a factor because this was only an issue of the past and I am no longer having it.

Leave a Comment

Your email address will not be published. Required fields are marked *

*

If you have a question about your site specifically or a general question about search, your best bet is to post in our Webmaster Help Forum linked from http://google.com/webmasters

If you comment, please use your personal name, not your business name. Business names can sound salesy or spammy, and I would like to try people leaving their actual name instead.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

css.php