SEO advice: url canonicalization

(I got my power back!)

Before I start collecting feedback on the Bigdaddy data center, I want to talk a little bit about canonicalization, www vs. non-www, redirects, duplicate urls, 302 “hijacking,” etc. so that we’re all on the same page.

Q: What is a canonical url? Do you have to use such a weird word, anyway?
A: Sorry that it’s a strange word; that’s what we call it around Google. Canonicalization is the process of picking the best url when there are several choices, and it usually refers to home pages. For example, most people would consider these the same urls:


But technically all of these urls are different. A web server could return completely different content for all the urls above. When Google “canonicalizes” a url, we try to pick the url that seems like the best representative from that set.

Q: So how do I make sure that Google picks the url that I want?
A: One thing that helps is to pick the url that you want and use that url consistently across your entire site. For example, don’t make half of your links go to and the other half go to . Instead, pick the url you prefer and always use that format for your internal links.

Q: Is there anything else I can do?
A: Yes. Suppose you want your default url to be . You can make your webserver so that if someone requests, it does a 301 (permanent) redirect to . That helps Google know which url you prefer to be canonical. Adding a 301 redirect can be an especially good idea if your site changes often (e.g. dynamic content, a blog, etc.).

Q: If I want to get rid of but keep, should I use the url removal tool to remove
A: No, definitely don’t do this. If you remove one of the www vs. non-www hostnames, it can end up removing your whole domain for six months. Definitely don’t do this. If you did use the url removal tool to remove your entire domain when you actually only wanted to remove the www or non-www version of your domain, do a reinclusion request and mention that you removed your entire domain by accident using the url removal tool and that you’d like it reincluded.

Q: I noticed that you don’t do a 301 redirect on your site from the non-www to the www version, Matt. Why not? Are you stupid in the head?
A: Actually, it’s on purpose. I noticed that several months ago but decided not to change it on my end or ask anyone at Google to fix it. I may add a 301 eventually, but for now it’s a helpful test case.

Q: So when you say www vs. non-www, you’re talking about a type of canonicalization. Are there other ways that urls get canonicalized?
A: Yes, there can be a lot, but most people never notice (or need to notice) them. Search engines can do things like keeping or removing trailing slashes, trying to convert urls with upper case to lower case, or removing session IDs from bulletin board or other software (many bulletin board software packages will work fine if you omit the session ID).

Q: Let’s talk about the inurl: operator. Why does everyone think that if shows results that aren’t from, it must be hijacked?
A: Many months ago, if you saw, that would sometimes have content from mydomain. That could happen when the url was a 302 redirect to and we decided to show a result from Since then, we’ve changed our heuristics to make showing the source url for 302 redirects much more rare. We are moving to a framework for handling redirects in which we will almost always show the destination url. Yahoo handles 302 redirects by usually showing the destination url, and we are in the middle of transitioning to a similar set of heuristics. Note that Yahoo reserves the right to have exceptions on redirect handling, and Google does too. Based on our analysis, we will show the source url for a 302 redirect less than half a percent of the time (basically, when we have strong reason to think the source url is correct).

Q: Okay, how about supplemental results. Do supplemental results cause a penalty in Google?
A: Nope.

Q: I have some pages in the supplemental results that are old now. What should I do?
A: I wouldn’t spend much effort on them. If the pages have moved, I would make sure that there’s a 301 redirect to the new location of pages. If the pages are truly gone, I’d make sure that you serve a 404 on those pages. After that, I wouldn’t put any more effort in. When Google eventually recrawls those pages, it will pick up the changes, but because it can take longer for us to crawl supplemental results, you might not see that update for a while.

That’s about all I can think of for now. I’ll try to talk about some examples of 302’s and inurl: soon, to help make some of this more concrete.

329 Responses to SEO advice: url canonicalization (Leave a comment)

  1. Hi Matt

    Very informative educating post!

    Power to you, Matt ๐Ÿ™‚

    So when do expect the call for BigDaddy? a week from now?

  2. Hello Matt,

    Thanks for all the info. It is great to finally get this info from a ‘real’ source.

    My question is, however, that would it be ok for a page that no longer exists to be 301 redirected to the home page, or site map? Or would this create problems?

    Thanks, and love the blog.


  3. Hi Matt,

    Great article. I’ve been doing the 301 redirects to the full URL for a while now. If nothing else, it gives the site a much more professional feel. All the major sites do it and I don’t get why it’s not a standard on server configurations. Most seem to default to the multiple URL versions.

    Happy new year.

  4. Thanks for the information! Subscribing to this blog has given me alot of information and I haven’t subscribed long ๐Ÿ˜‰

  5. Hi Matt,

    Quick clarification… quote..

    “Suppose you want your default url to be . You can make your webserver so that if someone requests, it does a 301 (permanent) redirect to

    I noticed that when you say to do a 301 redirect you didn’t include a trailing slash. Is there a distinct difference between and (with the trailing slash) ?

    Would it be worthwhile to make sure that any links within our site that point to the homepage be consistant using or not using the slash?


  6. “So how do I make sure that Google picks the url that I want?”

    Ok, so basically we tell the crawler which url we want to use by internally linking it consistently throughout the site? Would doing the same thing for external inbound links also help? Seems like it’s still a little out of our control. Can’t this be set in the sitemap? That would seem logical to me.

  7. that was some good info for the main webmaster, but most of us who are introuble because google could not handle 302 links and now we see on the test server, that the is back, caches from old 302 links are gone and many pages has been spidered by the mozilla bot, can we expect a ranking again when the real update starts.

    Thanks for your time

  8. Matt,
    Would you please give us some reasons which cause the URL-only listing? What should we do to correct this listing?


  9. Hi Matt,

    Nice post, I am especially happy that you mention the supplemental results. What exactly are they? I can’t understand the purpose of them or how they benefit the searcher. Theyยดre mostly a pain in the butt because they push other, often more valuable results, down.



  10. Are we to wait so much for the next ‘BigDaddy’ update?

  11. Great post, Matt!

    As a programmer who has done a lot of text parsing, I’ve been lobbying SEOs and Webmasters for years to keep their URL formats consistent. I think that, coming from you, the message will be heard more loudly and clearly than before.

    This is the kind of stuff that makes your site so worthwhile to me. I know, you’re Inigo Montoya, and spammers all have six fingers, but this straight-forward, tell-it-like-it-is content is what I enjoy most.

  12. Hi Matt & Everyone,
    Just yesterday I’ve set my 301 redirect, but my question is about PR and links.
    If you have links that suppose to bring you some PR to let’s say: instead of would the links PR effect the as well?

    If anyone knows the answer, please let me know. (I have that problem with an old site of mine)

  13. While I normally I don’t comment, I think this is a great post that really gives the low-down on what webmasters should do. Anyone can speculate all they want, but it is great to hear this from a reputable source, and with all of the information all together.

  14. ..and he even puts to rest the definition of ‘Canonicalization’. Great information, Matt!

  15. Hi Matt,

    Nice explanation. Could you go on further with some case like mine:

    I have several sites. Each of the sites is hosting a different language version of the same site (you will find my main site from my e-mail address and click of any of the links of this placeholder site).
    Site 1 is my main site
    Site 2 is the site in another language.

    For some technical reason I have not yet been able to host the site 2 at the same place than site 1( and I do not think of changing the host of site 1 since they have worked so well with us in the past). some of the php pages are however sharing the same database so that I asked the host of site 2 to 302 redirect to site 1( thus visitors are seeing URL of site 2, yet the pages are taken from a subdomain of site 1 because I want the visitors to keep site 2 in mind as I will remove the redirect as soon as I can).

    The funny thing is that because of the redirect, Google is not able to define the root of site 2. The site: command for site 2 is only showing the index page.
    All pages of site 2 are as URL only under the site: command of site 1 although site 2 has its independant directory.

    Your test DC shows URL only for the index page of site 2, fully indexed pages for site 2 UNDER the site: command of site 1.

    Does it means I will only have the choice of either loosing my index or all of the other pages? Isn’t there a way for google to determine that these subdirectories are independant from each other, just hosted together?

  16. Hi Matt,

    This may be your best post ever!

    I only have one stupid question: Is it better to spell out “” everywhere within a site (and in googlemap.xml), or to just leave it blank (as in “”)?

    I can see that it makes a huge difference in the Page Rank of a โ€œdefault.aspโ€ page, so I am just curious…

    Thank you for this very informative post!

    Brian M

  17. As said Matt GREAT post. It is nice to hear this information from someone at the source. As far as your non-www issue I think the “contest” and the words of your friends are going to help ya there ๐Ÿ˜›

  18. Smoke2Much, that’s a good suggestion. It never hurts to ask external people to link to your site in your preferred manner as well.

    Anh, normally if you don’t see crawled snippets and only see urls, it usually means that it would help to have more links to your site. More PageRank helps to get more crawling. The other common explanation is that you’ve got a robots.txt which forbids crawling a url. We can see the references to the url, but the robots.txt doesn’t allow us to crawl the url, so we can only show the uncrawled url reference.

    Brian M, I would opt for personally instead of It’s easier for users to remember and to type in, so people linking naturally are more likely to link to the page without the index.asp.

  19. Tony Hill, great question about vs. . This is one of those cases where I’d pick a preferred format and stick with it uniformly. I would lean toward having the trailing slash, because that’s what most people expect. Maybe I’ll just duck in and update that in the post.. ๐Ÿ™‚

  20. Thanks for the post Matt. It will be very helpful if you have such Matt-Sessions once a month ๐Ÿ™‚

    About Canonicalization – What about https and http versions? I have a site is indexed for https, in place of http. I am sure this too is a form of canonical URIs and how do you suggest we go about it?

    Thanks Matt, and wish you happy n prosperous Y2K6 ๐Ÿ™‚

  21. Hi Matt
    great post, many thanks. One of my older sites has a mix of www and non www internal links.
    The site is OK but I would like to change to all www but dare not lest duplicate content problems etc. I am on a windows server with no ht access.
    Would you advise changing the non www or leave well alone?
    Thanks in advance

  22. > It never hurts to ask external people to link to your site in your preferred manner as well.

    What if external pepple are linking in your non-preferred manner? On purpose? A lot?

    What safeguards exist for small sites?

  23. A few weeks ago one of our clients bought a new domain and wanted the traffic from the old domain to point to it. We set up a 301 redirect and the first domain completely dropped off the face of the earth in the search rankings within 24 hours.. (the first domain had a good page rank) – will the page rank for the old domain be redirected to the new one (it has basically the same content) in due time or will the SEO have to start again from scratch?

    I’m sure I’m not the only person who has experienced this, so it would be useful to find out more about it!

  24. Matt,

    Great and extremely informative write up once again. I guess in fact all of your posts about SEO will be informative to us because you know so much inside info.

    You’re doing a great service by showing webmasters how to “properly” SEO their sites and in doing so you likely contribut to the reduction of SE spam and mak it a better place for everyone.

    I too am wondering about the https:// thing by the way.

  25. McMohan, Google can crawl https just fine, but I might lean toward doing a 301 redirect to the http version (assuming that e.g. the browser doesn’t support cookies, which Googlebot doesn’t).

  26. Supplemental Challenged

    “When Google eventually recrawls those pages, it will pick up the changes”

    This is still false, and it is hard to understand why you keep saying it Matt.

    Google doesnot obey 301s or 404s involving suplemental listings, even if those 301s and 404s are seen every single day.

    Apparently this is still the principal reality disconnect at Google. Printing this false information is not helpful. (The other stuff is very usefule though.)

  27. Hey Matt,

    In all of the discussions on removing old pages from the supplementals everybody talks about serving a 404. Even you say, “If the pages are truly gone, Iโ€™d make sure that you serve a 404 on those pages.”

    This always confuses me. Why not serve a 410 so there can’t be any ambiguity as to whether the page is just ‘not found’ at the moment, or the webmaster is actually telling the bot, ‘Hey, this is gone, don’t look for it any longer’?

    Do some bots not handle a 410 correctly? Or am I (as usual) missing something simple stupid here?

  28. Very nice knowledgebase Matt. Thank you.

  29. Supplemental Challenged, a better way to say it is “When Supplemental Googlebot recrawls those pages…” I believe, it’s true that normal Googlebot crawling won’t affect supplemental results. Supplemental Googlebot can take (sometimes much) longer to get out and recrawl pages.

    jimbeetle, I’ll have to check to make sure that 410 (permanently gone) is handled correctly.

  30. You rock Matt!

    You answered this question perfectly for what I needed, I set 301 redirects just this week and was wondering if I had done the right thing. Looks like I have.

    Thank you

  31. Matt,

    Is there something to do on cases when you’re getting backlinks to versus ?

    Does it have the same bad effect as versus
    and if so, can I use an apache 301 redirect the same way for solving
    that problem?

  32. I’m also very interested in the ‘410 Gone’ response and how Google handles that one. Normally it takes quite some time for Google to ‘forget’ about pages returning a 404. Perhaps with a 410 there’s no question about the gone-ness ๐Ÿ™‚

  33. Matt

    You say “” and “” are different.

    What would be the HTTP GET request for “” and how would it differ from the HTTP GET request for “”?

    I have

    GET / HTTP/1.1

    for both.

    I agree that and are different, but I don’t think this applies at the root level of the domain.

  34. This is all a little confusing actually.

    I changed my urls in my blog to be index.html, keyword1.html, keyword2.html and so on, BUT in blog format the categories end in http://www.domain/category/. Is this a bad thing?

    If you want to take a look click my URL in my blog. Anyone?

  35. Very useful, I’m just in the market for a 301 because ‘someone’ decided to cache two versions of my homepage! No names….

    Surprisingly, my host told me they don’t support 301 redirects! So now I have to switch hosts too!

    My question however is related to some restructuring of my site, tidying up and putting files in folders etc.

    If I do not or am not able to implement 301’s for all of the moved pages, will these changes have an adverse effect? Will duplicate content flags start waving?

    Will Google eventually drop the old URL’s and only pay attention to the new ones and what will happen in the meantime?

    I am adopting 301’s as good practice from this moment forth but without them will sites suffer….?

  36. >> What if external pepple are linking in your non-preferred manner? On purpose? A lot?

  37. >> What if external people are linking in your non-preferred manner? On purpose? A lot?

    Adding the “base” tag to all pages of the site, and especially the index page, can help a lot. If you use absolute linking within your site (like “”), or use relative links which count from the root (i.e. begin with a slash, like “/folder/folder/page.html”) then the URL in the “base” tag only needs to be “” as in, the root domain.

    If you use relative links (like “folder/page.html” {NO leading “/” on URL} or like “../../../folde/page.html”) then the URL in the “base” tag MUST be the full URL of the page that it is on. The “base” tag can help canonicalisation even before you get a 301 redirect from non-www to www in place.

    Oh, and when I say “links” in this post, I mean URLs in clickable links, as well as paths to images, and any external CSS or javascript fies too.

  38. Thank you Matt.

    Great post. There aren’t many blogs where you feels smarter after you leave them… ๐Ÿ™‚

    For some reason, my mind had put canonicalization into the cache date bin, but never researched much further… kinda like when you think a word means one thing until you start using it in public. You quickly realize it has the opposite meaning. Ever done that?

    Thank you for answering the question.

  39. In researching this to understand the necessity, I discovered my main competitor has both the www and non-www version of his homepage cached by Google. (Non-www version, PR0. www version PR4)

    He’s still ranking No.1 for a very popular generic search. As are many of his internal pages for other permutations.

    So I throw this into the melting pot and say – How big an issue is this…??

    I understand as a matter of cleanliness it’s good practice but as for it’s effect on rankings I’m now sceptical….

    I would not necessarily attribute a loss in rankings to this. There are other factors in play. This is not ‘the answer’. Just one of them…..

  40. Some of the posts seem to be mixing apples and oranges.

    Please clarify whether it is important to standardize the canonization of links externally pointing to a site’s home page to links internally pointing to a site’s home page. Example below.

    Matt, you obviously are talking about external links to a site when you replied “Brian M, I would opt for personally instead of ….”

    But what about internal links? Is it OK to use “index.asp” or “/index.asp” (and not just “/”) for relative addressing?

    For example, I prefer to use relative addressing for the ease of maintenance and testing locally. Because I use Dreamweaver, I don’t have the option to use “/” for the home page links. I must use either “/index.asp” or “index.asp”. Neither of these match the external link to the home page in the format

    I need help in understanding if it is OK to use a different format for internal links vs external links for the home page. If the difference is minimal, then to me there is a much bigger benefit derived by still being able to use a tool that lets you test locally.

    I agree with Tim in questioning the true importance of all this in light of the big picture.

  41. Hi Matt, I think it’s great that the folks like yourself at Google take the time to work with the webmaster community.

    My question is, does Googlebot consider the HTML “base” element in web pages as part of the canonicalization process ?

  42. Cool, clear a doubt that had been troubling me for quite sometime! Thanks

  43. Hi Matt,

    Thanks for all this, u looks dude in blue shirt :). Well Google Sitemap can help us in the caching of supplement pages.



  44. So, I’ve been thinking about what sites can do to provide canonical-URL hinting to search engines. I’m wondering if perhaps a might be an option. Obviously, since I just pulled it out of thin air, it wouldn’t do any good unless search engines began to look for it. I’ve written up more of my idea elsewhere to keep this comment short. What do you think?

  45. Dear Matt,

    I do not understand the problem.

    Some providers doesn’t support 301 redirects and so many people have no possibility to canonicalize (strange word ๐Ÿ™‚ their url’s.

    I think I can’t be so difficult for a search engine to find out that the content is exact the same with and without www and so the url’s are identical.

    It’s time to fix the problem.

  46. Matt – Scott posted the following question to you on 4 January, so far without reply:

    “A few weeks ago one of our clients bought a new domain and wanted the traffic from the old domain to point to it. We set up a 301 redirect and the first domain completely dropped off the face of the earth in the search rankings within 24 hours.. (the first domain had a good page rank) – will the page rank for the old domain be redirected to the new one (it has basically the same content) in due time or will the SEO have to start again from scratch?”

    We’re about to proceed down exactly the same path (i.e. changing domain as a result of a rebranding) and we’re keen to minimise the impact on our hard earned page ranking.

    Any words of wisdom that you can pass on?

  47. Mary Johnson, why not use a local webserver? Apache, or if you have trouble setting it up, Xitami, for example? Both are free and really easy to use.

  48. Matt, you really should do something about this security code thing. It doesn’t work without JS. And you exclude blind people as well…

  49. Hi Matt,

    I found an example of a canonical URL issue.

    Two different PageRanks, same site.

    But in this case, I just redesigned this site and relaunched it Tuesday so that might have something to do with it (although I doubt it, I throw it out there as a possibility).

  50. Matt:

    When we see two results from the same site (with the second one indented), isn’t that a c14n issue? online poster printing,, yields two results from the same site where the second one it the homepage.

    Shouldn’t we only see one result from each site or at least see the homepage suppressed in favor of the inside page which is presumably more exact/granular a match for the search.


  51. Thanks for these tips. I was thinking the same before but now you confirmed my thoughts and I am sure it these are facts.

  52. dear Matt thanx for this wonderful information

    but can any one help my website was coming in good position in google but all of sudden it disappear from google

    my website url is

    can any body help me what to do


  53. Hi Matt

    Wonderful post. So the spiders are now a pair that work in tandem??

    Would one be considered a quality control check of the sites links??

    Trying to clear up other peoples mistaken ideas and thoughts.

    Thank you.

  54. Hi Matt!

    nice 2 hear from googles intentions, but here more questions than anwers.

  55. HI Matt.

    Always value your insight.
    I have ranking PR5 with more in links and ranking a PR4.

    I would prefer a www. If I do a redirect will the PR follow as well?

    Hope you can help clear this for me before I make the move.

    Thanks again

  56. Thanks MATT, posts like these are helpful to business owners like me who tend to optimise their site on their own and avoid SEO companies. I am a learner in SEO, but am picking up.

    Matt is right about and
    Example site I have noticed is: and

    One question that may prove a petty thing for you, but its burning in my mind. Suppose I link my internal pages like “pages.html” instead of complete url “” – will this effect with the result?

    The other thing is that linking requests come to my site and I never include a trailing slash and give them my URL as and not . If someone links to my site with and the other links as – will the two linkings be considered as linking to two different domains – just an info update that is not needed – its not important.

  57. maybe helpful or..

    think about words as circles in the theory of sets, and the momentary semantics of a word
    is the spot of the deposit

    lamp lamppa lampe lampada luminaire all the same circle and nearly same spot

    light Leuchte light fixture
    are next circle

  58. Very informative !!!! I am impressed….i am expecting a detailed article about big dadduy update ๐Ÿ™‚

  59. And words have axes of covering

    For example look at

    This looks fine to me

    But all other may laugh orโ€ฆ

    Because maรŸe
    Means dimensions or gages

    And this word/formula has an axis to the object of ALL
    By me it is to lamp

    In the search the object should be to Vesuv / Berg / mountain are next axis

  60. Another sorry but german is โ€ฆ

    and so the serp is a mix of errors

    There are MaรŸe first more a spoken – dimensions

    and Masse scharper s and more end e means – bulk or compound

    and say if it is enough!

  61. A SearchEngine buisness is to recognize the pattern which the user is looking after and
    to give him reasonable terms.

    Therefore it must know about
    1. language specifics (see grammar)
    2. psychological affectations of humans ( in their cultures)
    3. your stats of searches (are the expression of 1+2)

    I can say you something about point 1

    To build up some little filters of context, you had to bring
    the Gramma of the language to some short rules.

    Look at the system that works in You, from the eye to your consciousness.

    There are some arts of sensors in your eye.
    There are some filters before the signals comes into the brain, which
    works with the combined signals from the different sensors.

    So are you able to see first, what the first capital of importance for a biped is (motion).
    There are filers for edges; this is the first step of your search engine to build you the room you live in.

    Really in your mind comes only about 15% of all. If there were no compression, you could see less pictures per time and donโ€™t act because of impulse flooding in your brain.

    And read about the strategies criminalists use for their searches.
    Are axes of content between the words of any document and there are more axes like this
    Examples which can compress it.

    For my Site I can tell you a little rule you can sort them easy:

    Ever shorter ( not on letters ore words) the URL, ever higher it had to stand
    Or more level ( /../ ) ever deeper down to sort

  62. 11 Feb 2006.
    Hi Matt, Hope you can help here…In your entry above it is written…
    Q: If I want to get rid of but keep, should I use the url removal tool to remove

    You say NO, …So how SHOULD it be done? by a 301?
    I have had a 301 in place for several months now, and yet the non-www pages have NOT been removed, how long does it take? Is this the way to do it?

    and whilst on the 301 subject, is there a definitive way a 301 should be written?

    RewriteCond %{HTTP_HOST} ^ [NC]
    RewriteRule ^(.*)$$1 [L,R=301]


    RewriteCond %{HTTP_HOST} ^
    RewriteRule ^(.*)$$1 [R=permanent,L]

    OR EVEN…

    RewriteCond %{HTTP_HOST} !^
    RewriteCond %{SERVER_PORT} ^80
    RewriteRule (.*)$1 [R=301,L]

    which (if any) of the above are correct (seeing as all 3 have been recommended ๐Ÿ™

    Some assistance/clarification in this matter would be appreciated


  63. Matt,

    Would the following two be considered canonical URLs?

    The 2nd URL is an affiliate link posted on the affiliate’s site. When a visitor comes to the example site, is it necessary that the affid be stripped off the 2nd URL and redirected to 1st URL?


  64. Matt,
    How critical is it to use the

    RewriteEngine On
    RewriteCond %{HTTP_HOST} … redirect to the www version?

    I was experimenting with going without it, so I do have both versions of my domain active, but I always use the www version for links. I have read some claims that other search engines such as Yahoo don’t like 301 redirects.

    I’m doing terrible on Google right now. I don’t suppose this www issue could be my problem? I know of other people who’s sites are doing well on Google without it.

  65. Recently i have implemented 301 permanent redirect from to All my internal pages except home page is listed in Google without “www”. My internal pages are ranking high in SERP. After implementation of “301” will i see any changes in the SERP as well as PR ? Or it will remain the same ? If at all there is some changes, how long it take in Google to index all my pages with “www” ?

  66. Quality Guidelines – Basic Principles

    * Make pages for users, not for search engines.
    * … Does this help my Users? Would I do this if search engines didn’t exist?
    * … Webmasters who spend their energies upholding the spirit of the basic principles listed will provide a much better user experience and subsequently enjoy better ranking…

    Google are full of it! One the one hand they say produce your website for your users and not for search engines and on they other they tell you that you must do certain things just for search engines. For example:

    * Allow search engines to crawl your site without session Id’s or arguements that track their path through the site.
    * Don’t use “&ID” as a parameter.
    * Have other sites link to yours.
    * Fancy features [which enhance the user experience] such as JavaScript, cookies, session ID’s, frames, DHTML, or flash…search engines may have trouble crawling your site.

    …and now the latest gripe from the search engines, they can’t tell the difference between:

    ..and that is our fault, why?

    Google are full of it.

  67. Put in my vote that there should be a function in SiteMaps to let webmasters/siteowners choose which version they’d like to return in results (this would cut down on your db site considerably), assign all link value to one, etc. I would really help for those that have a dmoz listing with non-www and say a listing with the www. version. Maybe they already get the same link value as long as there is a link, but to be able to define it without going into htaccess or conf would be great!

  68. Google is behaving strangely to one of my sites’ PR…:(

    It is 2…3…5…6 ..keeps changing every hour!

    Looks like a problem w/ Canonical Issues..In the rankings, it has stopped showing “www” with the Url.

    Should i go for a 301 redirect from non www to www version or wait till Google Update is over?..Any suggestions would be great!


  69. Thanks Matt, your information is very helpful. I was wondering if you can help. We have over 1,000 supplemental results due to a site redesign. We have worked very in making sure they all have the most relevant 301 redirects. So far Google has not removed the supplemental results. Is thier anything I can do to speed up the process in having Google get rid of the supplemental results? Does supplemental results have any effect on rankings?

    Your help in this matter would be greatly appreciated.

  70. Canonicalization makes no difference. All this time people are chasing their tails worried about being penalized by the big ‘G’ for canonicalization issues.

    My major competitor has both versions of their URL cached by Google and yet they have been consistently ranking No.1 for ‘credit cards’ on regardless.

    They even have some really neat appandments to their search result with other internal pages directly linked from their listing and a ‘more results from this site’ link. Google apparently really likes this site!!

    They have been ranking consistently for years (as I had) for this popular search and they still are.

    I have set up redirects and I no longer rank at the top like I used to. OK, so I had some bad neighborhood inbound links for a while which didn’t help but setting up the redirects hasn’t made Google forgive me by any means.

    So if think it is the answer to getting back in the Google SERPs you are gravely mistaken.

    It is also highly unlikely to be the reason your site was removed in the first place.

    Personally I think this is all part of ‘G’s big power trip. Two guys from a garage that have forgotten their values….

  71. Thoughts on the trailing slash…

    Websites for visitors not for search engines.

    Go out in the high street with two pieces of paper. One with your URL showing a trailing slash and one with your URL and no trailing slash.

    Ask random people which one they are more likely to type into an address bar if they knew the site and wanted to find it.

    There’s your answer.

    Unless of course your site is wholly targetted towards knowledgeable developer types… Like this one…

  72. Colin: I’m with you!

    I (and many, many other people) have spent my life building quality websites that work well for real people. I’m a consultant and people come to me for quality work and quality advice.

    Now I have to do what my clients’ SEO “gurus” tell them to tell me to do – and it regularly contradicts not only my interaction design decisions (not to mention usability and accessibility). My advice is not being sought in these cases: I am being told what to do.

    Not only that but most what I get told does not hold water – it’s full of contradictions about session IDs (if Google doesn’t index them how come there are so many in Google’s index?) and redirects (they’re bad, they’re good, they’re essential) are just a starting point… And the SEO people behind all this are typically not very technical so – in effect – they are spreading FUD to larger companies to make money from them. Isn’t that called scare tactics?

    I’m not being precious here: I’m pointing out that we have crossed a threshold where what’s good for people may not cut it anymore. Effective websites are now Google-friendly – and this means holding back on the innovation and user interface improvements.

    Is this how it should be?

    I mean, think on this: Google and the like are essentially page-based. Web 2.0 is not.

    So perhaps we should not bother with Web 2.0 applications..?


  73. I recently had a good experience with the 301 redirect of pretty much my entire site. After two and a half years I figured out the importance of sitemaps, redirects, keywords, etc. So this morning all of our newly updated pages received their updated PR. The first level navigation pages received PR5’s and the next level PR4. The only problem I have is my home page for the www url is PR3 and my non-www url is also PR3. My guess is my homepage should be a PR6. ???

    I use windows server and IIS and have used 301 redirects throughout my entire site structure with success, but I can’t figure out how to do a 301 redirect on either the www or the non-www in the IIS console. I was trying to do a 301 on the directory, but wondered if you do the redirect on the index.htm file? I know with Unix you can write it into your httpaccess.txt file, but is that possible on IIS. Anyone know the answer here? Much appreciated.

    Great articles on this site by the way. Thanks Matt!

  74. Oh, the 301 redirects from the old files to the new were done about a month ago. Right after the previous update and right in time for this latest update it appears. So not sure if I just had lucky timing with the update or if it takes a month normally. From what I’ve been reading my guess is a month is not the norm.

  75. So, what is the correct .htaccess line to resolve this issue?

  76. How can I do a 301 redirect under Apache?

  77. Hi Mat,

    My question has to do with www version vs. non – www version. The non-www version of my site has a PR of 3. The www version has a PR of 0. Recently, I have been added to a ton of listing directories all linking to the www version. I don’t want to split backlinks between these two sites and would prefer to build on the non-www version PR. Should I do a 301 redirect from the www version to the non-www version? Will the backlinks from those directories still count towards my non – www version?

    Much Thanks,

    Ryan Tongg

  78. Matt,
    We have not used Base Hrefs in a long time (’cause we have not needed them) but is this not at least as valid if not a better way to take care of multiple domains responding to the same site? W3C is very clear about what a 301 re-direct should be used for AND this should also avoid 302 attacks.

  79. My site is on shared hosting windows 2006

    I wanted to redirect non www to http://www.domain

    First I used code below

    this code gave error message on firefox
    error message: “Redirection limit for this URL exceeded. Unable to load the requested page. This may be caused by cookies that are blocked.”

    Second I used thi code and it seems working fine for Intenet explorer and the mozilla firefox.

    my question is I hope I am not bordering the Google guides, help me please if I am.

    working code
    “” Then
    HTTP_PATH = request.ServerVariables(“PATH_INFO”)
    If Left(HTTP_PATH, 8) = “/default” Then
    HTTP_PATH = “”
    End If
    QUERY_STRING = request.ServerVariables(“QUERY_STRING”)
    theURL = “” & HTTP_PATH
    if len(QUERY_STRING) > 0 Then
    theURL = theURL & “?” & QUERY_STRING
    end if
    Response.Status = “301 Moved Permanently”
    Response.AddHeader “Location”, theURL
    end if


  80. I’m totally confused
    I wasn’t getting indexed so decided to put in a sitemap after making some file name chages to my site: I changed every page but the homepage index.html. (site is just about a year old)
    The site map area showed a non-www url for my site and wouldn’t take a site map. I use the www. for all links in my site out of my site and for inbounds. So I deleted that domain from the sitemap area., to me it didn’t reflect the domain I paid for which was www. (i’m pretty green to this stuff) I added the http://www.domain and it accepted the site map and everything is crawling complete with 404 errors. ๐Ÿ™‚
    I did this and am now finding out the indexing wasn’t happening probably because I need more inbound links.
    So in deleting the non www domain from the sitemap area how much damage have I done? Will everything get pulled for 180 days? how soon will that happen if it does? if it does pull both then does the one resurface after and the other disappear?
    I haven’t had any penalties and the only guideline not followed is the lots of links one.

    Thanks for any help on this

  81. Hi Matt, this is a very useful post for me. I’ve been wondering, why all of the sudden links from Google changed from to I didn’t understand this.

    Knowing what’s in this post, I apparently need an permanent redirect from to Now if I only knew how to do that on my Apache webserver…

  82. For rck
    There is an article here on appache 301 via htaccess. Im no programmer but i enlightened my programmers to this article & they were thankfull. Hope it helps you.

  83. I wonder. Were you ever struck by lightning whilst fishing ?

    BTW. It keeps saying (5th try) Invalid security code. Press your browsers back button and try again.

  84. VERY happy I found this website! I get sick of some of the “speculation” on the forums and it is nice to have the “final word”.

  85. – Initially our site was configured solely as and not
    – Google initially indexed all pages starting with (some http, other https)
    – Both https and http versions were accessible on the domain (which I know is not good practice. This also need to be fixed).
    – A redirect in the apache config file was put in place from non-www to www for all pages starting with
    – Ex: redirects to
    – By error, this was not put in place for pages starting with
    – Ex: does not redirect to

    – Since the pages starting with were not redirected to, both versions of these are accesible and indexed, creating duplicate content on Google.
    – Our home page: is the only page who has a page rank as all other links starting with www were not indexed by Google.

    – Is the best way to redirect all to and restict access to https only to those pages that need to be secured?

    Any advice or help would be appreciated.

  86. While in the process of changing my pages in .htaccess, part of my site was indexed. As a result, when I look at my Google sitemap account some pages are listed with both a trailing slash and no trailing slash.

    Should I be worried about the spider looking at this as duplicate content? I don’t want to be penalized. All pages now have a trailing slash.

  87. Hi Guy H, your tip worked great, thanks! ๐Ÿ™‚

  88. Is there a suggested canonicalization? I see most every authority site is now directing to and I’m wondering why. Isn’t the “www” counted in the character length of the URL?

  89. “but because it can take longer for us to crawl supplemental results, you might not see that update for a while.”

    A very crude cybersquatter hijacked my client’s domain for a short period of time, this was about a year ago. (I helped her get the domain back.) The cybersquatter used “bad neighborhood” keywords that are too disturbing to mention. I recovered the domain about a year ago and have done my best to repair the damage to her reputation. Now this respectable doctor (a year later) still has these raunchy keywords in Google’s supplemental results, long after the domain has been returned to the rightful owner and cleaned up. Is this why her PR is still 0 after a year and she can’t be found in the organic results?! (Not sure how long she owned it before the cybersquatter got his hands on her name…) Ugh!

    Is there anything else I can do to repair her reputation in Google? Anyone can agree that this cybersquatting hijacker has damaged her reputation in Google, terribly! I’m going to try the “automatic removal tool” next. I just can’t believe these raunchy results are still in Google after all this time. I feel terrible about it. Imagine how much business she has lost because of these old uncrawled results. Because of this she’s had to buy AdWords traffic, because it’s nearly impossible to find her buried in the organic results. I hope that Google will consider the webmasters of domains that have been hijacked and/or cybersquatted for a short period of time–it’s terrible what can happen to your reputation in a short period of time, in Google especially. Thank you so much for your time. Best regards.

    PS: She ranks well in MSN and not good in Yahoo
    PPS: I love this blog. I have learned so much here!

  90. hi Matt,

    I got two questions:
    Q1: I have been assinged to do 301 redirect from to

    When I use the following code:
    Response.Status = “301 Moved Permanently”
    Response.AddHeader “Location”, “”

    It goes into infinit loop, what else I can do about it?

    Q2: The search engin shows two different page ranking for the same page: and

    How can I make sure, it only one of them comes up in search enging?

    Thanks a million

  91. Thanks for the info. I always had a doubt about the URL’s which is now cleared.

  92. Dear matt

    thanx for the information

    i need your help to get all information abt google update is there any website where i can study all the information abt google updates till date


  93. Thanks for the info.

  94. I’ve seen this article referenced many, many times…definitely a useful resource! As always thanks for the insight Matt!!!

  95. It has been asked many times in this topic but I don’t see an answer.

    The question is: has PR4 has PR2

    Now we are going to use 301 on to
    What kind of effect will this have on the PR and the backlinks?

    Will you loose the PR4 and all of your backlinks? Or will it go to

  96. Dear Matt,
    thanks for explaining difference between domain syntax and in such a lucid way. I was confused to find different PR to my old site.

  97. Will a 301 redirect transfer the pagerank to the new location?

  98. Hi

    I have what I think is a simular problem. I have a site that is index on both http://www.domain.tld and the corresponding IP-address. SiteMaps did solve the problem with www vs non-www but I can’t find anything regarding IP.

    Is there any safe way to remove the pages from the index and keep them out?


  99. I have the same problem. Some of my pages are indexed as www and half are indexed as the ip address. I would love to find an answer.

  100. In my way i have the same problems.

  101. Hi,

    If I have a url:

    and want to redirect to:

    Should I use redirectMatch or rewriterule to do this, as currently I can’t seem to redirect urls with get params.



  102. Thanks Matt for the continued explanations and advice about this stuff. I have been reading up on Canonical issues for a while (suffering from one myself due to not knowing about them before hand and not using 301 protection), I have set up a 301 and on server name resolution so that all requests for the main index page go to (the trailing slash is always added anyway).

    Google still shows and in the serps and is docking my PR due to it. Will the 301 be “picked up” by the main googlebot and remove the “index.php” reference from the results in due course?

    Also I can’t fathom out why this sort of thing isn’t under the webmaster’s control? If I know that the result is WRONG then there should be a system to remove JUST that reference? Is this impossible?

    I was thinking that a verification code tag could be uploaded to the page (as per sitemaps) to prove you are the owner and google could see at once that any reference in it’s database to THIS page should be removed and if need be replaced with the TLD (or specified alternative).


  103. To clarify, I realise you can remove URLs right now (supposedly) and set your tags etc to aid this process, but there seems to be no specific tool to address the removal of results that point to a valid page “index.php” that you most certainly do NOT want to remove from google but just want to point to the TLD instead.

    I can remove a page I don’t won’t but not a page that is being retured twice in the results.

  104. Personally, I think everyone should stop kissing google’s a**. I do just fine without google. Most of their search results seem to be crap anyway.

    Admit it, accept it, and make google follow the people that own the sites. 100 million people think they have to do what google wants? Ridiculous!

    Old fashioned style works for me. REFERRALS FROM THE CUSTOMERS… Much more productive than some damn search engine that wants to dictate how I should construct MY site. Then wants to charge me hundreds or thousands of dollars to buy advertising in their site.

    I am sure that 1000 “gurus” will be chomping at the bit to explain why I am wrong. Save your breath. I don’t care what you say. I could care less what “canonical” refers to. Google is on my site everyday and if they stop immediately then i just save a little bandwidth. I am not going out of my way to please someone who doesn’t work for me and is not a customer. F**k them. If all of you thought the same way then google wouldn’t be causing you so many problems.

    I say take care of what you have and they will take care of you.

  105. What would cause Google crawlers to index our site under an https://www. url verses http://www. Google has crawled and provided a PR for pages that start with https pages (only) rather than http://.www.

    Our home page (http://) url has a PR of 4. I realize the pr does not mean much but it still is an indication of being crawled and recognized. The https:// has a PR of 2 as well as the inner pages of the site.

    The first item under ‘Indexed pages in your site’ list in Google is the following

    Title of the Website Site
    This is the desription of the site and what it is all about, etc, etc. – 43k – Cached – Similar pages

    Note the https:// instead of http://, previously it was the http:// in my old provider world.

    It has been 3 months since I switched providers and no pages are being re-indexed by Google on the http:. The site is over 3 years old and has never played in bad neighborhoods. This does not make any sense, it would seem that somewhere Google has been told that https:// is the main url to follow.

    Any insite would be greatly appreciated…

  106. Would google deciding to show my site at vs. have a result on google rankings? We have sustained a drop for a bunch of various terms and have noticed that Google has made those pages non www.

    I’m a bit of a novice, but also don’t understand how google finds non www. pages when we have never created anything that wasn’t www.

  107. Hii Matt,

    Gr8 to read about Canocinalzation frm an trusted source. Here m having an question to ask bout hope u’ll mail provide the solution asap.

    Q : Suppose if I provide an 301 Permanet Redirection from to and again that page has been redirected to, will it cause serious problem, coz m facing such problem or simply u can say Dual redirection problem. Instead of providing 301- redirection from to it is getting one more in between re-direction. Site search results is showing pages as Supplemental. Plz throw some lights on this “Double Canocanalization” too…………..



  108. Hi Matt,

    Thanks for the elucidating article!

    I have been using the 301 redirect since the start, but I wasn’t sure why…

    This makes it clearer.


  109. Hello,

    My website: have page rank 5 in google. But from last month, I cannot seach it in the google instead of I enter the search query by my domain name: Could anyone please, show me the reason and how to reslove this problem.

    Mr Ha,
    Vietnam Paradise Travel.

  110. I am using dynamic IP from DynDns and the only solution they can fix the problenm is the 301 redirection.
    I guess using a static ip will improve web page rating

    Good and informative, thanks

  111. Matt, I can’t find any definitive answers so I’m counting on you to give me the final word.

    Does url parameter order matter?
    ( vs.

    If it does, can you briefly explain? Thanks.

  112. Hello Matt,

    Thanks for your information on canonicalization. Please suggest us to do a liitle to our blogs, we have heard that “.html” is having more value that urls ending with “/” . so we made url rewrite options in wordpress and all.

    Now as we noticed we haven’t get any pagerank and updations for our blogs yest where there are about 1 year in age.

    So i request you to just give me a little suggesstion that we do remove domain from the google index and make changes to our urls and again we request google for reinclusion.

    If yes, please tell us a word “yes” will help us a lot. Hope your suggesion also will help many blogs on www.

    Thanks in advance

  113. Dear Matt,

    We have PR previously 5 for our website and all these are static pages. when we are check with iwebtool it is showing 4 is this because of canonicalization?


  114. I changed my preferred URL in web master tools and now a lot of non-www links are showing up in the search. It also seems to have affected my ranking since I get a lot less visitors after this.

  115. Great article, I especially like the comments! Heres what I’ve been using. From: Ultimate htaccess Article

    Options +FollowSymLinks
    RewriteEngine On
    RewriteBase /
    RewriteCond %{REQUEST_URI} !^/robots.txt$
    RewriteCond %{HTTP_HOST} !^$ [NC]
    RewriteRule ^(.*)$$1 [R=301,L]

  116. Matt,

    I’ve been reading up on this and my question follows over the actual domain put to folders within that site as well…

    For instance, would you also recommend :

    be changed to


  117. I have also tried playing around with the link in setting but still I get zero link-in count on google search for my domain
    I am positive there are links to it if anywhere but in few web business directories but google just does not seem to pick them up.
    Can someting be done there ?

  118. In general practice, and for Google in particular, is it a problem to have two separate URLs for the exact same web page – i.e., the web site would be cloned to display with either URL in the address line. example: and both displaying identical content (except for the respective URL in the address line)?

  119. Hi Matt,
    Thank you for sharing this info.
    I have only one question,
    which is the proper way to link to the main root:
    is there a difference?
    Thanks again Matt

  120. I found before several weeks that in the google webmaster tools there is an option. You can select how to be interpreted your url with or without www for and i’m already activated for my site and you can check it here

    thanks for that article

  121. I’ve used 301 redirection for one of my site, it works excellent for Google but adversely affected, Ranking in other search engines ๐Ÿ™

    Any idea Matt?

  122. Canonicalization? We`ve never heard before and never had problem.But now we have a “canonical” problem, i.e. our site is accessible via both and (without the www.) and google is indexing these separately ( returns 95 pages; returns 64). We decided version with, and also created 301 redirect file. Seems to be working. But now our Google search result is gone. We thought maybe we are not indexed again in Google (as has happened several times recently), but we are there. What shall we do to bring our site’s results back to as they were 2 days ago before we did the 301 redirct ? Will the next time Google spider crawls it search result will be same is before? How did this canonical problem arise ?

    Thank you.

  123. Great explanation Matt, in conclusion it’s not good to use and promote both form of urls?
    How long it take to google to fix the problem if I choice from webmaster google tools to use the domain with www?

    Thank you

  124. Hi Matt, thank you for explanations.
    Is this available and for subdomains? I mean, will Google see any difference between http://subdomain.domain.tld and http://www.subdomain.domain.tld ?

  125. Very informative infomation. I noticed alot of sites with url like I was just wondering how they do that?

  126. Can we talk about sub domains

    I use sub domains for tracking in Google addwords and offline marketing and sometimes on banners linking to the main site (links to the home page) (links to the home page) (links to the home page)

    Is this spamming? Will this low our result?

    If this is classed as spamming I do not want to loose the external links to the sub domain is it a good idea to canonicalization the sub domain?

    Please help


  127. Could sure use some help.

    I am working on a site which has multiple domain names, the main domain ranks well and until recently the other domain did not show up in google at all. However in mid february i changed the dns around for the the “alternate domain” to correct some problems I was having. During the past 2 weeks I have changed the layout of the site and added a bit of new content.
    Today the main domain moved a bit higher in rank, and suddenly the alternate domain’s “index.html” has also appeared in the rankings. From what I have read it seems having both domain names for the same site show up in google could cause problems for the site, if this is true what can I do to prevent any problems?

    Note: The alternate domain is the domain that the owner of the website would prefer to be shown in searches, however it is the one with very few backlinks. If a 301 redirect is the solution, would 301’ing the main name to the alternate cause me to lose position?

    Thanks in advance for any help in this matter ๐Ÿ™‚

  128. Hi Matt. Great post. You seem to have opened up quite a can of worms here. And it appears that there are not any clear answers to a lot of the questions that have been asked. So into the pot of unanswered questions, I will add a few more!

    I recently put a 301 redirect on my blog to combine the non-www domain into the http://www.domain. I chose the www version because in my research most people seemed to agree that it was the best choice. Unfortunately my non-www version had the page rank for my site (a 4 at the time I combined them). I also used Google’s webmaster tools to combine the two domains into the www version. My first question is, did I mess up? Should I have combined them into the version with the higher page rank and the majority of incoming and internal links? If so, should I now go back and change it? And if now, will Google eventually figure out that the new, combined, www version is the same and correct the page rank (currently unranked)?

  129. Hi Matt,
    Please suggest which is better
    anchor linkking Home with domain name or
    anchor linking Home with

    will our index.html page get supplement if we not give its link in any page and use for reaching index page



  130. Hi Matt,

    Recently we have had a URL replacer upgrade for our CMS so that we can replace the horrible aspx?page=22 to a nice products/products. The only problem is that now 2 urls are showing for the same page! what is the risk for getting penalised for this by the search engines and what ways are round this?



  131. Hi,

    Does anybody knows if this problem affect also the google, PR, I mean, when my site is without www, has PR 0, and when is with http://www., has PR4?


  132. So , it is better to use with www if that was how you start! If you change in the future, and cut off www, google penalize you?

  133. How do we fix this? seems that my web site auto adds, index.php to then edn of my usrl even when I do not want it to show up and every one thatinks to me, links in with just .com at the end.

  134. Thanks for the great tips here. I never knew that all these (,,, got some difference lol ๐Ÿ™‚

  135. luckily it’s only my php pages which have these issues (which there are only a couple) thanks matttttt

  136. Hi,

    How can I check that how many pages are in supplemental index at google?

  137. Matt,

    The vast majority of our inbound links contain “www” and we recently updated the site to have 301 to But, Google appears to already have a preference for our non-www listing. Will this 301 hurt us? Should we be concerned that Google preferes the non-www version?


  138. Helo, i have same question as what raheel asked. How can I check that how many pages are in supplemental index at google? Please advice…

  139. Thank you for this excellent free facility. I was never aware of this issue with Google and spiders until now. I had unwittingly assumed that all the global differentails pointed to my site, I think I will take you adivce and redirect the unnecessary linked pages back to my proffered domain.
    I note Igor concerns over viewing these seemingly duplicate pages as spam. I certainly hope this is not true and does not detract from page rank and site performance.

    Thanks for the advice,


  140. Thanks for the useful information. Lots of this seo stuff can be confusing and there are so many little things to know about to do it right.

    Please keep the good info coming. It is greatly appreciated.

    Regarding the ‘supplementary’ results in Google: I had some old files show up as supplementary because I changed the name of the scripts but the content was the same.

    Should I convert these scripts to redirect to the renamed scripts? I’m doing that now but I’m concerned that redirecting will penalize me.

    Any thoughts?

  141. Hello Matt,

    This is my first time commenting or asking a question on your blog. I felt it was my last resort for an issue I am having with a form of canonicalization on my website. The problem I am having is that the googlebot is indexing pages in the secure form https as well the non secure form http. I am finding that this is creating duplicate pages in the Google index and is killing the pages ability to rank well since they are showing as supplemental results. I found a blog where someone mentioned using the following php code would stop the googlebot from indexing any https pages.
    ‘. “n”;
    In your opinion do you think this would work? Is there any other options?

    Best Regards,

    Greg Knighton

  142. Hello. Im not really shure, but f.e. in space of internet in the Czech republic is using different system. You can write this url in bouth way and you get the same result.

  143. The vast majority of our inbound links contain โ€œwwwโ€ and we recently updated the site to have 301 to But, Google appears to already have a preference for our non-www listing. Will this 301 hurt us? Should we be concerned that Google preferes the non-www version?


  144. This is a great post, clarified many things I was doubtful for. But I still have a question: I recently added the http://www.- prefix in all my internal links in my blog, and added 301 response code to all non-www requests. I don’t want non-www URLs to show up in Google’s results any more. Will they ever go away?

  145. Matt and rest of the fellows

    The issue of Canonicalization can also be resolved by registering your domain to and then later you can submite your site for sitemap and it also provide usful information. It also provide a featuer called “Preferred domain” If you select option which states

    “ and are same”
    That can solve your issue right away, because if you dont mentioned if explicitly google will defautly take www and non www version URL as different sites and when it will index it will found similar contacts which can cause spamming and other issue.

    So try webmaster feature of google that helps in many other ways

  146. Thanks for sharing this info Matt, but when i redirected my website from the www to non www url’s all my links in the Google Webmaster Tools dropped done and even with the in google, So am just wondering is this normal and happens?
    Thanks again this was a really helpful information.

  147. Hi I found the one Scenario in which one company has registered multiple domain about 15-20.

    For an example

    if you click the following link for keyword “one stop motors” the company has number of websites ; and the most interesting thing is this websites do not redirecting to one site but have almost same theme, content and if you goto contact us link same telephone number which suggest this all sites have same parent company regardless these sites has been registered in same server or different servers

    Is it ethical or unethical way??? does Google punish such sites/company?

    Kindly Let me know


  148. Great post with a long discussion this topic.. Well Matt I want to know a url with www and without www which is better ??

  149. Hi Matt,
    Thanks SO much for sharing this information! I just had this problem when I switched to and it’s incredible what people will all tell you! Some were right and some couldn’t have been more wrong! Your post here helps me know which is which! Now if I could only pronounce “canonicalization”!!! ; )

  150. I have a question on URL’s when using re-writes… I with the rewrite ule you can use characters not used in regular URL’s such as the + sign, as an example

    The noticing the search result string with google doing a search on “my best keyword” the string is search~~~”my”+”best”+”keyword” so the quetion is, is there an advantage to using the “+” as a seperator?


  151. I am tring from to basically a www redirect to a non-www reditect but the sites are secured. Any Ideas?

  152. I am a newbie to this and am trying to get my head around it.
    I read these posts and decided that my site had a canonical issue.
    each of the folling go to my site

    Then I checked microsoft and google but they behave in the same way!
    So, do I have a canonical issue? Do Google and Microsoft have canonical issues? If so why is it an issue to me and what should I do about it.

    Apologies in advance if I have missed something really obvious here. I have read the posts a few times and everytime I think I undrstand it I read on and moments later have to admit to myself that I just dont get it. Can anyone provide me with my Eurika moment on this….

    Thanks in advance

  153. I finally figured out how to remove the subdir from the URL for my content management system site, without breaking the page relative links, so I thought I’d share it here, since the problem is mentioned by Spike in the comments above.

    RewriteCond %{HTTP_HOST} ^(www.)?$
    RewriteRule ^$ [L]

    # add back the missing /subdir/ for relative links but don’t rewrite the browser URL
    RewriteCond %{REQUEST_URI} !^/subdir/(.*)$
    RewriteRule ^(.*)$ /subdir/$1 [L]

  154. Wow! Early in the (pre-Mosaic, pre-Google!) days of the web, a mentor ‘beat into my head’ the necessity for canonical names, because browsers were unreliable in dealing with relative directory references. I’ve been in the habit of using canonical names for years– now I realize some of the other benefits!! Thanx for a great blog!

    -Brother Aviaf

  155. Maybe already asked, but what about

    i want my main page to be

    should i write it this way on the htaccess file?

    redirect 301 /index.php /
    redirect 301 /

    thanks a million !!!

  156. If I Click On One URL How Can I Know Whether It Is a 301 Redirect or It Is a 302 Redirect? Can Anybody Help Me Out In This Pls?

  157. Great post …one and only post that actually helped me better understand. Thanks for the post.

    Webhost moved some files …4 days later is working but is not showing up.

    On proxy servers both are appearing….is this a dns cache issue? Usually sies propogate 24-48 hrs later.

    thanks again Mario

  158. I docked here by searching “url redirection not good for seo ?” Any idea ?

  159. Hi Matt,

    Great post, has proved to be very useful. I have a html based site that origionally used index.html for the homepage, I have recently set up a index.php page instead so i can incorporate some other php content. I have set up my htacess file to solve the canonicalization issues on the index.html, without www. and without / to point to The problem I have is the new .php page. If I add the .php to my htaccess it sends users and search engines into a loop, which causes all sorts of problems. Is there a way to get around this? Thanks Richard

  160. excellent post matt

  161. My client recently implemented SSL on their website and the website is currently opening in but in Google the Http:// version is cached. Normal http traffic has been getting to their site fine, but the question is could this https issue be causing some recent drops in seo rankings across Google, Yahoo and MSN?

    Please HELP!!!!

  162. What if I want to make this repair on blogger, shall I resubmit?

  163. I recently changed my index page from index.htm to index.php. The .php page only has a page rank of 2 where as is page rank 4.

    Is that pagerank 2 being factored in by google?

    My htaccess is specifying what page types to look for…

    Thanks for the info ๐Ÿ™‚

  164. Nice information. I had not clue. The 301 redirect is great. Will do that right away!

    Golf on!

  165. Hi matt,
    thanks for the post. Now i know, i have done the write thing by 301 non www to www. But, one more thing i have to change again ๐Ÿ™ my inlink have both styles, i.e. and ../example.html. I think, i am lucky enough that i did not get the dup-content penalty in G, as all my pages are indexed in G.


  166. I am trying to figure out how to use a htpaccess.txt file to make this fix. Anyone know how? I want everythign to go to my page but can’t make it work.


  167. Matt, Your posts are so informative

    I am sure i am not alone when i say that i have never heard of canonicalization and it will take a couple of reads of this post to get my head around this concept

  168. Matt,

    I was wondering if you or anyone could give any suggestions whatsoever regarding this matter. What a nightmare ๐Ÿ™

    I am having some major issues with my site listing on google. For some reason, our development server/domain is showing in place of our site. I don’t even want it to show in google. How do I get rid of it and get our site listings back? I am having 301 redirects added to all pages on the development server/domain. Is there any other way I can speed up the crawl process and get this fixed. This is just killing my rankings and sending potential clients to the wrong domain. Google does makes up 80% of my traffic.

    To see the problem first hand google and look at the domain showing in it’s place ๐Ÿ™

    I have no idea how this happened, please help!


    Matt Johnson

  169. i’m trying to use the redirect 301 but i think Big Daddy, has come to those not using or don’t know how to use Redirect 301. Thanks Matts

  170. Hi Matt, thanks a lot for your help!

    Could you help me understand what’s happening with my site? Until now, I was using OR for inbound links. This is until I understood there was something wrong in that and it was better to be consistent. The site was redirected to (from htaccess), so when users would type in, it would become It is still like this. Anyway, my webdevelopers have made some changes recently in the htaccess file, then change it back when I asked them too. In the meantime, my Page Rank changed to… 0 ! and there is no inbound links anymore when I search on Google.
    However, when I go to, then Page Rank comes page (ok it’s only 1 but still it’s important to me).
    What is happening? Is it possible that Google removed the url or is it possible that the robot just passed through at the same time?
    If you have any idea and can help me, that’d be great!

    Thanks again Matt


  171. Hi Matt,

    I just discovered your post today. Really informative.

    Last year I wrote an article on the same subject although not so detailed, for my website and for a magazine that I write for:



  172. I’ve not had any problems with 301’s, took me a while to get around them but they are so powerful. However, is there any limit to the number of records you should have in a .htaccess file. My file has grown and grown with tweaks and changes to page structure which thankfully are now stable. But surely the 301 redirect should really only be a temporary fix e.g. 6 months while you correctly point all your back links and the search engines drop the old URL from it’s index. Is there any issues/redundancy by leaving the 301’s in place long term?


  173. Thanks Matt!

    I just built new website and was looking how to go about www and non-www URL. Your post was the first I found and led me to complete solution.

    I also amended settings in Webmaster Central!


  174. To answer Bingo’s query –

    The ‘www’ prefix is simply an alias or nickname (a ‘CNAME’) for your domain’s host. This alias has been registered in DNS by your hosting partner. Thus the DNS contains two records that point to the same host (the one hosting your site) and your 301 is redirecting to the alias. All the time the ‘www’ alias exists in DNS, and you are redirecting to it, your 301 must remain in place.

  175. Very interesting post here!

    Since it is (presumably) complex work for Google to pick the right canonical URI for duplicate content, would it be preferable/desireable to provide as few URI’s as possible for any given content?

    I would assume so (correct me if I’m wrong of course). While it is fairly common practice to have and be two URI’s with the same content, would it not be prudent (if the webmaster could easily do so) to just return a 404 for and make all of your links the former?

    A slightly tangential issue that I simply have not been able to find a solid answer on the web elsewhere… all other things the same, will a .html extension on a URI provide any benefit over no extension on the URI at all?

  176. I have been told to use www instead of no www. Is there any SEO benefit to doing so? My understanding is that it doesn’t matter as long as it is consistently applied.

    The only(small) benfits of using www are
    1- Ability to restrict cookies when using multiple subdomains.
    2- Most email/text editors recognize as a link.
    3- Third party sites may include it when linking to your site.

    Is this significant enough to not adopt a non-www convention for my site?



  177. Does the GoogleBot care about the ordering of parameters in a url ie

    treated as the same url ie should I invest any time ensuring that multiple parameter urls always have their parameters in a set order?

  178. thanks for the info, it really can be a little overwhelming staying current on all these things but your blog has opened my eyes and allows me to truly appreciate all the work that goes into building successful websites.


  179. Great article. Thank you.
    We are switching our entire site over to a new look and feel. We have html pages on the old site and don’t want to loose rank for some significant keyword phrases. If we change all of the extensions to php – will that cause any rank related loss or will the bots just deindex the old pages and pick up the new ones since the titles/content etc. is the same. I appreciate any response.

  180. Question – I have which I want to redirect to the IT guys says that it will loop since the redirect is domain to domain. What are we doing wrong? And should we even be doing this since a Google search brings back as my home listing????


  181. Thank U Matt, actually i got this article from digital point i asked question their regarding canonical url. as u described here i get clear about canonical url and clear my other confusion about this.

  182. I know this is an old post and i did not read all of the responses so i am sorry if this was been asked.

    My question is simple, in regards to www and non-www in Google Webmaster Tools you can tell Google what your preferred domain is i.e. or

    to me this seems like Google knows about both domains and asking me which one is correct, meaning i dont have to create a 301 redirect to the www domain. or my assumptions correct or is something else going on?


  183. Finally a clear and concise explanantion of the issues.

    I was fairly sure I understood them but had a nagging doubt because everything Ihad read was really buried in techno speak or just badly worded.

    This article is excellently written and puts the point across clearly.

    Thanks Matt.


  184. Is this advice best for larger sites? Are sites under 20 pages in need of 301 redirect as well?

  185. Thanks for the article. I think I’ve read it three times, each time by searching for different keywords at different times. ๐Ÿ™‚

    Something worth noting though..

    When redirecting www to non-www and vice versa, many bloggers do it with a plugin. I think it is overkill.

    Implementing this is server-code, in my opinion, is the right way to do it.

  186. I’m having troubles myself with a domain that has been re-indexed on the (non-canonical) https version ( in case anyone’s that interested.

    There were a couple of days where if you went to one of the secure pages on the site (during the buying process) that set the base href to the https version. Obviously the spider found that and starting re-indexing the site on the https version. It’s a bit galling, because the site doesn’t even rank for the company name now. I can’t help but feel that Google have got an itchy trigger finger about matters like this. Surely removing a canonical domain from the index shouldn’t be so easy?

  187. This topic help me a lot. Thanks. I keep think about www and without www. now I know what I’m going to settle the problems. Next time I’ll visit your book to get new useful information

  188. Canonicalization – A new work I’ve learnt today. Thanks for the info. I’ll take you advise on redirection to www for my site.

  189. Hello Matt: Nice to see this post getting a lot of traffic (I found it in Google ^-^).

    You said, “Q: I noticed that you donโ€™t do a 301 redirect on your site from the non-www to the www version, Matt. Why not? Are you stupid in the head?
    A: Actually, itโ€™s on purpose. I noticed that several months ago but decided not to change it on my end or ask anyone at Google to fix it. I may add a 301 eventually, but for now itโ€™s a helpful test case.”

    So, how is the case study going? Google pointed me to your domain. ๐Ÿ™‚

  190. URL Canonicalization is something completely new to me. I never even knew this existed until someone pointed it out to me on a forum and then after reading this article it all became clear. Keep up the good work and keep more good articles like this coming! ๐Ÿ™‚

  191. Hi Matt

    Thanks for this topic, it answered my questions perfectly about canonicalization and have taken your advice about the 301 redirect.


  192. Hi Matt,

    my question is from a slightly different angle. And please tell me if this is simply bad practice or not — I have this year’s website in a “2009” folder, therefore the new current homepage is I have an HTML autoforward from the root index.html file to this and was planning on replacing this with a 301 re-direct for SEO purposes. I have more web 2.0 features in the 2009 folder, but tons of equity on the root domain,, which search engines seem to be seeing seperately.

    What is the best way to connect these two? Canonical or 301? or should I move the site in to the root folder?


  193. Thanks for the article Matt. URL canonicalization seems to be a much bigger problem nowadays with pagination and such. It’s time to adapt accordingly.

  194. We are trying to eliminate our competitors from using our content. It creates duplicate content which is supposedly frowned upon by Google. Is there a good tool out there for finding duplicate content in real time as our competitors add it?


  195. Is on better than the other? I personally prefer the www as apposed to w/o it; it just looks cleaner to me. If its w/o the www, something seems amiss.

  196. I just came across this blog. I redirect my // to // by using .htaccess but how do I know it’s working??

  197. Hi. Thanks for the post. With the help of this post and google webmaster central blog’s post, i’ve worked on canonical tag and here is my little report of experience of canonical tag, if you want to have a look:

  198. Hello Matt
    I got 2 on topic questions for you:-
    Is .com
    and .com/ same or different(Please remove the space)

    Also if I get links to may domain with or without slash does it make a difference. Is the link power shared between slashed and non-slashed domains.

    2nd Question- For non-coder bloggers like me it is difficult to add the canonical tags manually. Does a plugin like AIOSEO for wordpress do this work effectively. AIOSEO says it does insert canonical tags for help in Google.

    Please try to answer if you can.

  199. Hello Matt,

    How can it be redirected to /foldername/ from /foldername/index.php without redirecting error?

    I have tried with .htaccess but it shows “Redirect error” in webmasters tool.


  200. Nice post about canonicalization. May I ask, when you say canonicalization of urls, it just mainly talks on the www. vs. non-www urls? I am trying to decipher why my blog got a PR slap from Google. It just recently received a PR upgrade from Pr1 to Pr2 but it was not long lived. After just less than a week, it was slapped to PR0. I don’t know what went wrong?

  201. You say it’s a good idea to send a 404 Not Found header for pages that no longer exist. Isn’t this the point of the 410 Gone?

    Does Google understand 410 headers? If so – are they interpreted in the same way as 404 headers? Or does Google presume that 404 is temporary and 410 is permanent, or vice versa?

    Thanks for your helpful articles ๐Ÿ™‚

  202. Great info here, Matt! I have been reading people writing about canonicalization for years, but never really understood it. Problem solved. I also wondered if the “no trailing slash” vs. “trailing slash” had any bearing at all on this issue. Interesting that you suggest always using the trailing /.

  203. You just solved my biggest headache that i have been having for a week…Thanks a ton!!! ๐Ÿ™‚ Really impressed.

  204. Hi Matt.
    I m new studying on the web searching . I acutally can not understand yr article exactly . But is got something .
    I hv noticed yr hv old archives in yr blog. I will study from the very 1st page for lesson one .
    Reallly Thanks for your helpful articles ๐Ÿ™‚

  205. Hello Matt and thanks for your article! I have one of those domains where half my links point to and the other half to I have been nervous about adding a 301 redirect in IIS for fear Google may not account for the incoming links that currently point to Is this somthing I should worry about, or is the 301 redirect Google friendly?

  206. Hi Matt,

    Is there any chance you could do an updated post on canonical urls?

  207. Hi. I’d like to know if for the pages process crawling is there any diference between this two types or urls? or My question is, it is easier to crawl a page which is closer to the root folder or it doesn’t affect?

  208. I’ve been using 301 redirect for one of my site. Thanks for re-confirming that it is indexed my Google. Its great to get this information from the right source. Thanks Matt

  209. Hi, I’ve used the script on this site for my PAT Testing website.

    Essentially, if any url is missing the ‘www.’ it redirects. My main PAT testing site has over 80 pages, and I can’t write a 301 redirect for every page!

  210. Hi Matt ..

    Thanks for a good post that suits my n00b reading capabilities .. ๐Ÿ˜‰
    I’m designing a Magento solution these days and found a canonical URL extension and I’m wondering if that’s a good solution or may cause any problems (in some cases)?

    The box said “Requires Windows or better”. So I bought a Mac..

  211. Matt,

    You titled this blog “url canonicalization” but I see that less than half its content talks about this subject directly. This blog look like it could have been two separate blogs. Some of these other comments also suggest you give more information about this subject.


  212. Thanks for this post!

    You’ve helped me solve my problems after looking at my site on Google with the site: command.

    Thanks again!

  213. Very informative post on canonicalization. I had nrever head the word used before in coding but it makes perfect sense that I should hve know about it for SEO. I have been designing web sites for ten years but never began studying it last fall and I can’t learn it fast enough. Just when I think I have a handle on it I find there is so much more to learn. I really respect you guys that share your knowledge with us neo-geeks.

    Thank you for sharing it with us.

    John Roth

  214. There is a setting in webmaster tools for the preffered format.

    I recently added a 301 to keep things tidy, but google already seemed to know that I wanted it to display with www – so not really a problem.

    If Matt Cutts doesn’t worry about it on his site, I doubt any of us need to.

  215. Can we remove canonical problem with Google webmaster settings to choose preferred domain,Please tell me something
    Thanks for this nice post.

  216. Thanks for this post. You could also set the preferred domain through webmaster tools.

  217. Thanks for a very informative post Matt! I’ve started URL canonicalization.

  218. Hi Matt,

    Thanx for this post.
    I only have one question and that is: Does Google see a webpage with a trailing slash on the end and the same page without as a double pages within your website?

  219. How can I standardize the canonization of external and internal links to my site’s home page?

  220. We have taken your advice on board and gone for a 301 redirect for our blog site as there is continuous posting there

  221. canonical tag. I want to start using it but im not sure how to use it right.

    I have a national site and I want to avoid the duplicate content blues.

    So im fine with making unique pages for the states and even the countys thats the easy part. I just dont want to piss google off.

    So for the towns should I set it up to canonical to the county page that it belongs too?

    does the canonical tag do a redirect? what i have read says it treats it as saying the real page info is located here. But does that mean it wont even crawl the page?

    I think i have a great question that NO one touches on.

  222. Hey Matt. I have a situation for which an answer would probably benefit other webmasters in addition to myself. One of my clients, a website that sells nursing scrubs, has been building external links to their site for years. Most of the links that come to their site use the non-www version of their URL ( when linking. Recently they’ve switched to a new shopping cart system, which forced them to use instead of the non-www version. They are trying to get their new version of the site indexed quickly by Google, and they’ve already setup 301 redirects to send their old pages to their new ones. My question is this: Would it be better for them to try to get their new shopping cart provider to change their site back to the non-www version, or should they just direct all incoming HTTP requests to the new www version? My concern is that Google will reduce the value of their incoming links since the incoming links point to a URL that is now being redirected. Thanks in advance for a useful answer.

  223. Google Site Maps Problem:

    Site Configuration > Settings > Preferred domain

    When changing the display it seems it has not verified the domain the opposite of the domain ( needs to verify ownership of and vise versa).

    There seems to be an issue (recently and not in the past) that some users cannot set the preferred domain within the webmaster tools. Even though the verification is seeing a file and says the site is indeed verified. There still is an issue with Google wanting to verify both in order to make any changes to a preferred domain. Makes solving the url canonicalization a bit troublesome unless you add both.

    Solution: You may need to put in both your root domain and the www version by adding both sites within your Google Sitemaps (Webmaster Tools) section.

  224. How do you redirect every non-www url version of my web site to a www one. I mean every single page. I am using Joomla CMS. Is there any way to do that automatically or do I have to do it manually.

  225. I have built a site using WordPress and use the All in one SEO Pluging. It gives an option to provide canonical URLs (I am saying yes, although still not sure why). I understand that it is handy for the index page (like the example you gave) but for a blog where there is only one reference to a post, is it really needed? What does is do? Is it just an indicator to the search engine or does it mean something to how the blog is built?

  226. Thanks for clearing all of this up Matt. I have just been made aware of the canonalization issue on the google help forum and hope I have now cleared it up for my site with 301 redirects (fingers crossed). One issue that many people seem to forget is the internal links pointing to index.html (me included!)

  227. What happens at Removal of a cannonical tag from a page? (In a situiation where
    the tag was added to wrong pages)

  228. Matt

    I have a wp plugin by Yoast fix this form me automatically.

    Hope this helps someone.


  229. Hi Matt!
    Just a question, please (if you have enough time ๐Ÿ˜‰ ):
    I have a domain www. XY .com
    than another 2 domains www. x .com and www. y . com which are better ranked as www. XY .com (and whose names are part of the name in the first URL). If I do a redirection from and to, is it spam?
    If this is not dramatic, is it better to do a DNS or a 301 redirection?
    Thank you in advance

  230. Hi Matt,

    I’m using wordpress in my blog. and my permanent URLs starting with like that.

    Now tell me… That method is correct or not…?
    http:// or http://www... which one is best… ?

  231. very helpful thanks!

  232. hi Matt,

    Seeing as this article is a good 3 years old, I’m wondering if Google has altered the way it treats and

    especially this Q&A in your article:
    Q: I noticed that you donโ€™t do a 301 redirect on your site from the non-www to the www version, Matt. Why not? Are you stupid in the head?
    A: Actually, itโ€™s on purpose. I noticed that several months ago but decided not to change it on my end or ask anyone at Google to fix it. I may add a 301 eventually, but for now itโ€™s a helpful test case.

    did you eventually add the 301? if so, why? and if not why? Or is it still “an interesting test case” ??

    thanks in advance, Dave

  233. Great information. A lot of people don’t realize the impact of a simple redirect.

  234. Hi Matt I made a tool which gets the canonical code for htaccess quickly for webmasters. It works with apache. See what you think?

  235. I have a problem. my index.asp page got cached which is creating duplicacy. How can i redirect it to domain name. like to My site is hosted on windows server.

  236. I’m using WordPress and use the All in one SEO and Redirection Plugin .Will a 301 redirect transfer the pagerank to the new location?

  237. I’ve got a sticky mess with a html site and wordpress blog in the sub directory, the main site is www. but worpress blog is no www. this seems to be confusing my sitemap submission.
    I’m here looking for some advice on htaccess coding.

  238. When I redirected my site from a www to a non-www urls, it had the adverse impact on the links in the Webmaster Tools.

  239. I want an opinion as to whether URL:buycheapdigitalcameras .biz looks like spam to search engines? I’m sure you all know alot more than me and I would love your opinion. Thanks, Steve

  240. Do i need to use url canonicalization on blogspot sites?

  241. Is there any difference between and ? Or is it just visual? Do search engines prefer one over the other?

  242. I’m happy to stick with what ever wordpress plugin sets for my URL, but If I could coose I’d like to run with www.

  243. I had put the canonical tag on my website home page long ago and it used to verify that it’s working fine. However lately when I performed a website health check, it showed that the website might have canonicalization issues. Is there any update suggested by Google, in the way the canonical tag works?

  244. Hi thanks for the great post which seems to still be valid after all this time, my site was PR3 I changed the url and lost my PR after 6 weeks or so it came back as PR1. I then change the home page URL to something my CMS would allow me to have as a unique address. I have put 301 redirects forcing the non www to www, and old pages to the new one but my PR is all lost. I even put the canonical tag on in the hope that Google would take a hint from that but not so far. So now I have no PR do I just have to wait will the algorithm give back the PR? at some point My site is a web design and seo company so the PR is essential for customer confidence. As far as I can tell all URL that can lead to the Home Page are always 301 redirected to the same url. I don’t think there is any thing else I can do to give the hint. Has anybody got any suggestions? Thanks for reading this Sandy. Maybe it a penalty for changing prefered URLs?

  245. Just give the tags some time, they should kick in eventually and the health check will start to notice. Remember you can do canonicalization yourself in addition to the tags.

  246. How about my static pages Matt? Should we also use the Canonical link even if we are sure there will only be one page and no other page leading off from this or with content from this page?

  247. This is an excellent article. I just went through this process on my site this weekend, spending about 5 hours setting up 301’s for loads of old links that google had indexed. I was handling them before but not passing a 301, I have also gone through the website removing all references to default.aspx and so directing to the root to keep it clean. I have also added 301 code for the www problem. I am looking forward to the next majow sweep by google to see what changes this might make to my site and hopefulyl traffic.
    Good effort – thanks matt

  248. Does canonicalization work for mirror sites or domain alias?

    Let’s say I have my website using and having another domain, exactly mirroring the first site via domain alias.

    I am concerned that both of my domains get rated low on page rank for they are having exactly the same contents.

    What are some good solutions to prevent these domains being seen as a duplicate site by search engines?


  249. Thank you for this blog and all the REAL information. There is so much crap out here and I am grateful for having access to Matt’s info. I am a newbie and with this info I am slowly but sure getting there. The content and the comments are educational as URL canonicalization was a new term to me and I am making some corrections to my sites.

  250. yeah, I will just keep my wordpress plugins installed that do this for me. It’s been a while since this post was written so I wonder if much has changed since then? Is using the tag still best practice?

  251. wordpress blogs have a place to specify site address and blog address. Should they both be same (www and www)?

  252. I have this habit of including ‘www’ version in all the links I try to build externally. However, I have now noticed that more pages from ‘non-www’ are indexed by Google rather than ‘www’. So now, what is the solution.Shall I start using ‘non-www’ in future, re-direct ‘www’ to ‘non-www’ or is there any harm keeping it that way. What is the probable reason why Google chooses one version of url for indexing than other.

    Please help. Thank you.

  253. Right now my canonical url is If you try your are 302 redirected to (The 302 redirect is something I just found out about [TODAY!] and has to do with how blogger redirects subdomains. I’ll be fixing it son.) I was thinking of changing the canonical url to Thus, would stop redirecting and would 301 redirect to

    Would this confuse Google? Would I accidentally cause it to loop between the two domains? Is there a penalty? It’s been this way for a very long time which is why I ask.

  254. Hi Matt,

    noticed that since you wrote this post back in 2006, you normalized your domain to the www version, why did you decide to go with the www vs non www version?

  255. Hi Matt

    Is there any other method that can be used for different domains fetching the same content.
    Like I have some domains on UK cities/areas. If certain visitor attraction is same for both areas, same content is fetched under those two domains through DB . Example –

    These above url’s are showing the same content displayed from database.
    Does these pages will be considered as duplicate pages, How could I avoid the consequences of this?

    Do I need to prepare different content for each visitor attraction for each domain?
    Please Advise.

    Vibha Raj

  256. Hey matt,

    Just a quick question, and after you’ve read this you’ll think that it has already been covered before somewhere else on the web-although the following topic has been discussed-it was discussed over 3 years ago. Pardon my sinicism but that’s some time ago and things have moved on a lot since then, more importantly the Google Search Algo.

    My question is URL’s and the benefits of having a file extention added to the end of a URL as opposed to nothing or a trailing slash, e.g. – File Extention – Trailing Slash – Nothing

    Is there any particular benefit to any of these or is it the case that it really doesn’t matter as long as the rule you are using stays uniform across your site?

    Kind Regards,


  257. Thank you for this information, very useful. My website uses the non-www domain through the whole contents consistenly (due to the way the links are built), although someone may still want to access it via www. Is it worth adding a 301 redirect. Also, is there any preference for Google which version (www or non-www) is canonized?

    Many thanks.

  258. Hi matt,
    when a user search for “example”
    will ranks better in search results than
    (Because is shorter and search engines love short urls)


  259. If you change in the future, and cut off www, google penalize you?

  260. What I don’t understand is that why this feature (and loads of other features too) can’t be added on analytics page. It seems kinda silly to have both /webmasters and /analytics. Maybe it’s just me but I would love to have just small “tick boxes” on the analytics pages and be able to tell google how I want my site to be indexed, if I dont want some pages indexed etc.

  261. Hi Matt! Can you please shed some light with regard to base tag vs. canonical tag. Can we use base tag to signal the right url side wide instead of canonical tag although it appears to be made for indicating other things.

  262. thanks a lot for the tips matt.

    so SEOers have two options:

    1. if they want subdomain “www” to be part of their SEO effort, they should set up canonical
    2. if they don’t, just don’t do that

    correct me if i was wrong.

    and i think Google’s Webmasters Tool has provided us a tool to set our preferred URL.

  263. Yes, I didn’t know this was such and old problem, I am just learning about it and educating others. Thanks for the post and insight. I am glad it made its way to 2010.

    FYI, if you use wordpress, and your web host allows .htaccess, its very easy to do. A non www. redirect that is.

    Good Luck, Thanks !

  264. What about using the format I’ve been doing this from the start because my web domain register/host told me that it does something so that the url will appear with or without the www.

    Thesis 1.7 gives you the option to turn Canonicalization on. Should I turn it on?

  265. Alejandro Mery

    Hi, just for the fun of it I wanted to tell you about and … it seems that google team doesn’t know about 301s ๐Ÿ˜‰

  266. My understanding about links is that the links with the “www” are not considered the same as those without the “www”. This means if you have links both ways coming to your site they will not have the strength they would if they were all either with or without the “www”.

  267. This is all highly important stuff when it comes to SEO otherwise you are effectively splitting your ranking. For example, if you haven’t linked to then Google will see this as two different sites and therefore the traffic will be split over the two sites rather than combining them into one. As we know, Google uses page views and visitors as a way of guaging ranking so it is important that you lump everything in together for the best ranking.

  268. Hello, please I have a SEO question regarding the inclusion of ID number in a article. My CMS requires me to enter an ID number in every article, I read many articles and it seems that Google gives less weight to keywords from left to right.
    At this point, it is preferable to use which of these two types of URLs:


    *Following the theory that I wrote above, I would say the second solution. What do you do you suggest me

    Thank you?

  269. Is there any difference using .htm or .html for urls regarding seo? I do not like the directory urls with “/” besides that I need to create tons of directories to actually have urls.

  270. @Steve
    No difference in using .htm or .html.

    I agree, having an extension on actual page links is more intuitive for the user to realize that the link goes to an actual page, and not a directory. On the other side though, if you ever change page extensions, you’ll have your hands full with broken inbound links. Just have to weigh the ups and downs.

  271. An old thread, but a topic that is as up to date as ever. I just wish I’d started as I meant to go on with the use of www on some of my sites. Now I have a mix which I don’t think helps with SEO at all.

  272. Thanks for the advice. I have a question for everyone. Does wordpress have this feature built in? When I type in “” it redirects to Is this what canonicalization is?

  273. when redirecting via 301 www to non-www. should we include the ending trailing slash or not? for example: vs, are they considered 2 different sites?

  274. Hi Matt,

    Great post as ever. I would also like to know whether the trailing / makes difference to the url?

  275. A very useful article (even though I’ve found it 4 years after it was published!). Many thanks.

  276. Hi Matt,

    Thanks for the great post. A lot of great info.

    Do you have any comments on what format is best to pick? Meaning is there a difference for google wether or not WWW is used in the URL when it comes to how long the whole URL string is? ie. is longer then the same string without the www.

    Any difference?


  277. Thank you for this article. I was wondering if there will any issues if I use parked domains. Say main site is and is using as a parked domain. any seo impact?

  278. Hi Matt, as you said you used it as a test case to not 301 to I am curious what came out of this test?

  279. Years ago I was taught that you don’t use WWW. and you do the redirect because it gave you a little boost with SEO.

    If this ever was true, now — as long as there is a redirect does it matter?

  280. Hi Matt,

    In 2006-01-04 you preferred to use trailing slash links better then non trailing slash. Since SEO has been more developed today, do you still prefer to use trailing slash?

  281. Thanks for the tips and pointers matt.

    You’ve been a real help to me the last year or so!

    Thanks again buddy


  282. Matt, i dont see info about canonical url and more domains inside one site. Example: if you have 2 or more parked domains to one website and in code all is well formatted in this way: rel=canonical url=mainsite/page. So, is this practice ok in google?

  283. Is it ok for Google if I use 301 (permanent) redirect to move my index page to a sub directory? Would I be penalized for that?

  284. Great Post Matt.

    I’ve heard there can be an effect on how you are indexed after setting up a 301 redirect. I was working on a website that had www, non www, index, home, welcome, and # as there home pages.

  285. Google recognizes my url with a trailing slash but most of my backlinks do not have the slash. Has these affected my search engine rankings?
    What I am actually asking is if you get different backlinks(some with the slash and some not) will those without a slash go to waste and not be counted to calculate page rank?
    Also can you get a penalty for having different looking url? I heard google might view it as different but identical pages thus a penalty?

  286. Thanks for the clarification on url canonicalization. I second Martin’s question about the trailing slash.

    So if a site has a lot more links to the www. than the version, is it better to go with the www. then?


  287. Thanks for the info, I’d been wondering abou the implications of canonical url’s for some time now.

  288. Hi Matt ,
    very Good Morning. Thanks for your so many suggestions available on NET.
    I was going trough your presentation about canonicalization.

    I am little bit confused some where. Please advice me what to do.
    and so other varaints…

    In each and every case, index.html page is executed.

    say. is my canonical page. Other variants are non โ€“canonicals.

    As per you adding a element with the attribute rel=”canonical” to the section of the non-canonical version of the page will tell Search Engine that this page is most priority page among the identicals.

    So I put in /index.html page.

    My question is : What happen when the correct version of the page i.e. is executed. Is there any side effect / harm ? Will Search engine find canonical page with in the canonical page.. causing loop etc.?


  289. Hi Matt,
    Lovinโ€™ your style
    Truly nice and well written article.
    The ideas are strongly pointed out and clearly emphasized.

  290. Though I have heard about canonicalization before, it was not defined so clearly as you have done here. Thanks for that. Also for 301 redirect idea.

  291. i still confuse, which one is the best for SEO ?

    right now i still use

  292. That’s great! I’ve been having problems with duplicate content, struggling, but this solves my problem. Thanks

  293. I had passed you blog on this issue to my web master but I do not think she really got it since I still have my site showing up both with www and without it. Is there an easy explanation to setting up a 301 redirect?

  294. Hi Matt,

    This is a bump for the question about http and https. When I do a site search on Google using either, it returns just the http version.

    Also – where can we submit questions for you to answer on your YouTube channel?


  295. Matt,

    A developer broke our canonical url link so that all product pages pointed to the same duff url. This resulted in all product pages being removed from the google index. We have (about to) fix this. However, my comments and questions are not just about our site but everyone’s (Hence I write here instead of webmaster forum).

    1. Google should know that if the canonical url is a non existing url it should ignore it?
    2. There was no feedback to this issue in webmaster tools so we were left wondering what had gone wrong. I suggest that canonical url issues are included in future releases. This could include also, if a user specifies diff formats such as www or without www.
    3. I couldn’t find anything useful which does not does not say, “Once you fix your canonical url”, indexeding will incur within x days and rankings will be back to normal.

    Anyhow, some feedback. It’s great to have the tag but I would have hoped the google bots would have been a bit more clever about this rather than removing us from the index completely for product pages. Any information on the above would be great, it feels like it may be safer to just not use canonical urls at all because now we have lost all rankings.

  296. Thank you for the useful notes, owing to this article my web page is now error free.

  297. Hello,

    Matt, Could You confirm “Iโ€™ll have to check to make sure that 410 (permanently gone) is handled correctly.”?

    Seems Google is checking urls that don’t exists… I see about 3000 hits per hour comming from google bot – every page does not exist. How to say google that that pages have gone? Can I use robots.txt ? It’s killing my website performance. I saw about about 50 000 not existing pages scaned…

    examples: – – [01/Jul/2011:22:07:34 +0200] “GET /……” 404 20885 “” “Mozilla/5.0 (compatible; Googlebot/2.1; +” – – [01/Jul/2011:22:07:41 +0200] “GET /…” 404 20840 “” “Mozilla/5.0 (compatible; Googlebot/2.1; +”

  298. Sir,

    My website is displaying in search result as instead of What should I do? I’ve already asked this question to Google sites and Google webmasters help forum, top contributors and other help forums but I’ve not found answer for this question. Can you please help me in this matter?

    My website is completely educational and I’ve written all contents my self for students.

    Your comment will really helpful for me. Please also visit my website and give me suggestions (if any).

  299. Setting up proper 301 redirects is so crucial. I came across a client that hired someone who was trying out 301 redirects. I suggest contacting a specialist if you haven’t had any experience with this before. You can prevent pages not only from being indexed but seen entirely.

  300. To answer the question about the weird word “Canonical”: It comes from the greek word “Kanonika” (Original word: ฮšฮฑฮฝฮฟฮฝฮนฮบฮฌ, but printed in latin characters here so that everyone can read) which means Normal, or Proper

  301. Thanks for the clarification!

  302. Hi, very good information thanks, but I have two questions:

    1) What about pages that contains internal links to the same page like, the next would be duplicated content?:


    2) 301 Redirect? is not better using rel=”canonical” (less server load)?

    Thanks in advance

  303. Hey Matt,
    I wanted to ask you about the fall of traffic due to canonicalisation is possible? as Since 3-4 days my analytics are severely down ie., fall of more then 70% If you can guide me what remedy should i do on my blog then would be great.

  304. I want to stop 2 months old 301 redirection domain and i want to get back to my Old domain.
    Can you please give your valuable suggestions . Thanks you.

  305. my blog posts are not indexed by Google, but now i know the reason I will try to resolve it. thanks!

  306. Hey Matt,

    My website was earlier without www but I had added it to the URL later rectifying it.
    Now my question is after I changed it do I have to again start off with my SEO activities with the fresh URL or earlier ones will also one counted for ranking. My ranking has gone down a lot since I changed it.
    Please suggest.

  307. It is amazing how this subject is still valid after 5 years of posting!
    Now, you can chose your preferred domain from google webmasters tools.

    Thanks for the good information!

  308. I have recently installed SSL certificates on my server, and am now running also https version of my website. I had many doubts and interests whether shall i have duplicate content or not, and how the SE will deal with this.. but as far as i see, this is not really a big issue regarding duplicate content (http vs https). Do you agree?


  309. Hi Matt,

    Do you recommend as a best practice for an ecommerce site that has NO reason for indexing HTTPS pages to ALWAYS canonical the HTTP page.

    Thanks so much!

  310. I feel that a general issue with search engines is that they want the web to be organized so that it can be displayed in their search results. It requires that we avoid duplication of content over the Internet and this is impossible and not natural. Sure, we want to avoid duplication of content in a given channel such as a search engine, but duplication of content among different channels is very natural. No one should be responsible to avoid duplicating content from other sources, except when there is a copyright violation. Often, authors are happy that we duplicate their work, some times without even expecting an attribution. It is the job of Google to avoid duplication of content in their search results and it is the job of webmasters to avoid duplication within their respective site, but that is it – it should not go beyond that. For example, many different newspapers can reproduce the same article. What is wrong with that? I buy one newspaper from a publisher that I trust and I don’t mind that the same content is duplicated by other publishers.

    This issue is particularly interesting in the case of a piece of information that must be duplicated with a different contact person each time. This happens all the times. Brochures often have a space reserved at the end so that a different local phone number, etc. can be added at each location. Outside the context of search engines, this is not a problem. However, it becomes very problematic if you want the same brochure to be distributed over the internet in a search engine friendly way, which is necessary if it is to be visible.

    I have discussed this issue here Any comment will be appreciated.

  311. Hi Matt,

    Even though opening a website with either,,, or all opens the website. But still I personally believe that any website must be always redirected to And it surely does better as compared to other url formats. It helps in achieving the better ranking too.

  312. When i did the redirect
    it gave me the error that security failure and website too is not opening up

  313. Hello Matt,

    Question : Which Is Best ” www ” Or ” non-www ” ?

    Answer : ” www ” Is Best.

    Reason : As Matt Is Using ” www ” For His Website. ” www ” Is Best. ๐Ÿ˜€

  314. Great blog! Have decided to redesign my website to something more modern and more effective and looking into redirects and all that. Thank you for making this confusing area seem so clear and understandable! Very useful information, thanks!

  315. Hi Matt, i realized your site have canonical issues itself LOL

  316. yah my website too.. it’s been 3 months now, but i still have no page rank. ๐Ÿ™

  317. Hi Matt, many thanks for the explanation, I’ve just come across this term whilst trying to optimize my website, and I had no idea what it meant! Thanks again.


  318. Sweet! Very interesting resource. I am very passionate about SEO stuff. I will sure come back for more interesting stuff.. ๐Ÿ˜€ Keep up the good work!

  319. Nice article. I added redict to www today.

  320. its advisable that website should convert from non www to www.

  321. Does canonization help your SEO?

    does it move you up in the ranking?

  322. Damn..after so many days I figuredout the way to fix this in Google Webmaster tools by adding code to www and non www and redirecting the non-www to www. Am I done fixing it? just wondering if there is anything else that has to be done? Thanks!

  323. Hi Matt-
    Great post. However, I have curve ball question for you (I think):

    I’m analyzing a client site’s spam backlinks to submit using Google’s disavowal tool. When I review their backlinks using (as well as Webmasters) I get two different scenarios:

    The www has over 140,000 backlinks (ouch for me, I know).

    The non www has about 8000.

    Right now we’re redirecting www to the non. However, from a disavowal perspective, should I be concerned about the 140k on the www once we remove the redirect to the non www?

    Personal note: I realize you probably take truckloads of heat from SEOs since the launch of Penguin. Honestly, it was the perfect dose of medicine for my business model. I’ve always tried to be 100% Google compliant with my client sites. Relevancy is what I preach to them.

    Keep up the great work. I think your involvement doing videos about issues related to search are dynamic.

    Take care Matt.


  324. Great post. One question though: does wordpress (as a cms) automaticly do this canonicalization? So i heard from a friend. Or no matter what cms you use you still have to let google now wich url you prefeer. I am a newbie so that’s why i am asking this!

  325. Thanks Matt for the great article that I’ve been rereading for years…comments, too. I’ve learned it’s always better/easier to start off keeping canonicalization uniform when building the site. It’s really a headache trying to fix linking to multiples. I know there are different opinions on this page BUT I always go with and send links to Trailing slashes, too.

  326. Hi,

    I tried for url canonicalization for my site and i have succeeded. But while opening my sitemap.xml it throws error like Error loading stylesheet: An unknown error has occurred (805303f4)

    Anyone please help me to resolve.

  327. But according to the google page speed we should not make a 301 redirect. Because it consumes more time to show the website. Because someone will call it than it will redirect to your www version and it took alteast more 1 sec than the normal time.

  328. Thanks matt
    Very informative educating post

  329. Thank you