(I’m taking my wife somewhere really soon, so I’m just going to dash out a quick post.)
There was a Techmeme discussion this weekend about whether Microsoft should chase Google in search or find their own “Big Hairy Audacious Goal.” Into that discussion came a post by Ryan Stewart about being removed from Google’s index. It turns out that Ryan’s blog had been hacked, and Google does remove hacked sites from our index to protect our users. I left a comment at Ryan’s blog, but while I wait for it to be approved I thought that I’d post it here as well:
Hi Ryan, my name is Matt Cutts and I’m a software engineer at Google. Sorry to hear that your blog got hacked. I know that it’s disappointing if you don’t show up in Google, but there’s another way to look at it. It looks like your blog was hacked to show “buy pharmacy”-type links, but what if the hackers had hosted malware on your site? Then every user to your site might have gotten infected just by visiting your site. That danger to Google users is one of the reasons that we temporarily remove hacked sites from Google.
I’m glad that things look clean now and I’ve revoked the “hacked site” flag for your domain. I’d expect your domain to return to Google within 48 hours, if not sooner.
By the way, we did try to contact you. We sent an email to contact [at] digitalbackcountry.com, info [at] digitalbackcountry.com, support [at] digitalbackcountry.com, webmaster [at] digitalbackcountry.com, and a gmail.com address on May 19th at 21:25:23 with a subject line of “Removal from Google’s index.” I believe if you had logged into our webmaster console at google.com/webmasters and proved that you owned digitalbackcountry.com, we also would have left a message waiting for you there as well. That webmaster console is the primary way to request reconsideration in case your blog has been hacked.
We do try to communicate with hacked blogs where we can, and we also do blog posts to try to help prevent hacked sites and for site owners to recover from hacked sites. Some example posts that we’ve done in the past:
http://googlewebmastercentral.blogspot.com/2007/09/quick-security-checklist-for-webmasters.html
http://googlewebmastercentral.blogspot.com/2008/04/my-sites-been-hacked-now-what.html
https://www.mattcutts.com/blog/how-google-handles-malware-a-historical-overview/The only last point I’d make is that users tell us loud and clear that they don’t want to be sent to hacked sites, because of the potential danger that they represent. Even though it’s stressful to be removed from Google, I hope you understand why Google might not want to send users to a hacked blog.
Again, thanks for cleaning up your site and you should return to Google’s index soon.
How Google should handle hacked sites is a tough question, but personally I think Google does a better job than other search engines of protecting our users and communicating with site owners about hacked sites. For example, here is an excerpt of the email that we sent to Ryan on May 19th:
Dear site owner or webmaster of blog.digitalbackcountry.com,
While we were indexing your webpages, we detected that some of your pages were using techniques that are outside our quality guidelines, which can be found here: http://www.google.com/webmasters/guidelines.html. This appears to be because your site has been modified by a third party. Typically, the offending party gains access to an insecure directory that has open permissions. Many times, they will upload files or modify existing ones, which then show up as spam in our index.
The following is some example hidden text we found at blog.digitalbackcountry.com:
Acyclovir Adderall Adipex Alprazolam Ambien Ativan Biaxin Bontril Bupropion Butalbital Carisoprodol Celexa Cheap Phentermine Cialis Online Cialis Cipro Clonazepam Codeine Darvocet Diazepam Didrex Diflucan Effexor Ephedrine Fioricet Flexeril Generic Viagra Glucophage Hydrocodone Online Hydrocodone Levitra Lexapro Line Xanax Lipitor Lorazepam Lortab Meridia Nexium Norco Viagra Tramadol Soma Phentermine Valium Norvasc Buy Acyclovir Buy Adderall Buy Adipex Buy Alprazolam Buy Ambien Buy Ativan Buy Biaxin Buy Bontril Buy Bupropion Buy Butalbital Buy Carisoprodol Buy Celexa Buy Cheap Phentermine Buy Cialis Online Buy Cialis Buy Cipro Buy Clonazepam Buy Codeine Buy Com Lvivhost Online Viagra Buy Darvocet Buy Diazepam Buy Didrex Buy Diflucan Buy Effexor Buy Ephedrine Buy Fioricet Buy Flexeril Buy Generic Viagra Buy Glucophage Buy Hydrocodone Online Buy Hydrocodone Buy Levitra Buy Lexapro Buy Line Xanax Buy Lipitor Buy Lorazepam Buy Lortab Buy Meridia Buy Nexium Buy Norco Buy Norvasc Buy Online Xanax Buy Oxycontin Buy Paxil Buy Percocet Buy Phentermine Online Buy Phentermine Buy Propecia Buy Provigil Buy Prozac Buy Renova Buy Seroquel Buy Soma Buy Tadalafil Buy Tamiflu
[…]
In order to preserve the quality of our search engine, we have temporarily removed some of your webpages from our search results.
(The rest of the email goes on describe how long the blog will be out of Google, and where to go in order to get back into Google’s index faster.)
Getting hacked is not fun. It’s just not. But I think Google does the right thing for our users by removing hacked sites from our index temporarily. I also think we do a pretty good job of trying to alert site owners that they’ve been hacked — more than any other search engine does. We alert many webmasters about hacked sites not only via email but also with our webmaster console.
Do I want more competition in search? Absolutely, because it keeps everyone on their toes and working hard for our users. But I think Ryan’s specific situation actually shows that Google is trying to do the right thing for site owners and users. Ryan, I hope there’s no hard feelings that your site was removed from our index after being hacked, and now that it’s clean you should be back soon.
From experience I can say the turn around time for a site which was de-indexed or blacklisted within google using the re-inclusion request is very good. As long as the turnaround time remains good I think its a good service.
Matt, just today I was doing some research on hacked sites (seeing how many were still using outdated WP installs), and I noticed that it looked like less than half of the ones that were showing the footprint for that intrusion had the “This site may harm your computer.” flag on them.
What would cause multiple sites with the same infection to have some with the flag and some without?
As a follow-up to what Shoemoney said, lat week I noticed that one of my blogs had been hacked again, and had the flag in G. I fixed it and submitted the reconsideration request, it had the flag removed by the next day. The site was never deindexed, and had already been added to GWT, so I’m not sure if that had any impact on the turnaround times.
It would be nice if there was an automatic notification to our linked Gmail account whenever a message appears in the Google Webmaster Tools message center. Most webmasters would probably notice email more quickly, especially if they have a Gmail preview widget on an iGoogle personalized homepage.
Michael VanDeMar, that warning is for sites that deliver malware, and not every hacked site serves up malware. We also don’t have 100% recall.
Outstanding and reassuring. I think G is going above he call of duty in many ways here. It also reminds us to keep up-to-date on patches and notifications about our platforms. I’m amazed at times to see how lazy site owners are about doing this – not wanting to pay their webmaster for the hour it takes is absolutely ridiculous.
Nice work. It’s nice to see the email efforts, so as not to “force” subscription to webmaster console.
I’m impressed at all the measures that are taken to notify a website owner…great work!
Its always hard to decide where to draw the line. Of course we’re not talking about getting listed in the local phone book and having someone cut your ad out of all the books just before they get distributed. Over all I think Google is handling spam well but I question why a site with spammy links would disappear entirely instead of the site(s) they were spamming…Of course there’s no way to keep everyone in this equation happy unless we can eliminate spammers entirely…I say we implement the death penalty 😉
I appreciate this post. This shows the “softer side” of Google (or maybe just Matt). It’s good to see Google protecting users as well as webmasters. It also gives me a new perspective on the benefits of the webmaster tools.
What a good idea to take your wife out as well. Smart guy!
Thanks; Mike
Wow, I have just recently upgraded to the latest version of WP.. i think its 2.5, not sure. Is this version a lot safer, than previous? This is the first article i’ve read on blog hacking, kinda scary…
Matt – I know that, I was saying they were all hit with the same hack, so the likelihood is that they all contained the same malware. I know my sites did when hit with it. Just do an [inurl:wp-content/1/] search to see what I mean (not all of those are hacked sites, but most are, and there are hardly any of them left now).
Could it be that if the pages that contain the malware (since only a few of the affected pages on any given site do) don’t have enough links pointing to them (ie. aren’t important enough) that the flag won’t trigger? In other words, is the flag only for the main index?
Hey Matt,
Thanks for the comment and the followup post. While it may not have some out as clearly as I wanted in the post, I fully agree that Google should take actions like it did to keep it’s search results clean as well as remove incentive for hackers. So I’m definitely not upset with Google for removing me. It was a couple of main things that caught me off guard.
1) No kind of warning. Google can obviously very quickly (and I assume algorithmically) detect spam, so some kind of heads up would have been great. I think I just assumed – to my detriment – that having a legit, well-intentioned blog would have triggered something in Google to send a warning message before the “you’ve been removed” message.
2) How much I rely on Google and how much they can leverage that. I’m a capitalist, so I’m not blaming Google for having the best search results and getting the most marketshare – that’s great. It was just a shock to realize that I’d be willing to do basically anything and give any kind of information up to make sure I’m included in Google’s index.
As far as I’m concerned Google hasn’t abused that power and I think the nature of the web would make it tough to do so. But having it taken away was a big wake up call that to me, as a webmaster, the other search engines don’t matter. Whatever Google wants from me, they’ll get :).
So far the re-instatement process has been fine. Clear communication now that I’ve associated my URL with my Google account and hopefully it’ll be resolved quickly.
Thanks again for dropping me a note and providing some feedback.
Matt, I applaud your efforts and Google’s for providing a response to folks who have been hacked. Hacking is a nightmare for anyone affected. However, this is not the only issue we are dealing with on a daily basis. Scrapping and bot bombimg has become a nightmare for some of us too. The worst part is that the scrapers’ sites are delisting site’s where the content was originated. They can do that as simple as grabbing your metas and/or title pages combined with other techniques. And Google is not responding promptly to the manipulation of their rankings, or providing information on how to deal with that in a timely manner. Some cases are reported with evidences, but seems like Googlers are pretty tide up to deal with each report/complaint. By the time we gather evidences and make the report, the damage is done and the site dropped from SERP. You can only think the impact this has for small sites.
The need to implement a reporting system through Webmaster Tool to help us with those issues has been discussed and requested in several forums. So please help us voice our concerns with your team.
Personally I spend more time fighting spam, scrapping and possible hacking that promoting my products. In fact, all this week I have been dealing with a potential proxy hijacking (or 301 hijack or massive scrapping, who know exactly what it is) in spite of IncrediBill’s recent report that proxy hijacking appears to be an issue of the past with Google. It’s confusing sometimes to figure out what we really are against: 301/302 hijacking, proxy hijacking, scrapping, etc And yes, the bots are blocked in .htaccess and other hacks are implemented. But tweaks are never enough.
I think personally signed engraved invitations should be sent.
They are just as likely to get ignored as any.
Some time ago, John Mueller personally sent out thousands of notices to sites he had found that had been hacked that he had found both before he joined Google and also, and especially so, on his own time.
The “results” he got back were bad enough to make one cry. I would be willing to bet more than 95% of them are still spewing malware and/or hosting spam links or even entire spam domains.
If a person doesn’t follow Internet standards and support and reply to e-mail sent to addresses as required, let ’em rot.
Will sending people who are more than likely to not read or take a notification seriously accomplishing much or just waste resources that could be better spent elsewhere.
More often than not, as is evident in this case, the “webmaster” seems to think that ignorance is an excuse to rant.
I don’t know about most people but I don’t think I would be publishing the fact that I don’t know what I’m doing, although most of you here probably already know. 😉
It seems the new rule of the Blogosphere is “Rant first, learn later.” 🙁
@Craig,
I did read the notices. I didn’t see them until later because I don’t check my Gmail account very often. But it wasn’t a notice saying “we’ve detected your site was hacked, you should fix that” it was “your site was hacked, we’ve removed you from our index”.
It’s really not so much of a rant – I did break the rules – as it was a warning to others. And it also made me realize that I’m pretty much willing to do whatever Google wants so I can be included their index. That just made me stop and think.
=Ryan
rstewart@adobe.com
Michael VanDeMar, I’m not as familiar with the exact heuristics for flagging stuff, but the fact that we’re pretty careful about what we malware, plus the need to have a certain level of confidence, means that we won’t catch/label every piece of malware out there (probably even when the hacks are nearly identical).
Craig, I completely understand and sympathize with what Ryan wrote. From my perspective, Google handled the situation pretty well according to the process that we strive for. But for him, it was still scary and utterly foreign. There are still tons of webmasters who haven’t registered to receive messages in our webmaster console (and plenty of webmasters who haven’t even been to google.com/webmasters ). It’s tough because Google wants to help the sites that we detect, but by default the web isn’t really built to allow large-scale contacting of webmasters. We’ll keep trying to think of ways to do better though.
It may be wrong to remove sites from the index if their hacks do not embed links to malware sites.
If the body text was still the same, the fact that there was hidden text and hidden links really would not have gotten the site high enough on the organic SERPs to really be visited for those hidden keywords.
Also, why not just remove the individual page that are hacked – but not the whole site consisting of many relevant pages that were left unhacked? Those pages still are providing a service to the readership and potential searchers.
Another option would be to warn users on the SERPs that a page is suspected of being hacked – perhaps by putting the word ‘hacked’ in red next to the url or title of that page.
This compromise would not only be more fair to the innocent Webmaster – but allows the user to make a choice to get the info on the site that is still valuable to them. 😀
The only exception would be a totally destroyed site that redirects or links to malware or trojans – they should be banned.
Matt,
May I suggest adding a point to the current Webmaster Guidelines explaining that Google does remove hacked sites from its index to protect its users. Webmasters who have registered in Google’s webmaster console (google.com/webmasters ) would be receiving alert messages.
I like Ryan’s point that it was disturbing to realize how much he needs Google. I think it’s disturbing for any webmaster who relies on Google search traffic for a large portion of their incoming visitors.
As much as we try to use white-hat SEO, diversify our sources of visitors, etc., Google search remains a huge portion of the visitor mix for a lot of sites. It’s a bit disconcerting to accept that forces beyond our control can change how many people Google sends to our sites in a flash, like turning the tap on or off.
And whether Google is acting reasonably or not, when that tap turns off… it’s hard.
This is excellent work, Matt. I had two WordPress websites hacked at the end of January and spotted them rapidly before this had a chance to cause them to be de-indexed. I did inform Google just in case. Nevertheless in the first 24 hours the pirate code had spewed out many thousands of false pages on my domains that gave links to pharmaceutical websites. Unfortunately these were ranked highly given the high ranking of the domains that had been used. I’m sure the hacker made considerable sums of money on these false pages.
The type of process that you describe, Matt, would hopefully reduce the economic advantage of such hacking. It won’t go away but at least it can be made less attractive for the hackers.
George Kirikos, personally I think that’s a great idea.
Julian, I would upgrade from WordPress 2.5 to WordPress 2.5.1 if you can.
Miss Universe, often we do remove just the hacked section of a domain, not the entire domain.
Ryan Stewart, as soon as we detect that a site is hacked, we want to take action pretty quickly. If you gave a webmaster (say) five days to correct their hacked site, that’s five days that the site could be infecting users, for example. That said, I’m open to suggestions about things we could do better. I would like it if the message center in the webmaster console could send emails, for example. Another possibility might be not to remove the site from Google’s index entirely, but to show an interstitial warning “This site may have been hacked.” That would let intrepid users still get to the site, and we could put messages on the interstitial to point the site owner in the right direction. The process that we have in place now is (in my opinion) better than other search engines, but I’d still like us to make things even easier/better for site owners.
Hi Matt,
Thank you for this tools. We have been victim of a huge hacker attack for 3 days last week that showed how we were vulnerable.
The advices “check your logs” and “check your website for common vulnerabilities” are very good… here is how it works:
1. The hackers tried billions of passwords to access our database server (the log files are inflating dramatically)
2. The hackers used SQL injection to add dumb content and destroy our database
3. Our website was completely down
We took measures against it but for 3 days there was no business at all, it was really scary.
I wish I could say that I received an email from Google saying my website was attacked but it was not the case (Google apparently send an email if a website content is replaced by spammy/dumb one but not if a website is down)
Matt, how does Google know a site has been “hacked” as apposed to the owner simply serving up spam?
I must say, since the Advent of Bloggs, hackers seem to target them over any other site type.
Dave,
I doubt Google care the motivation of a site serving malware, whether it is the owners or someone else’s malware is not a relevant concern in not directing folk to that site.
Matt,
I’ve cynically commented in the past that Google will hide sites that might expose me to nudity by default – and labels this a “safe search”, but puts a tiny warning on only a small proportion of search results sites that will attempt to infect my computer with malware. If nothing else this is an abuse of the word “safe”. I don’t really mind the default “prudish” search, since it hides a lot of mediocre dross, but could we are least give it a sensible label?
News that Google drop pages that serve or link to malware is welcome. I for one wouldn’t mind a more aggressive stance on this from Google. Although I appreciate it isn’t Google’s job to police the net.
I’m intrigued though how effective this is. Did Google attempt to mitigate exploits from the recent automated SQL injection attacks? As many pages still exist in the index with evidence of SQL injection.
I guess there is a risk of accidentally removing pages from people discussing online security otherwise.
Did the SQL injection trigger any sort of spam warnings at Google, because it must look like many online advertising systems.
http://www.shadowserver.org/wiki/pmwiki.php?n=Calendar.20080514
I doubt they care either, but that wasn’t my question. Also, not all hacked sites have malware, some are simply hacked to deface, or add SE spam.
Is there ANY spam filter on the planet that would have let through that email from Google? =D
By removing the “example hidden text” part it would at least had a chance.
Google handle hacked sites perfect in my opinion, they could just remove from index as webspam and thats it, back to square one.
Google send you an email, then its up to you to sort it out and secure it, As long as it dont take weeks to get back indexed then i’d say google handle this perfectly
@Ryan, I understand but when your culpability in what happened only gets little more than a sentence in your entire article, it seems more like link-bait riding on the “Google is too powerful ergo evil” wave than anything else.
It is ironic though that some of the malware of some hacked sites targeted Windows platforms to harvest log-ins so if Microsoft doesn’t follow suit to the level of others and remove hacked sites from their indexes, they’ll be providing hackers not only with the best “tools” to use to hack but one of the best malware distribution systems available as well.
So, if your wish for Microsoft to improve their search to compete with Google includes hoping that they will step up, that’s fine, otherwise Search Live will be helping the infestation that StopBadWare, Google and others are trying to prevent.
Then, who will be blamed?
Good post Matt. I know Ryan and I been seeing a lot of activity recently over his blog. For one reason his blog is PR 9 because he has incoming links from Adobe where he works I guess. Recently some noobs around net has found that his blog has no follow. And is a quality PR 9. I remember his traffic was very low looking at Alexa and him telling me. But recently there was a spike in his traffic because some losers around net probably told others or sold his blog url to other people.
That being said, I think that is one of the reasons he got hacked. This pagerank thing makes people evil lol. Anyways I think google does good in dindexing a hacked site but i doubt he gets loads of SE traffic most of his traffic is direct. But google has to protect others has u mentioned about the malware issue..
I just logged into my webmaster console, and saw that one of my sites was no longer “verified.” It turns out that the “Google-Sitemaps/1.0” bot tripped one of my anti-bot pages and got itself blocked from that site. My anti-bot page is listed in my robots.txt. I also just added a nofollow to the link to the trap.
#1. Can the Google Sitemap spider /webmaster console system also send us an email message when our “verified” status changes?
#2. Why is the IP address not resolving to something useful? 74.125.16.35 doesn’t resolve to anything at all.
#3. Why is the GoogleBot using a Firefox and IE user agent? To check for bad webmaster practices I assume.
Maybe the Site Verifier should not use the same crawl servers (IP space) as the regular GoogleBot uses.
Wouldn’t including the hidden text in the notification email trigger all major spam filters, thus not reaching the actual inbox of the receiver?
This is one of those double edged swords, been there myself in april, and thanking god I knew what to do to fix it. The thing that comes immediately to attention though are the folks who don’t know how to fix the hacking malware issue. What can we do to help them out? Not everyone would know how to find, fix, and get back into any search engines good graces.
Does a “ban”/reapproval of a site because of spam or similar have any impact on a site’s Google ranking? Like lowering page rank gained from the time a domain has been in the index or similar?
and then there is the whole chinese electronic store hack that is getting into gmail accounts, and sending out letters from the address book…
Matt a very detailed response to be banging out on a Sunday over Memorial Day weekend…. you better have taken the wife somewhere really nice.
Enjoy the rest of the holiday with family and friends.
This may be slightly off topic but I think related.
I have a few home sites I run out of my closet (http://drumgit.com, http://drumalicious.com, etc.) that are based on wordpress. I’ve been a bit behind the curve on keeping them up to date and as a consequence only noticed last night all my google adsense on drumalicious.com was referring to drugs and drug sites.
Apparently many (1000’s) of sites got hit by this hack and it appears to be a mod of the CSS of the theme templates.
I actually upgraded everything last night (base packages, plugins, vanilla themese, etc) and reapplied the adsense code, and the drug ads are still showing up.
I’m thinking there’s has to be a way google can spot this, if the adsense of a site changes drastically (from drumming to drugs) then it’s likely it’s been hacked. It would be awesome if google could then send a friendly email to the adsense subscriber about it.
>Another possibility might be not to remove the site from Google’s index entirely, but to show an interstitial warning “This site may have been hacked.”
But surely it would encourage the hijackers, if they continue to gain from their exploits? As I see it, it benefits webmasters and searchers alike if Google cracks down as fast as possible on hijackers. If hijacking doesn’t pay that should at least discourage some types of it.
Matt,
this year my site was hacked and removed from Google index. I never received a letter. I’v emailed Google and never got an answer back.
I fixed my website and after one week I got back to index. No one here is complaining from being removed. In my case If I knew the reason I was removed from the beginning, I would be much more comfortable with the situation.
I love Google Search and competition will make internet even stronger.
Stay in google guidelines, use their tools, and got no problems with them. 🙂
Microsoft are working on money, profit, google in innovation and vision? how could they beat Google then?
Can you imagine walmart, shell oil, or pilsbury having a discussion with an individual customer about the problems they were having at the store.
This is why I like Goog, it is a small neighborhood store where we can all sit around with the owner on Sunday, talking shop and chewing on beef jerky or a pickle.
By the way Matt, when are you going to stock Orangina?
DK
AussieWebmaster, we had a nice meal together, but I should still do something nice for her tonight. I was going to work on a blog post, but maybe I’ll try to do something romantic instead.
Jean, I agree that we do need to take action quickly when we discover a hacked site. But I’d also like to keep an open mind about ways to guide site owners on how to correct the hacked site, especially given how stressful a hacked site is.
purposeinc, if we stocked Orangina in my nearest micro-kitchen, I’d have to exercise for an extra 10 minutes every morning. That stuff is good. 🙂
Matt,
Our good friend Barry has posted a relevant article Google’s Safe Browsing Diagnostic Tool
Matt what mailaddreses do you use to send warnings to is there a standard set Google uses eg suport@ webmaster@ or do you use mail addresses in the DNS ?
If there is i can set up procedure to make sure thease addresses are monitored.
Link droppers. Try post without dropping a link.
Google has a symbiotic relationship with us. It looks just as bad on Google to not have a good site in its index.
Think of https://www.cia.gov/ getting hacked. Google may remove it. How bad would it look on Google if someone did a site search on cia.gov? Would the average non-seo user think “um, cia.gov messed up..” or would they think “Google is not complete…”? I think they would blame Google.
http://www.networkworld.com/community/node/27093
Sounds good Matt but any official Google mailings seem to go directly into my spam folder in Gmail. I don’t think it’s just me? (something to do with missing SPF records on the sending domain?)
You also get a large red warning telling you not to trust it 🙂
[quote]…any official Google mailings seem to go directly into my spam folder in Gmail. I don’t think it’s just me? (something to do with missing SPF records on the sending domain?)”[/quote]
I was trying to figure out why mailings from one of my sites was always going to the SPAM folder of gmail accounts.
So I e-mailed Google from my gmail account. Their reply got sorted into my SPAM folder. 🙂
Maurice, I think we go for a selection of often-used addresses like webmaster@, postmaster@, etc. I think in the case that I blogged about, we also went the extra mile and scanned the site itself to find a custom email address and sent a message there, too.
Hey Matt I have a quick question about something you said…
“We sent an email to contact [at] digitalbackcountry.com, info [at] digitalbackcountry.com, support [at] digitalbackcountry.com, webmaster [at] digitalbackcountry.com, and a gmail.com address on May 19th at 21:25:23”
Does this mean by default you email those generic email addresses? So anyone who has a website should have at least one of those generic email addresses?
Thanks
“I must say, since the Advent of Bloggs, hackers seem to target them over any other site type.”
Unfortunately, blogs are usually easy targets… It’s one of the problems with open source being the dominant medium for them (WordPress, Drupal, etc.), and with most bloggers being part-timers and/or amateurs at best.
People often don’t upgrade once vulnerabilities are discovered and exploited (and the hackers are happy to reverse-engineer the source code of every version of these platforms that is released — searching for the chinks in the armor).
This is a good move from Google, even better that they attempt to inform the webmaster, where they could otherwise be oblivious. I’ve seen examples where hackers have hidden phishing sites on servers without the webmasters even knowing about it for months, choosing to stay under cover.
Maurice, go for accepting relevant RFC 2142 addresses, then everyone else should be able to contact you as well as Google.
http://www.ietf.org/rfc/rfc2142.txt
Matt,
Just for clarification about the hacked blogs – I’m assuming you’re referring to blogs on Web sites that are using software like WordPress. How can we be sure our myblogname.blogspot.com or ditto.wordpress.com is secure from hacking (besides moderating the comment box)?
Thanks!
/offtopic from hacked sites
Hey Matt,
Kathy here again from Hystersisters. I’m concerned because once again my site:www. results are leaking. Once I got some things situated based on our last exchange the numbers went up and I did the happy dance. I’ve gone on about my business as a webmaster, handling the support needs of the members and staff on our site. Recently (in May) I noticed that traffic was down a bit which is not normal at this time of year. Checking stats further I see that the number of pages indexed in GOogle has shrunk down. Thoughts? Thanks for your help!
Kathy
Does any permanent penalty or warning flag get attached to a domain that has been hacked in the past? Or does once you clear up the hacking does the site retain its original strength with Google?
Google rank page, not sites. As Google SERPS are in perpetual ever-flux, only those pages with a higher than average REAL PR, would retain their original position. Basically, most pages on the Web fluctuate up/down in the SERPS constantly.
I am willing to bet that most people that get hacked don’t even know it until it is too late. Staying ahead and taking the extra precautions is worthwhile.
No matter what you do your site is not invincible. The best action is just to make it as difficult as possible.
Competition makes the world go round.
I’m so happy my blog didnt get removed from googles index when it got hacked (and it even stayed hacked for about 1 month). I seem to have lost quite a bit of pagerank thou, but whatever, as long as my blog can be found through google.
Thumbs up to google for trying to contact owners of hacked sites before removal. I seriously didnt think they would do that.
Matt,
Very interesting post, it shows the caring side of G, which is really nice!
I also think its great that you are trying more ways of contacting webmasters/owners rather than just through Webmasters Tools.
I have had a site hacked before, the way I found out about it was an email from my hosting company – good quality customer service.
Hi Matt,
I know it’s rather improbable. But what if a penalized site comes back with the same top-positions as it was b4 the penalization – Is there a possibility to manipulate google’s robot and thus re-consider the site regaining the top-position without google’s cognizance?
Pls don’t laugh about this question , I just want to know your opinion about that becasue:
w w w . c o c h e s – b e l g i c a . c o m
was using constantly illegal practices to get top-rankings joining top-positions until google detected the fatal behavior.
Now it’s strange that this site is joining again top positions in the spanish site with exactly the same top-results like they did before when they used houndreds of repeated keywords splitted on 3 or for 4 websites linking to them.
Of course we understand that a re-consideration could have happen
but it’s strange that this site is joining again the same top-positions with more than 10 keywords from the beginning of the reinclusion or expiration of the imposition.
Attention…attention..
Matt is just about to post one of his best articles in 2008:
5 Things You Don’t Know About Matt Cutts 🙂
Matt,
How long would a reconsideration or in this case re-inclusion request take? Is it days or weeks?
I think the idea of putting an interstitial warning about the site being hacked is a _great_ idea. It’ll make it easier for a site owner to find out even if they don’t use any of these email addresses or Google Webmaster Central, both through going to the site itself and their users emailing them to tell them (it leverages the internet public to help get through to the site owner).
The only ‘downside’ is that it would be embarrassing for site owners – I think this is more an ‘upside’, as it will lead to site owners being much more careful and much more aware of security issues, and will hopefully lead to better security.
I checked my site and it doesnt seem to have any symptoms of being hacked… do you see anything wrong with datacard dot net dot in ? I have been looking around for answers all morning and I still dont see any issues..
Could any readers possibly comment on what could have gone wrong that it got deindexed?
This is not the only problem .. there are a few more of my sites which got deindexed today for some reason.. I would love to sort out the issue if any but how on earth do I find out what does google not like about the site? the e-mail addresses listed in there where Google notifies .. I dont have any of them unfortunately.. and I have only just added these sites into webmasters tools and I see no messages there either.
Could some one be kind enough to have a quick look and post their views about why this domain got deindexed and if there is something I can do to sort the problem out.
Thanks
Matt,
I understand why Google has to what they do when a site is hacked. The hard part for me is that
1) I was not notified by anyone when it happened back in December. It was not until I saw the malware warning did I realize that there was a problem with the hosting company.
and
2) Fast forward to today, 5 months later, I have not regained any search engine positions lost when the majority of my site was de-indexed.
No hard feelings though. 🙂
Hey Matt,
Quick question. When a site becomes flagged as “hacked”, and then is cleared up and un-flagged, does that site lose any standings in comparison to pre-hacked standings in the SERP?
“I think we go for a selection of often-used addresses like webmaster@, postmaster@, etc. I think in the case that I blogged about, we also went the extra mile and scanned the site itself to find a custom email address and sent a message there, too.”
None of those approaches would reach me, because those common addresses are the ones the spammers use. I’ve also made a point of not publishing emails in bot-readable form, for the same reason. So unless Google is going to start understanding emails that are obfuscated with javascript, with ascii codes, or put in images, then this is going to be a problem. Munging is becoming essential, as is choosing a non-obvious email address.
Still, I don’t see what more you could do to contact webmasters, other than to scan for common ways to mung addresses, and decode them.
There are a number of IETF RFCs that specify various addresses that a given host should accept email on depending on what services the host and/or domain provide.
Most have become defacto standards although they have been codified into an RFC; http://www.rfc-ignorant.org/rfcs/rfc2142.php
As for receiving spam on them, there are so many spam filters, both client side and server side that are effective that spam shouldn’t be an issue.
But, for those who choose to make one’s self available to communicate with or more often, don’t understand the need, I agree, what more can be done, basically nothing. 🙁
On a semi-related note, Matt, is there any effort or anything that can be done about companies that lose their domain names and that are squatted by a spammer? Maybe with Google’s awesome registrarly powers, go back through the WHOIS records over the course of time and email the old email address (assuming it isn’t of the form name@squattedname.com)?
I can think of at least one prominent case where a major corporation lost their .CA to one such spammer.
Hello Matt,
I like the write up (email) you did on this topic. I own about 60 high quality web sites and blogs. Unfortunately the amount and number of evil spammers and hackers has grown greatly over the last few years. Especially attacks from China and Russia. I know I have sent some emails criticizing Google in the past when Google makes mistakes, however I am a Google shareholder and I do think Google does an outstanding job over all. I also think you have done a lot to increase the value of my Google stock and I what to thank you for that.
So I hope when I email you about Google defects and problems you realize that even though I can be tough and perhaps too harsh about what I consider Google defects, it is all focused on improving Google.
I wish Google would do more to penalize hackers and spammers. I have written Google about known criminal hacker spammers asking Google to delete there sites from the Google index. These hackers are not afraid at all and in some cases Google ranks them highly and this also allows more people to be harmed by hackers and spammers. So please do more to penalize these evil criminals.
Also I publish my email address all over the Internet. How can I do this when so many people can not, and have to play all these silly games to protect their email addresses from Spammers. I know this site is not for recommending products, however in this case please allow this one. http://www.cloudmark.com is by far the best spam protection product on the market and allows me to freely publish links to my email address all over the Internet and does a great job of protecting me from spam without playing any silly games to protect and hide my email address from the spammers.
Hi Matt,
Why do I get the strange warning pop up when I click on my name link from your blog?
I notice that does not happen for others.
You can delete this message if you like, as I just want to understand why.
@Tom Forrest: IANMC (I am not matt cutts) but it would appear you’re trying to enter an email address in the url field. For sites that require a login, the format is http://USERNAME:PASSWORD@domain.com. So the @ from you entering your email address looks like an odd login request.
Matt it always amazes me how much you and Google do to help people as much as possible when their experiencing problems. It looks like your team sent out several emails and did everything you could to alert Ryan.
It does suck for the webmaster but as you pointed out, malware could do a lot more damage for his users.
Matt,
Feel free to delete the previous two posts.. lol.. its late night at the improv again.
Anyway what I meant was:
Here is my solution. Since Google keeps a cache copy of the site… What if they served the visitors this page during the process that the webmaster fixes the site? During this period the webmaster is sent a warning and told that the visitors are being sent to a “safe” version of their website until the problem is fixed?
Any possibility of that?
Ben
I’m amazed at times to see how lazy site owners are about doing this – not wanting to pay their webmaster for the hour it takes is absolutely ridiculous.
Matt,
Removing a hacked website or blog is imperative to safeguard your users’ interests. We understand that.
Whats painful is the delay before the website comes back up. You have already addressed that, but a simple suggestion is: Include that in Webmaster central. Keep the webmaster informed about whatever is going on with his/her website.
The more information Google provides to webmasters (even if its one way communication), the more people will respect Google. For a normal blogger, his/her blog is perhaps what Google is to you.
I have a website called LondonHotels4u.com, banned or penalised on Google for 2 years.. hundreds of emails, re-inclusion requests etc.. leads to nothing. Not even a “We won’t include it”.. just silence. And thats what leads to fall of major businesses.
Similar story
I have just had the same thing happen to me. Hackers somehow managed to get into my site and add a load of porn links embedded in the page. Thing is, the site seems like it’s been delisted from google, but I can’t see anything wrong on my webmaster page. I’m pretty sure i’ve fixed it now, but not sure if i need to ask for a relist. Does anyone have any suggestions?
@Ben C, your proposal seems to cross the line from fair use into copyright infringement. Plus, if a site is hacked it’s not easy to sort the good content from the bad: perhaps none of it will be left as the webmaster intended.
Archive.org already does a pretty good job of recording older versions of websites. Perhaps an intermediate solution would be to indicate when the last clean version was available, so people can go and look up the older information there.
I’ve just seen this warning on my first ‘proper’ site as opposed to someone’s blog – the UK’s Immigration Advisory Service. Click my name for a screenshot. Anyone got a way to tell them …?
You live and learn – just found this topic by default as I’ve discovered our main competitor has been using this method – ie getting his main keywords on hundreds! of sites/blogs by hacking and now occupies positions 1 and 3 for the most lucrative terms.
I originally thought he’ been buying those links – doh!! and reported it as such – he’s still there so don’t know if it’s been effective – but where do you report such activity – it seems mighty unfair that the innocent party gets temporarily penalized while the perps:) go unpunished!
Recently I found that someone had hacked my site and injected spam links into Google search results. I had google alerts set up to keep an eye on this and luckily nipped it quickly, however, it seems some of my pages ended up in Google’s cache with the spam links included. Is there a way to request a refresh of Google’s cache for a web-page?
The site is 100% clean now so a re-inclusion request seems inappropriate, especially since Google is still sending my site traffic (albeit it seems there has been a slight drop). I’ve tried searching for an e-mail contact or other way of requesting this but there doesn’t seem to be a listed method. Is there a way the cache can be refreshed so that these spam links are 100% gone? Or should I simply wait for the cache to get refreshed. Also, is there a set schedule for when the cache is refreshed?
Thanks for your tips Matt, they are very helpful to us web-masters.
Hi Matt,
We run many websites. With the increasing sql injection attacks we found 2 sites with vulnerabilities had be hacked on the 13th of June. The hack places javascript links to malware on the site. On the 16th of june we saw Google traffic drop on both sites significantly.
We fixed the vulnerability and have submitted a reconsideration. My questions is:
Are the messages in the webmaster tools an automatted system?
Google was obviously aware of the issue and reacted in the best interest of the user. However we only discoved the issue days later and a message in the webmaster tools would have been very useful.
Hey Matt-
I’ve read through most of the posts on this blog…Our site was attacked by malware two weeks ago. We received all of the blacklist e-mails from Google, and our site was fixed immediately. We are now in the process of having our site re-built in .asp.net platform with hopes of protecting our site from future attacks.
Our search terms were #1 or at least on the first page for the vast majority of our keywords, and now they’re gone. Our positions have completely dropped off. I know we are indexed because I can find some of our keywords ranked really low (ie page 10-20).
I’ve read a lot of posts on other forums of horror stories of people having fixed their sites and 3-4 months have passed, and their sites still haven’t been returned to their high rankings.
Your original post to Ryan (first post at top of page) said that his site would return to the top of search engines within a couple of days. Can you tell me why isn’t this happening for us?
We are also a paid advertiser and do upwards of $75K of advertising with Google, and our paid listings were able to be up within a couple of hours of us fixing our site.
What is your opinion on our situation? There is no way for us to gauge whether or not our placement will ever come back. Should we continue to wait, or try to start finding other ways to get our site back up to #1 on natural listings?
Thanks so much for your time,
Matt Morris
Thanks for your post Matt.
One of my sites has just been hacked (first time). Fixed by moving the hosting. I now just have the very anxious wait to have the site re-included into your index. I had hoped I had acted quick enough – sadly not.
I looked for a website monitoring service that could possibly detect if one’s site has been hacked – without luck.
Do you know of one? Many thanks again.
One of our websites got hacked with hidden text, hence Google removed us from index. We completely cleaned the hidden text and went through the reconsideration process.
Here’s my problem. Google IS the search engine of choice. Many companies, like ours, depend on this search engine for our business. We’ve been online for over 11 years and never had a problem. All of a sudden, we get hacked and are now removed from Google for a minimum of 30-days. THAT HURTS! Especially for something that’s not our fault.
I think that Google should have a more expedited reconsideration process for first-time offenders and/or for sites that have a long, positive history.
One of our websites had also been de-indexed from Google, I have a webmaster account but no mention is made of the site. I’ll try to request reconsideration today. Seriously though, hackers are giving genuine website owners a hard time by exploiting our weaknesses.
My site has been hacked. I removed what I could and submitted to Google. Google said it still found malware on my site. How am I to identify what should be removed? What “words” or “strings” should I look for?
Has Google a monopoly over the internet?
The procedures seems to be in place, but the manpower? Google owns their own algo, so it can flag anything it wishes. Hackers hack, and we have to clean it up. Would be nice to see some more on the treatment of these parties and the legal ramifications to discourage such activities in the future.
I don’t get it. Some sites have more than 100 links on a page but still they have a Pr > 0. But other websites having less than 100 Outbound links and having related website links couldn’t get any Pr.
Well my site was hacked and dozens of links of Viagra were installed hidden behind the CSS.
I didn’t noticed until the site was removed from Google results few days ago. When I found the problem I removed all this content and filled a re inclusion request in the webmaster tools.
Nothing happened yet. and of course no body from Google even bothered to give a hint if the message was read or if this is the reason the site was removed. But it is good to know I am not alone with this problem..
After reading these posts, I feel like I’m re-living the nightmare of the past 5 weeks. Simply put, like most, our website was hacked via our blog, loading spammy drug links and defacing our htaccess code. In time, our previously superior SERPs were gone one day and shortly thereafter, business come to complete halt.
Google was great to work with and after 5 weeks of hard work, the site and code was cleaned and Google put us back in the results… and within 2 days, our previously high SERPs returned… but not the traffic.
Here’s the question: It’s been over 2 weeks and our Google traffic has only returned to approx 35-40% of it pre-hacked levels. I would assume that traffic would come back to comparable levels, at least 80% of previous levels. No one has been able to provide an explaination.
The only consideration: our blog was the point of attack and was taken down. The new blog went up yesterday… a fully cleaned version, secured, etc. The blog was somewhat active, with approx 300 posts. But still… might provide some google links to the site.
Anyone…. an explaination? and if so, ideas or suggestions what I can do next to uncover any lingering issues or problems?
Thanks
Matt, your link to get back in the SERP “(The rest of the email goes on describe how long the blog will be out of Google, and where to go in order to get back into Google’s index faster.”) is 404 🙁 Could you provide a working current link with similar information?
Thanks!