Closing the loop on malware

Suppose you worked at a search engine and someone dropped a high-accuracy way to detect malware on the web in your lap (see this USENIX paper [PDF] for some of the details)? Is it better to start protecting users immediately, or to wait until your solution is perfectly polished for both users and site owners? Remember that the longer you delay, the more users potentially visit malware-laden web pages and get infected themselves.

Google chose to protect users first and then quickly iterate to improve things for site owners. I think that’s the right choice, but it’s still a tough question. Google started flagging sites where we detected malware in August of last year. This February, the webmaster console team and Google’s anti-malware team took a big step toward closing the loop for webmasters:
– The webmaster console started listing example urls with suspected/detected malware.
– Google began attempting to email site owners when we detected malware.

Today, the two Google teams added even more functionality into the webmaster console:

– New: Request a malware review from Google and we’ll evaluate your site.
– New: Check the status of your review.
* If we feel the site is still harmful, we’ll provide an updated list of remaining dangerous URLs
* If we’ve determined the site to be clean, you can expect removal of malware messages in the near future (usually within 24 hours).

I like that Google will keep updating the list of dangerous URLs for a site, and that they’re working to remove malware warnings even faster when sites clean up malware. That will help site owners diagnose their problems and get them fixed faster. What’s just as exciting to me is that while I have written about malware unofficially in the past, Google has ramped up official posts about malware on Google’s online security blog.

I’m glad that the Google’s anti-malware team has been doing all this stuff to alert site owners if they’re hosting malware. I don’t think it generates any money for Google (if anything, it costs machine resources and engineer cycles to tackle malware), but it does improve the web as malware gets taken down faster. I guess there could be an indirect effect as people trust the web more and maybe surf more, which is good for everybody.

49 Responses to Closing the loop on malware (Leave a comment)

  1. It’ll be a useful feature. Now we can avoid links to dangerous pages, that’s great.

    But other sites which are not listed in WCT accounts and probably have links to dangerous pages now can get lower positions in SERP, aren’t they? Is there any dependence between such links and SERP position?

  2. That PDF was an interesting read.

    Does that mean that Google is going to interpret JavaScript (an VBScript) code to detect obfuscated dangerous code?

    Or the first phase would be that allobfuscated code is marked as potentially dangerous?

  3. I think it’s great Google is taking these steps!

  4. More communication and less obfuscating is always better. Congrats to the two teams involved.

  5. is there any way so that we can find out the list of these sites.so we may not link or visit that pages or websites

  6. Great stuff, Matt (and the webmaster central team)!

    What happens when the payload site is offline (eg the source of an injected iframe) – do you still notify the webmaster of the hack? Do you have any plans for notification of SEO hacking?

  7. Very userful, but why did RussellGrant.com get hit? See screen capture here:-
    http://insidemyviralmind.blogspot.com/2007/08/russell-grant-in-spyware-shocker-he.html

    I’ve never found this site to be in anyway harmful.

    Paul

  8. OK, good technology that allow you to delete your own officiel Google blogs πŸ˜‰

  9. > I don’t think it generates any money for Google

    Malware results are highlighted in Google search results, which improves Google search results, which makes it more likely people use Google search, which makes it more likely they click on AdWords, which generates money for Google. Everyone wins πŸ˜‰

  10. I reiterate what I said here. It should be easy to omit newsgroups like alt crackers, cracks, dk.warez, etc. from Google Groups. Google should have started there.

  11. Matt this is great news and a very cool feature.
    Are you considering banning these sites altogether?
    If they are harmful, they shouldn’t exist in the index.

    Thanks

  12. What if a hacker submits his website for malware review, and after a thumbs up from Google, installs malwares on his site? How often does Google re-check a site for malware?

  13. It’s at least an approach to start protecting users first and then go on improving things for site owners. – sounds a bit like “shoot first, ask qustions later”. πŸ˜‰

  14. I am so glad Google is working on malware. Anythin to clean up the web. We need to get rid of the troublemakers. More power to you!

    Raj, administrator@lookupcreditcards.com

  15. Is it better to start protecting users immediately, or to wait until your solution is perfectly polished for both users and site owners?

    I don’t see this as a difficult question, although I must be missing something. Protect users immediately, but allow for false positive reporting and use the feedback from false positives to improve the quality of malware detection.

    Not only that, holding off on an issue like this increases the difficulty of being able to deal with it, since new malware variants are developed on what seems to be an hourly basis.

    Hang ’em high, Google! Dangle ’em from a big ol’ rope in the street!

  16. Hello Matt
    I wanted to ask you something
    I have read somewhere Google PR has not been exported as it is being changed into Google Webrank.How much % of it is true?

  17. so if I was have link dangourse should’t google first
    tell me about that at least on google webmaster?

    i think not all webmaster pro to know that
    and all this depended on google

  18. anything that helps users/site owners detect malware more easily is a step in the right direction. kudos to google for stepping up.

  19. egorych, I think that this is a straight warning. If we think you have malware, we show an interstitial to users. But I don’t think that it affects your ranking.

    JohnMu, SEO hacking is a more difficult topic. I’d say we’re open to contacting webmasters (and we’ve emailed plenty in the past), but we’re still at an earlier stage for that.

    TOMHTML, they’re completely unrelated technologies. πŸ™‚

    Multi-Worded Adam, I agree that you need to protect users first, but we certainly took some flak from sites that claimed not to have malware. Including on this comment thread. πŸ™‚

    Paul Reilly, it looks like something was on the root page of that domain on a fetch at 2007-08-10 09:50:59. I may try to look into it deeper later, but the whole point is that the owner of that site can check in the webmaster console to get the specifics themselves now.

    Halfdeck, we haven’t seen that happen much (any?) in practice, but we’d be open to adjusting things if we saw people trying to abuse the review in that way.

  20. This is great news Matt. It should really help to clean up this problem. Congratulations and Thanks to everyone at Google involved in the project.

  21. I think Google should offer the option of an e-mail alert if it detects malware. I’m guessing people check their email more often than their webmaster console.

  22. This is very good news! As someone who’s been hit with malware on several occasions, I’ve been anxiously waiting for Google’s progress in this area. I look forward to additional improvements. Hats off to Google’s anti-malare and webmaster console teams!

  23. Well great,

    I managed to get a virus last month that knocked me out of commission for a week until I could get my hard drive replaced. It came from just visiting a web site. I tip my hat to you guys over there at Google who are showing some real concerns for us end users.

    Now, how about leveling the playing field in the search engine results pages for web site designers who want to design web sites rather than fruitlessly spend the day soliciting, begging, borrowing or even having to resort to buying links in order to stand a chance at decent rankings in competitive markets?

    Unfortunately, I am one of the old fashioned SEOs, since the coming of Excite, who believe that the art of search engine optimization is in the quality of craftsmanship of the content, whereas SEO has little to do with off-site factors at all; a dying breed amongst the swarm of link strategists who make it appear as though they are performing SEO.

    It is a question of ethics Matt. Is top ranking content uniquely important, credible and authentic, or does it only appear to be so artificially?

    Sorry for the tough comment. I grow fatigued arguing in SEO circle with link strategists that don’t know they are actually practicing blackhat optimization.

    Also, don’t you think it is about time to scrap another good idea gone sour . I am referring to PageRank. Quite a massive volume of manipulation has resulted from the frenzy to push a little green bar and in getting links from the high guys?

    Apologize again for the tough comment. In all respect, I understand your predicament.

    kind regards

  24. Dave (original)

    Is it better to start protecting users immediately, or to wait until your solution is perfectly polished for both users and site owners? Remember that the longer you delay, the more users potentially visit malware-laden web pages and get infected themselves.

    So long as “immediately” doesn’t give the scum an advantage in reserve engineering the protection?

  25. Dave (original)

    Now, how about leveling the playing field in the search engine results pages for web site designers who want to design web sites rather than fruitlessly spend the day soliciting, begging, borrowing or even having to resort to buying links in order to stand a chance at decent rankings in competitive markets?

    The play field is level and objective too. I believe what you you really mean is, my site pages aren’t ranking how I want them πŸ™‚ There ARE only 10 spots on page 1, 2…

  26. Matt:

    recently, while search for some of my keywords, I ran across a site that ranked very high for one of my company names, but is not associated with me in any way. I looked at the source and found this:

    [code]

    if(window.yzq_p==null)document.write(“”);

    if(window.yzq_p)yzq_p(‘P=M_.xedG_Qkc2E7LY2AUvnwBmGDoD0EbCbkAADGc6&T=13ntefut0%2fX%3d1187147328%2fE%3d23732888%2fR%3dst%2fK%3d5%2fV%3d1.1%2fW%3dJ%2fY%3dYAHOO%2fF%3d3984618243%2fS%3d1%2fJ%3d4742BFD1’);
    if(window.yzq_s)yzq_s();

    [/code]

    is this some sort of BlackHat technique for ranking high for a particular keyword?

  27. Sorry, the [code] block didn’t work, and most of the code is JS. I can forward it to via some other means if you’re interested in it from a SPAMMING point of view.

    The other thing that I forgot to mention is that this code was after the tag.

  28. Keniki, take a deep breath and take a break from the comments for a bit. πŸ™‚

  29. Dave (original)

    Don’t worry, Keniki is delusional on his good days. Either/or dropped on his head at birth or his tin-foil hat is too tight πŸ™‚

  30. This was NOT very well done. Whenever they are flaws in the detection system, innocent Websites can suffer – which essentially means business site owners will lose potential revenue.

    Why is it okay to take a chance and alert siteowners about potential malware – but stop alerting site owners about being banned?

    If Google has halted the later policy because of deceptive spam emails falsely claiming to be from Google Webmaster team – WHAT ARE YOU GOING TO DO if false emails begin about malware??

    The term malware is an unprofessional generalization – if you are going to attempt to warn potential visitors away from a site – then there should be more detailed term used about WHAT exactly is on the site that people should be wary of.

    NOT ALL Webmasters have a Google console – so is it fair to them?

    Presumably, if you are going to contact a Webmaster you suspect of having malware – then there should be a visit to the site to determine what so called malware exists – then say specifically what the objection is – or admit the algos made an error.

    Just like there are spam detection teams – there should be malware manual verification teams.

    But the REAL objection is the term MALWARE – that should be substituted with a more detailed, specific term. πŸ˜•

    BUT …..there’s more!

    Remember, just a few months ago – Google itself – received undesirable worldwide publicity from a research group about being the WORST offender regarding privacy policies. Of course, this made many people in Google angry – and prompted replies about the unfairness of the report – as well as possibly prompting changes in cookie policies.

    So make an analogy about how angry the good Webmasters must feel when they get branded with the malware warning on Google SERPs – not the evil Webmasters, but the good ones. …………………………………….

  31. But the REAL objection is the term MALWARE – that should be substituted with a more detailed, specific term.

    See the Usenix paper, which Matt linked to. The abstract says:

    …we
    identify the four prevalent mechanisms used to inject malicious
    content on popular web sites: web server security,
    user contributed content, advertising and third-party widgets.
    For each of these areas, we present examples of abuse
    found on the Internet.

    Further down it says:

    3. DETECTING DANGEROUS WEBPAGES
    Before we describe how to detect malicious web pages automatically,
    we need to explain our definition of malicious.
    A web page is deemed malicious, if it causes the automatic
    installation of software without the user’s knowledge or consent.
    We do not attempt to investigate the actual behavior of the installed software but rather identify the mechanisms
    used to introduce the software into the system via the
    browser.

  32. Oh dear – I have taken at least 15 minute search all parts of the webmaster console to make a request on all my sites but was unsuccessful in finding it. Sorry Matt, please could you explain exactly where it is.
    Thanks

  33. A Google mantra is “focus on the user”. Yes?

    So identifying potentially hazardous sites is the right thing to do. You can add meaningless complexity (user rating malware efects, etc) later – I’ve like this move from the time I first noticed it, and like everything else, your army of monkeys will evolve it.

    This time, Google has treated webmasters as users. A change introduced for the wider community has resulted in changes for webmasters. Also good.

    Rating the burblings of Russell Grant as malware for the mind was a stroke of genius. Oh, you mean it wasn’t because you’ve taken against the junk memes of astrology? Drat πŸ™‚

    Nice Usenix paper… Thanks. It’s good to see some of the sources/resources that influence Google, and that not all of them are internal.

    Cheers, JeremyC.

  34. Good! It used to be the case when certain search terms were so full of maleware, they could basically be considered as leading into a red-light district. People just couldn’t surf there safely.

    The more that is done to ensure that people searching from reputable sites like Google do not end up in those bad areas, the better.

    Thanks!

  35. Paul Reilly, I asked the team about it and it looks like that domain had an iframe, and that iframe what was delivering a browser exploit.

  36. Matt, can you elaborate on this? Is it something they did unwittingly, or is it deliberate.

    Can being affiliated with a site such as this be dangerous ground? e.g, if I linked from my website to russellgrant.com (particularly, if they link back,) will the red flags in google go up for my own website?

    I don’t know if it’s been mentioned anywhere, but are google considering putting warnings on their toolbar should we navigate to flagged websites by modes other than Google?

  37. Dave (original)

    Lewis, why would intent make any differnce when the end result for user is the same regardless?

    Why would you even consider linking to a Malware site that can harm your site visitors?

  38. Pehaps if you didn’t know it had malware? Or if it didn’t have malware when you linked to it, but then it got some.

    I’m not saying intent should make any difference; I was just interested if it was something deliberate or not. Perhaps something a nasty ad service had served up…

  39. Wow that is great news Matt. Thanks so much to you and the team for including this type of protection in Googles search results. Now all you have to do is concentrate on Googles search results.

  40. Matt can never win with you guys.

  41. Matt can never win with you guys.

    Sure he can. He just needs to learn sleight of hand three-card monty style, how to mark cards, or how to hustle pool.

    Search engine stuff, though? Matt’s standing between two seven-year-olds with an ice cream cone.

  42. Dave (original)

    Hmmmm, ice-cream πŸ™‚

    Matt can never win with you guys.

    I don’t think Matt tries to “win”.

  43. Detecting malware in search results is great. I’m sure the techs there have already thought of it, but it would be nice (as long as it doesn’t slow down your browser) to implement malware detection into the Google Toolbar (as an option). It might make your toolbar much more useful than other toolbars – especially to people that are sick of removing malware from their machines.

    Lewis might have been saying this, but not sure.

    -tim

  44. * If we feel the site is still harmful, we’ll provide an updated list of remaining dangerous URLs
    * If we’ve determined the site to be clean, you can expect removal of malware messages in the near future (usually within 24 hours).

    I saw this notion a couple of weeks ago on a website, i think this is a very good cause! helps protecting the users! keep it coming guys πŸ˜€

  45. Great artical Matt,

    I would get it out to the public as soon as possible rather than waiting for the totally cleaned up and perfect version.

  46. Hi, Matt. I think this “malware flagging” may severely hurt a few sites from well-intentioned institutions operating on politically sensitive niches and, therefore, more vulnerable to hacking attacks.

    I have comented about the “Coletivo Feminista” (http://www.mulheres.org.br/) case at my blog (http://alexiskauffmann.multiply.com/journal/item/570/Vixe_Maria_O_Google_virou_xerife – sorry, it’s in portuguese).

    The Coletivo Feminista is a very well-known brazilian feminist militant organization and their website has never been harmful, except for those who think feminism is “ideological malware” for that sake.

    The Google “Malware Alert” was so scandalous (Don’t access this site! It may harm your computer! blablabla) that it definitely did more harm to the institution’s corporate image than mere de-indexing would do!

    As John Carlton puts, “Google cops” should evaluate closely these possible side effects before jumping in the head-cutting business. You know, Google has too much power now. Would you sleep well if you discovered the next morning that you’ve sent an innocent man to the electric-chair? Those “malware alerts” are pretty much alike death penalty. If you find out the next day that you were wrong, how many people will have seen and trusted your judgment failure?

    Please, understand me: I believe that, at the time Google evaluated the website, it was maybe really harmful. But surely it was cracker’s fault, driven by ideological, political motivations.

    I am so sure about it because I know those ladies at “Coletivo Feminista” and the web is a secondary concern in their lives…

    Cheers,

  47. It is great that Google are combating Malware in such a way.

    At the end of the day the web is where Google’s customers are, so the nicer the environment the more it will get used.

    Keep up the good work πŸ™‚

  48. I agree with Google on this one. Some malware protection is better then no protection.

  49. HI Matt, is there such thing as “Google Live Indexing?” When someone posts an article, does it get indexed asap, or does “live indexing” refer to allowing xml publishing protocols in the backend and pushing content out to various platforms?

css.php