SiteAdvisor Study

Kevin Delaney over at the Wall Street Journal has a short article about a recent SiteAdvisor study of potentially malicious web sites. According to the article, SiteAdvisor claims that about 2% of regular web sites may expose surfers to “risks or nuisances,” while the number for search results is about 3% and the number for search engine ads is higher.

I could pick nits about this study (for example, SiteAdvisor’s definition of a bad site includes asking for email addresses), and it’s quite rare for me to hear complaints about badware on Google. But I do hate scuzzy behavior, especially in our search results.

In fact, several weeks before I found out about this study, we added a new provision to Google’s webmaster quality guidelines because of the WMF vulnerability:

Don’t create pages that install viruses, trojans, or other badware.

I’ve said that before, but it’s nice to make it official. Just as an aside, I’m surprised no one noticed this addition. I thought SEOs watched our quality guidelines with eagle eyes? Gary Price, I miss your uncanny ability to notice changes on a website. 🙂

Google’s statement to the WSJ made it clear that we don’t want junk in our ads, either:

Google Inc. said in a prepared statement that it prohibits ads that promote spyware, viruses and other malicious software and removes them when it becomes aware of them.

That’s a fine response. But given how much I hate web pages that install malicious software or abuse browser security holes, I’d like it if we did even more to protect our users.

36 Responses to SiteAdvisor Study (Leave a comment)

  1. Talking of badware, personally I have never seen any site in the SERPs that installs viruses, trojans, or other malwares. That mostly depends on what you are looking for, IMHO. If you are looking for some illegal / unethical material, then you can expect such a problem from the site appearing in the SERPs.

    Even though, this is the responsiblity of the software vendors to protect their end-users by taking security measures, yet it would be great to see how Google will be tackling with such sites. 😉

  2. Hi Matt

    I imagined that the Googlebot would be able to weed out the sites which try and install badware. Maybe I’m overestimating the bot’s abilities!

  3. I tend to agree with SEO Junkie. This isn’t always true, but generally speaking when I encounter spyware/badware/malware, the first thing I do once I’ve cleaned it off is take a look in the client’s browser history to see what they’re doing. 99 times out of 100, the culprit site is there and usually it’s something that poses a risk, such as porn or sites that have about a gazillion ads and very little actual content.

    As much as I’d like to see them stop going to those sites, the problem is that you could be accused of having a potentially vested interest in doing so, depending on what exactly you blocked.

    For example, what if you blocked Claria/Gator/GAIN/PrecisionMyAssTime ? In the screw-you-and-sue-you society in which we live, that’s an open invitation for Claria to call out the legal dogs on big G due to the AdSense program.

    (Side note: I’d have no problem if Google blocked websites running Claria ads myself since that little gator guy pisses me off, but I’m just showing that there’s a potential for major hassle.)

    Where exactly is the “badware” line defined?

  4. Dave (Original)

    Agree. Those in glass houses shouldn’t throw stones! To be honest, I’m glad most that visit certain sites (black hat stuff) get a taste of their own medicine.

  5. Hi Matt

    I personally do not think enough is done about ‘badware’ websites, while extraordinary efforts are now being upped to deal with ‘spam’ sites ‘badware site’s still seem to proliferate and I think a solution is there should be a rapid response to any badware site that is reported through Google’s ‘report spam’ program.

    Several times recently we have come across sites that when opening have multiple windows open. There is a well known domain seller that for some reason doesn’t check the domains it sells correctly and allows for this to happen. An example I was looking to buy a domain for a client and going through the ‘third party seller’ I opened the domain and at least over 180 pages opened fast on me with some real grim spam content. To let you know that this site was indexed by Google.

    You simply can’t control all these sites but you can control who is indexed by you and feel there should be faster and more rapid responses to complaints about ‘badware sites’ is it possible when dealing with these sites to notify the ISP/Host as well so they can take action?

    I fully understand it’s an ongoing battle with these purveyors of ‘badware’ but I still think it’s a corner of the web that ahs not been explored fully or effectively in dealing with the culprits.

  6. Dave,

    Very well said. I feel like repeating it. “Those in glass houses shouldn’t throw stones!” 🙂

    Well, normally such malwares comes from visiting those shady sites. Visit a porn / cracking related site and you have a spyware, trojan horse, virus or some kind of malware 99.9 times out of 100.

    And If I am not mistaken, it’s not a hard nut to crack for SEs to find a way to detect such sites by faking their user-agent etc.and checking for any security alert much like a normal browser.

    Isn’t it possible, Matt? 😉

  7. I would think it’s almost impossible to tell between legit software and badware without human intervention. Some sites send a visitor to a page with automatic downloads on the page load… Why can’t we all just play nice???

    If I were to put badware on Oiskas I know that word would get around quickly and people would stop coming.

    Kudos to Google for attempting to stop this kind of stuff!

  8. Come on Matt, most ‘good’ SEOs don’t ‘need’ to keep up on the google guidelines.

    The only interesting thing I have seen in the guidelines in years and years is the fact that google has finally admited that a link FROM a ‘bad’ neighborhood can hurt your rankings.

    Quite a turn around from the old standard, that nothing anyone can do (except youself) can hurt your rankings in google.

    Now all google needs to do is put up a list of ‘bad’ neighborhoods so we all know if that new link is going to hurt us or not…

  9. The fact you feel you have to point this out says a lot about present day SEO’s IMO

  10. Webmasters and SEOs can get caught up in the insanity of attaining higher PageRank, SERP’s, datacaenter analysis, page cache, linking schemes and so on. For me, a quick trip back to the Google guideline pages is a Zen like experience. It reminds me that it’s all about building a community. A web community connected by relevant links and great content which hopefully will garner you tons of traffic.
    Keeping it simple is a good start… Black text on white pages!

  11. It can be tough to define malware/badware/scumware, and I think Google does a good job on malicious stuff in our results (and others concur), but I personally wish we could do more to stomp out things that exploit browser holes and similar stuff. I don’t know if that’s the official Google stance, but it’s my belief.

  12. Matt:

    The fact that Google at least has a proactive approach to stomping out scumware/malware/spyware/grayware etc is notworthy in of itself. The one item I would be interested in hearing about is if G has any plans on more proactively curbing ill-intended client & server side redirects. Granted the client side redirections are easier to spot, but I have noticed more and more “ip-delivery” schemes, specific http_agent / http_referrer redirections. The server side methods are significantly more difficult to detect and prevent. This becomes a larger problem when generally white-hat & established sites begin offering “Pre-sell” & “Advertorial” pages that pass trust, traffic, pr and also targeted/anchored backlinks that deliver significantly different content based agent/referring url. Gbot still rewards the anchored backlink to the target URL despite the final content post-redirect. Personally I think that mal/scum/spy/bad-ware extends to technology used to exploit bots to gain serp/traffic. Really makes it difficult to happily skip along the whitehat highway when your ducking the blackhat tractor trailers 🙂 lol

  13. How about adding warez crackz etc to the discussion. Crack sites often try to install badware, even if they don’t their advertised content is illegal.

    I looked at one of these sites advertising a crack of one of my applications, I was curious to see if the crack actually worked, I never actually got the crack because they require an ‘installer’ to be download, which, not being a complete idiot, I wouldn’t do. I was tempted to set up a quarantned machine to see what I got, I’d like to see the crack, if it exists, to defeat it. However, life’s just too short! Looked at the right way you might consider a crack to be a compliment.

    Some would say that if you download a crack you get what you deserve, I would be tempted tp agree – sort of.

    Searching Google today for warez gets 16,400,000 hits, sure some of them are anti warez statements, but number 3 for instance calls itself a ‘Software piracy portal’ As this is clearly an illiegal activity and fairly easy to distinguish programatically shouldn’t G do something to filter out these sites from their results rather than concentrating on spammers who may be a pain, but mostly aren’t exacly Evil, just desperately trying to boost their rankings, rather than actually comitting a crime?

  14. Dave (Original)

    RE: “The only interesting thing I have seen in the guidelines in years and years is the fact that google has finally admited that a link FROM a ‘bad’ neighborhood can hurt your rankings.”

    Fiction: A competitor can ruin a site’s ranking somehow or have another site removed from Google’s index.

    Fact: There’s almost nothing a competitor can do to harm your ranking or have your site removed from our index. Your rank and your inclusion are dependent on factors under your control as a webmaster, including content choices and site design.

    Lots0, care to point out where you read in the Google guidelines that a “link FROM a ‘bad’ neighborhood can hurt your rankings.” ???

  15. Matt said;
    >>>I personally wish we could do more to stomp out things that exploit browser holes and similar stuff. I don’t know if that’s the official Google stance, but it’s my belief.

    Matt, I hope you talk to Larry and Sergy and can convince them to agree with you and get them to take some real action.

    I personally believe that all the search engines do have a responsibility to eliminate the scams, fraud, illegal activities and badware of ALL kinds in their indexs.

    These scams, frauds and badware are exposed (brought) to users by the user using the search engines, so IMO the search engines should remove them or pay for the dammages caused by the fraud, scams and badware that users find in their indexes…

  16. @lots0:

    I can’t say I agree with that statement at all. Holding ANY search engine financially responsible for damages as a result of “scams” which they may have indexed is somewhat absurd. Unless someone *could* show the SE was acting with gross negligence or was financially partnered/profiting from a “scam” it would/could/won’t ever be held liable. SE’s index content from a global marketplace, have no affiliation with the sites so nothing short of an INTERPOL investigation will scrutinize their results. With the amount of blackhat going on and the multi-national points of origination they stem from it is completely un-enforceable for all but the most serious “crimes”. Coupled with the fact the G for instance is proactive in it the quality of results, publishes guidelines, and thus far has employed due-dillegence in resolutions of these types I whole heartedly doubt this will ever happen; nor would I ever personally hold them accountable unless they were in fact grossly negligent. For example would you pursue legal action or retribution from a telephone directory publisher, newspaper/classifieds, or flier you find on a street that advertises “good attorney” who milks your for every penny and provides no results? Let’s remember it is in G’s best interests to provide relevant, accurate and targeted results … they gain nothing by omitting known “crap-sites” and ultimately it may hurt their marketshare and client base. Leave legal battles/arguments for the local governing bodies that are dutied with the task… trying to hold a marketing agency (yes it sounds painful but lets keep our eye on the ball) responsible for censoring and moderating several billion pages of content is completely unrealistic let alone not their job.

  17. What is absurd is giving the search engines a free ride.

    Google defended its self against a law suit stating that they and only they are responsible for what is in their index and that their search index is entirely google’s OPINION. The US Federal Court agreed with them.

    So according to a US Federal Court if google (or any search engine) has a Scam website in its index, it is their opinion that this site is good enough to be in the index and rank in the top 1000 sites for that search query, so they are in fact responsible, like it or not.

    I am amazed no cases have made it to open court yet. I hear out of court settlements are a wonderfull thing for some folks…

    The search engines say on one hand we are the only ones responsible for our content, its our OPINION, then turn around when it suits their purpose and say that they are not responsible for their content…

    The search engines can NOT have it both ways, I think even they realize that.

  18. I agree with CJK on this one. As much as it would be nice to see big G really get hardcore on these types of things:

    1) they don’t owe anyone anything in this regard. They’re a search engine, not a censor.

    2) As I pointed out earlier, where is the line defined?

    3) Google has got to be extremely careful to avoid potential conflict of interest situations where they block an adware site from the SERPs. Since ad brokers create adware, and since Google is an ad broker now (among many other things), they’ve got a fine line to toe here.

    4) Where would perverts find fresh sources of porn once The Hun and Tommy’s Bookmarks become stale? WON’T SOMEONE PLEASE THINK OF THE PERVERTS?

    (Hey, someone had to lighten this thing up.)

  19. >>>Unless someone *could* show the SE was acting with gross negligence…

    The site is in their index and they know it is doing somthing illegal…

    Hmmm if that is not gross negligence I don’t know what would be…

  20. @lots0:

    Definition of gross negligence:

    “…giving the search engines a free ride…” – Far from free by any and all quantifiable metrics ( Do you have any idea of what kind of financial resources it takes to become a 2nd tier directory/search engine let alone one of the big 3???) . To me it sounds like sour grapes for whatever reason on your part. We all generally make a pretty good living by the SE’s one way or another….. some days better, some days worse… and I have yet to see a “spam/scum/mal/bad*ware” site listed in the first 3 pages of any search I have done… but then again as someone else mentioned; if your searching for “warez +porn +google hacks +cracks” spend a few dollars on some good antivirus/antispam/anti-adaware software becuase every SE has them in the results…. and generally in that genre of search “you get what you pay for/deserve”…

  21. Dave (Original)

    Lots0, care to point out where you read in the Google guidelines that a “link FROM a ‘bad’ neighborhood can hurt your rankings.” ???

    I ask the question above but guess you didn’t see it 😉

    SEs never will, or should, take “responsibility to eliminate the scams, fraud, illegal activities and badware of ALL kinds in their index”. If they did, they would have a law suit on their doorstep each and every day.

    SEs are only a means to an end. Taking “responsibility” for things beyond their control would be financial suicide. Should a phone company take “responsibility” for publishing a phone number of ‘cowboy’ mechanic?

  22. >>>“…giving the search engines a free ride…” – Far from free by any and all quantifiable metrics ( Do you have any idea of what kind of financial resources it takes to become a 2nd tier directory/search engine let alone one of the big 3???) .

    You totally missed my point (Somehow I am not surprized). I was talking about a free ride as far as morality, ethics and maybe even the law.

    LOL, no not sour grapes, my sites do well enough in google (better than most), just thinking a little deeper than the surface.

  23. >>>Should a phone company take “responsibility” for publishing a phone number of ‘cowboy’ mechanic?

    As far as I know, no phone book has ever said what it published was its ‘OPINION”…

    Enough with the lame analogies… please… 😉

  24. Dave (Original)

    RE: “Lots0, care to point out where you read in the Google guidelines that a “link FROM a ‘bad’ neighborhood can hurt your rankings.” ???

    Third time lucky perhaps.

  25. Dave (Original)

    RE: “As far as I know, no phone book has ever said what it published was its ‘OPINION”…”

    Why does “OPINION” change anything in your mind? I could post a URL to ‘bad site’ and say IMO it’s a great site. I believe you would be hard pushed to make me “responsible” for any misfortune you encounter as a result of clicking the link though.

    What about when Google lists a site that has ‘code’ to do something and the code is poorly written and the end result is your PC blows up. Is Google “responsible”? Of course not, it has no direct control over ANY sites listed in its index.

    Perhaps you think it’s ethical for Google to take responsibility 🙂 LOL!

  26. What about when Google lists a site that has ‘code’ to do something and the code is poorly written and the end result is your PC blows up.

    I’d like to know how to code that, seriously. I know a lot of stupid people who really shouldn’t have access to a PC.

  27. How about pages created that don’t allow use of the back button traping the user. Any way to detect these from the bot level? a quick example is, Not a bad site, but uless you land on an inner page the Back” is disabled.

  28. Dave (Original)

    RE: “I’d like to know how to code that, seriously. I know a lot of stupid people who really shouldn’t have access to a PC. ”

    He he. You must some people I know 😉

    Oxford, I just drilled down on dhl and the Back button worked fine.

  29. Is you javascript enabled? is fine its their main global site.

  30. Hi Matt

    I noticed the changes just did not feel like posting them as often when I quote Google in forums I get fed “google also spreads lies etc” couter attack and who needs to put up with others attitudes???

    It is easier I have found to NOT enlighten others than to put up with BS all the time.

    I would also add, I used the no badware post, to get a potential clients to stop his use of such, so thank you.


  31. Dave (Original)

    I guess Lots0 is REALLY reading the Google guidelines now 🙂

  32. It can be tough to define malware/badware/scumware

    “Badware (n): Anything that screws over a computer and pisses off the guy who ends up having to fix it.”

    Start with the jerkwads from Claria and go from there. 🙂

    Seriously, maybe the way to at least get most of it would be to detect the more commonly used driveby installs (Trojan downloaders, popup or interstitial ads containing Javascript prompts, “CONGRATULATIONS YOU’RE OVER 194,345,982ND VISITOR TO THIS SITE” come immediately to mind), and nail those. You’ll probably nail a lot of the small-fry guys in the process, and you’re left with a bunch of little crappy rinky dinky things that would do minimal damage.

    And I’m all for that faking the user agent suggestion. 🙂

  33. Why can’t google just send a virus scanner to each website. If the website comes back + for virus, then kick the website to the curb? And how do some websites still manage to put up a pop up ad even when you have pop ups turned off. That is one of the most annoying things that I fine when im searching the web. And also when you try to click the back button and you can’t escape. All of these pages should be banned.

  34. Tucows is a respected and trusted software vendor, and unlike McAfee has a unbiased review system. There is no reason other than unfair competition that McAfee should be commiting Libel aganist Tucows.

    To understand the situation completely we must look at the soure,

    It’s important to say from the beginning, it would be impossible for almost any company to be any worse than McAfee. McAfee is a known criminal organization, and it doesn’t surprise me in the least that they are attacking online merchants that may sell competing products. McAfee is currently restating 10 years of earnings, because it stoled money from share holders. First by purposely overstating earnings(people are already in jail for this one), and then by back dating options(resulting in the theft of $150,000,000 from share holders).

    I have to ask myself at this point, what gives McAfee the moral authority to attack legitimate websites. When it would almost be impossible for any company to match McAfee’s criminal behavior. There are literally old folks that had their retirement stolen from them by this company.

    Finally, who are these people doing the reviewing? It took me less than two minutes to sign up. There was no verification of my identity, residence, or anything else that would make me accountable for my statements. You can become an advanced reviewer by responding to a couple of more questions, none of which have anything to do with establishing your identity, just your supposed ability. Making it easy for anyone to lie about their resume.

  35. The company “Smart PC Solutions”, a developer of easy to use solutions for the optimization of your PC, has become the victim of a negative rating on Mcafee’s “SiteAdvisor” just like many other sites, most having no idea that they have received a negative review or why. This causes direct financial losses for many companies but in actuality it redistributes money in favor of McAfee, which sells its anti-virus solutions to terrified users who do not delve very deeply into the details and believe the unjustified ratings.

    This is an obvious case of unfair competition via their security software sales promotion by destroying other companies’ goodwill.

    McAfee’s “SiteAdvisor” assigns a color to each site to indicate safe, caution, or warning ratings sometimes based only on average users’ comments (besides other things). A big red cross (warning) definitely states the presence of a virus and/or spyware activity on the rated site. Their motto reads as follows: “Protection from Adware, Spam and Viruses”. Here comes the surprise: the company “Smart PC Solutions” has nothing to do with spreading viruses yet it has received a rating of a Big Red Cross – beware of the virus threat! Most of the software provided on “Smart PC Solutions” is freeware utilities and sometimes users fail to achieve the desired results as two customers’ comments have testified. The majority of the user feedback on “Smart PC Solutions” has been positive and appreciative. Do you know where McAfee placed the comments on the performance of the “Free Data Recovery” utilities? They put them in the “Bad Shopping Experience” section. Amazing! Do you see any logic here? Smart PC Solutions supplies Free Data Recovery software (worth over $50.00) for FREE (as well as many other useful things)! The first question to McAfee is: “What does shopping have to do with the Free Data Recovery products?” Let’s go further and raise the second question to McAfee: “What does a shopping experience have to do with the virus warning rating?” Where did they find products containing viruses or spyware on the site? McAfee misleads or intentionally deceives people, by intimidating them with erroneous negative ratings that could present a case for litigation. Now a question to the readers: “Do you still trust this system of evaluation?” There is an interesting fact that the site itself has a significant number of negative evaluations by users but the site is not marked with even a “Caution”! Draw your own conclusions!

    As the owner of “Smart PC Solutions”, I wrote a letter to McAfee’s legal department requesting a removal of the negative rating, but was astonished by a reply from an official representative of McAfee stating that they put us in the same basket with spyware producers because some public association has rated our FREE anti-spyware solution as “not strong enough” and that it contains advertising of our other products. Information about that was found in one user comment. So what? We have created our own anti-spyware solution for our customers to remove widespread unwanted components. We do not pretend to be the leader in this field. This was not our aim. Here is the third question to McAfee: Since when has market evaluation been entrusted to random researchers and public associations in such an important matter as issuing guilty or not-guilty verdicts! There is not a word about this on McAfee’s site! Recently, several “independent evaluators” have emerged mooching on the anti-spyware market, and it is a well-known fact that they often are consultants for anti-spyware companies so their opinions could potentially be biased towards one developer or another.

    I wonder if many of your users understand that by trusting McAfee, they trust various paranoid evaluators, whose true interests are dubious. When making a deal with McAfee you, as their customers, did not intend to deal with a club of amateurs and individual exterminators. When buying a car, you do not enter into a contract with a club of energy-saving engine fans or a club of some brand-name fans, and you do not allow them to dictate conditions!

    There are millions of sites on the Internet now, thousands are emerging and disappearing every day. As an IT expert and company owner, I am curious as to how SiteAdvisor is able to rate them and update their database in real time since it is an enormous amount of information. They found a simple solution – they rely on users’ comments posted on SiteAdvisor. Users already observe a slow-down in their PC’s operation when the system is installed! As a matter of fact, McAfee bought SiteAdvisor as early as April 2006, and my guess is that the workload will keep increasing. The problem is partially solved by users, who produce ratings and who McAfee relies on. This is too simple and unreliable a solution for such a responsible matter! The fact is that the company labels one site or another as potentially hazardous for it’s own reasons, not on the recommendation of users-evaluators. This is the ultimate truth of the project, which in fact smells like slender. As far as site development is concerned, I have the following perspective: We have an obvious case of redistribution in the anti-spyware solutions market. Where the market disposition does not exist, it is being created. There has been a significant rise in the number of passionate users’ rights activists with a maniac attitude. Their real motivation is very questionable. The problem is largely forged to pump up anti-spyware hysteria, to frighten users and then sell them a “solution” to the problem. This is unfair moneymaking. McAfee earns as much as you lose from users avoiding your site being scared away by fake ratings designed simply to sell them a security solution. If you are not a site owner, you will buy their security solution when you see the fake warning ratings. In both cases regular customers bring their money to the anti-virus company.

    Microsoft, possessing much more powerful resources, does not attempt to evaluate all sites. It simply created an inexpensive and effective solution, Windows Live OneCare, as the market leader ought to. In view of unprecedented success of this solution, McAfee was left with nothing to do but to take up emergency measures to secure its market share. This is a clumsy attempt to retain the vanishing market. With such an approach they will first lose the credibility of software developers like Smart PC Solutions and then the users, who will see their fake ratings and notice how benign sites get negative reviews unfairly.

    One can get a negative rating on SiteAdvisor just for a link to a site that is considered hazardous by them. It would be good if there was a uniform policy for everyone, but unfortunately this is not the case, and the policy is indeed selective. SiteAdvisor does not analyze the context of a given linked site. It is simply impossible to do for the entire Web. The selective policy of Site Advisor is clearly represented by the positive ranking given to a huge social network called MySpace. This social network has over a million user accounts, and there have been cases when spyware developers spread infected video files on the pages of MySpace users. Right in the comments of this site, there is a link to a report by the famous anti-virus company, Sunbelt, about finding infected video files in the MySpace system. Here is a paradox: SiteAdvisor does not take the information of a reputed company into a consideration. In the case of “Smart PC Solutions”, the opinion of a “random observer” is taken into a consideration! A lot of negative responses about the system are given in the comments on the site. The same selective policy is observed with the well-known American software registration service, defamed by SiteAdvisor and marked as hazardous as a result of links to sites not related to the company. There is no single negative user’s comment! All comments read that Plimus is safe for online purchasing!

    In my opinion, SiteAdvisor pursues a selective policy because it fears potential legal actions by big and reputed companies knowing the full truth about its system of rating. I think lawsuits will follow soon.

    It is clear that McAfee has just recently acquired SiteAdvisor, and that there is a need to scare the public, but they have done it at the expense of many small site owners. I think there will be an upsurge of anger from business owners who suffer losses from unfair ratings.


  36. Matt – What are your thoughts on the upcoming effects to AdWords ads? I was suprised when McAfee called to sell me the recently acquired HackerSafe product on the basis that it would “significantly improve click through rates as results of their partnership with Google”. Something here does not pass the sniff test for me.

    ——— Below is from a recent email solicitation ———
    Highlight your site in Google, Yahoo! and MSN search to millions of McAfee users
    • Soon, all McAfee Secure clients will have their McAfee Secure trustmark highlighted in Google and other search results for tens of millions of McAfee users.
    Search highlight example: