If you need a search fix, we just posted a video on the official Google webmaster blog. Essentially I recreated a talk I did for Web 2.0 and posted it online. You can also watch it below if you’d prefer:
and you can also view the presentation slides I used or watch the slides directly below:
I’m making a resolution this year that when I do a substantial (not just Q&A) presentation at a conference, I’ll try to recreate a version of the talk later on for the people who couldn’t attend the conference. I’d like Google to communicate more and more this year, so this is another step to help with that.
Hat-tip to Jonah Stein; he was one of the first people to highlight the phrase “virtual blight” and how that blight makes the net a worse place.
Thanks Matt!!!
Matt, with Google (along with many others) offering disposable Web sites for FREE, don’t you feel like you are battling the right hand of Google themselves. I.e Google has pooped in its own nest?
I have said for many Years, that FREE Websites = a disproportional increase in Webspam.
Imagine your Neighborhood IF unlimited free housing was up for grabs. Now imagine a new free drug being distrubuted to the same Neighborhood called PageRank (PR for short). That should give a peek into the future of Webspam π
You mentioned the Googler that got hacked with a site of static pages.
This is normally done on a shared hosting account. Someone has some popular script running on another account and the way most webhosting companies set up their PHP to save resources (and put more people on the same server) allows someone that can exploit that script to modify all the other websites on that server. It doesn’t matter if they’re in the same scripting language. They just go manipulate or add files.
I’ve seen this happen before, which is why I started putting clients on my own servers. It’s easier to provide a secure environment than to explain why their’s porn on a website you created for someone.
It’s also why I’ve developed my own tools for building websites.
You mentioned that spammers are lazy. That’s not necessarily true. They’re not lazy, they’re very smart. They don’t go spam all these sites. They use scripts that do it. They find a vulnerability in a script that many people use, or a vulnerability in a popular service to do the work for them.
For instance, if they know phpExploitMe 2.1 has a vulnerability they have their crawlers do a Google search for “powered by phpExploitMe 2.1” to grab a list of exploitable sites then go through all those URLs and attempt the exploit.
They don’t usually target a single service unless it is high profile. They know most of their efforts will be blocked or quickly deleted so they go for quanity. Out of 100,000 sites they might deface, maybe a couple hundred don’t get noticed. That’s fine by them.
Blog comment spammers also try to target older posts. If you’re using Blogger.com, unless you check your comments feed or have email alerts for new comments, you don’t know if a new comment shows up on an older post by looking at the dashboard for that blog. So many times they don’t get noticed. Again, most use automated tools.
If you guys want to make Blogger a little easier for users to detect spam, you might want to consider adding a way for users to review comments in chronological order without having to turn on comment moderation.
There are some people that spam manually. (Then there’s also the people that thought they picked a clever domain that used to be owned by spammers and sounds spammy and frequently gets moderated)
I was tailing my logs after upgrading my application. I noticed some spammer from india hitting my page. This didn’t have the telltale signs of an automated comment spammer. They actually managed to get a post past my spam filters and akismet but since I saw him doing it at the time, I was able to delete his post and ad the domain name he was spamming to my filter. Then I watched as he kept trying to post to other blog entries.
It was amusing to watch for a bit because the pattern was definately a human visitor getting frustrated and reloading pages after posting then going back to check on previous posts until they got fed up.
The spam bots usually get caught but the manual ones are a little harder. I’ve had to just block ip address ranges for certain countries because that’s where a lot of them seem to come from.
I also block ip address ranges for popular dedicated server providers since that’s where a lot of the bots come from.
It works for me because I”m not publishing any web services that servers would use and most of my sites have a US centric focus.
Dude. Why cant you just video the actual events and post that? I feel bad you have to re do it.
panzermike, nobody would pay then and pockets would not be lined π
But, I agree, the whole event should be videoed and uploaded and charge $1.00 to view it. I have asked Matt this many times….
Thank you Matt for that very instructive video.
I’m glad you decided to recreate your speech because a video + slides is always much better than slides alone (less boring, more precise) !
Thanks for sharing this. Blight is a new concept for me.
Thanks Matt! Great video!
Its always good for webmasters and SEO’s to learn more about webspam and learn more about what to look for in terms of possible vulnerabilities and to get into the mindset of the blackhats.
Be good to see more of these kind of videos in future.
Matt, this is great! Thank you and Happy New Year! -Steve
Great video. I would have paid for that! π
One thing not related to spam caught my attention in the video. It was a comment about nofollow and that you should not nofollow all external links. I have been nofollowing all my external links. Is that a mistake and does a site get penalized for doing so? I’ve noticed a drop in my pagerank and was wondering if that was the reason?
Now is this the sort of things I would like to see or hear more of from the Webspam Team for 2009!!
Thanks Matt! This helped a lot.
OnlyMe
jonah stein, I’d been meaning to do these blog posts forever and finally drank enough caffeine to push me over the edge. Thanks for pulling the panel together and moderating it.
htnmmo, great comment and a lot to think over. I guess I think that there’s at least a couple types of spammer: supersmart (tends to be bleeding edge and doesn’t mind working hard) and the lazy kind (more likely to want to buy a script or a CD with 10 million email addresses and just run a program).
John, you definitely don’t need to nofollow all external links on your site; I would not recommend doing that. You want to nofollow the links that (say) you don’t want to vouch for, where the links are not trusted, or (if you want to abide by Google’s quality guidelines, which I recommend) paid links that pass PageRank. A much more in-depth answer is http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=96569
That video was awesome!
Here’s a tool I’d love to see Google provide (maybe it does already??):
You mentioned that you could setup alerts like site:mydomain.com viagra. Google should provide a service that sets this up for you. You should just be able to sign up for the “Alert me if there is any bad-neighborhood content on my site” and you guys should take care of compiling the list of bad terms.
I was about to put in an alert for each of these terms:
http://codex.wordpress.org/Spam_Words
But why should I have to do this? It should be included in the webmaster tools IMHO.
Again thanks for the video!
Hi Matt,
I have observed that my last few comments in your blog were not accepted. I guess the reason was that I used a link in the URL option that we have in the comment posting form. I will now comment without linking back to my site and I hope my comments can now get approved.
I want to be a part of this great blog or community and I wish Matt accepts this comment.
Anyway, the post if great and I already posted a comment here which was not accepted.
Is there any other reason for my comments to be not accepted? Any reason other than that I linked backed to my site?
Thank You and I hope this comment get approved.
Matt you are doing a great job in this blog.
Regards
Hey Matt, how about getting those gmail guys to get us an “unlabeled” default label added… with 20+ labels, and sub-lablel’s it’s impossible to – everyone!!
Come awn man π
Matt,
Thanks for the great info, especially the badge recycling bit. One thing I have run across recently is finding do followed comments in reputable blogs that have been spammed with adult terns etc. How do search engines view this and how is it tackled since the domain of the actual company/blog is reputable but some of their links are spam?
Thanks
Matt – great idea!
A transcript would be particularly nice though, as that’s much faster to read.
Oh Matt – if you don’t mind me taking you off on a slight tangent, we’ve been having problems with the new Sitemaps submission system, which is refusing to accept an RSS feed (which validates in both feedvalidator.org and the W3C feed validator with no errors or warnings). Am happy to share URL via email.
i need to remember to start making some recycle boxes π
thanks matt, great vid.
BUT if you do trust a site and they do wrong, you are lumped into their “bad neighborhood”, correct? I’ll take no answer as a “Yes”.
Matt, I just loved the video and really liked the way you explained the different ways spammers can attack a site and the concept of virtual blight. Seems I can never get enough of your insights. Thanks Matt.
Matt,
Thanks for doing the vid. Really useful.
Thanks very much π
Paul
Matt:
What an informative video; I am not a webmaster and found the information presented ‘basic’ enough that even a Marketer could understand. Thank you for taking the time to represent this information.
i am concerned about directories, if someone running a directory and have a lots of links going out. do not you think it is best to use nofollow to all of them? nofollow is non trusted link, does it make any difference to pointed location.if a website have a lot of nofollow links point towards its, does google take any action against it or just ignored it?
Thanks for the slide Matt, also i would like to tell you that you are a good artist π
Thanks for the nice Video (btw. I like your artful slides).
Talking about losing a domain or more exactly about “Expiration Date”.
I renew my domain every year so the Expiration Date is just one year away. Now, on so some shinny SEO Blog, I read that this could affect my rankings because it is more likely that the Google algorithm puts more trust in a domain paid 3 years in advance than a domain that’s only paid for one year. My webhost hasn’t this feature to pay in advance, should I move?
Thanks for the clarity. I have so many people pushing for the email marketing but each and every time I shoot it down simply because you never know where the names actually came from, They always tell me that they are opt in and are looking for our products but they never seem to look me in the whites of my eyes when they say that. Your video lets me know that spammers are alive and well and I will stay away for dark short cuts if you know what I mean – seems like nothing but heart ache there.
Thanks for this video very interesting ! but next time can you put english subtitles ? because you articulate very well but sometimes i have missed some words π
Have a good day for all !
Some excellent reminders in here and video is the way to go. No way would I have read through an article with all this information, but to see it live is really impactful.
Thanks Matt,
Terrific video, I especially liked the landing page examples on slides 6 and 7 and the hacking example on slides 10 and 11. The examples really drive it home just like the examples you gave us at the SES San Jose convention.
Great video on blight and it summed it up in a simple enough manner that even I could understand. I am glad you do not need to nofollow all external links from our site, thanks for the clarity on that as well Matt.