Archives for September 2008

Twitter added nofollow to “www.” links in their Bio field

Yesterday John Battelle emailed me to ask about Rae’s post. This will be a little inside baseball to some people who don’t live and breathe search and Twitter, but I figured I’d take what I emailed to John, add some pictures, and post it here. Here’s the email:

Sorry for the delay in replying; I’m really behind on email because I’ve been talking about Chrome this past week.

The short answer is that back in July I saw this post http://www.davidnaylor.co.uk/twitter-backlink-tip.html . David Naylor was pointing out that Twitter intentionally nofollowed links in the “Web” part of a Twitter profile, but that you could embed a link in the “Bio” field that would flow PageRank:

Excerpt from DaveN post about twitter

Dave’s blog is read by a bunch of SEOs, including quite a few blackhats, so it was pretty clear to me that Twitter might get hit by spammers who jumped onto Twitter just to get PageRank, or by bots who signed up a ton of accounts automatically trying to get links.

I wasn’t sure of Evan Williams’ email address, so I took my best guess at two of Evan’s emails and dropped Evan a quick note pointing out Dave’s post and that spammers might start attacking Twitter soon because of this. Because I wasn’t sure of Evan’s email, I also sent Evan a Twitter saying “@ev, dropped you an email about (the post that Dave did)” That was all in July, and I forgot about it.

Evidently just in the last few days, Twitter changed that Bio link to a nofollow link. A few thoughts:

– My guess is that spammers have started to attack Twitter more, probably at least partially to get links/PageRank. In an August post at http://blog.twitter.com/2008/08/turning-up-heat-on-spam.html , Biz Stone mentioned that Twitter was hiring their first full-time spam fighter. For a company of ~25 people, a full-time spamfighter is a lot of resources. Evan/Biz have seen how dedicated spammers can hurt users’ experiences at Blogspot, so I imagine that they want to keep Twitter really clean and lock out any spammers early.

– It could be that as part of the process of looking at spam attacks on Twitter, the Twitter team asked “What are the incentives for people to spam Twitter? Are we leaking links to spammy sites anywhere?” If they were asking those sorts of questions, it makes sense if they decided to nofollow the Bio link to prevent spam accounts from attacking Twitter.

– I dropped Evan an email about Dave’s post just as a heads-up in July in case he wasn’t aware of it, but people have been talking about gaming Twitter for links even before that, e.g. http://nickwilsdon.com/do-you-want-some-followe-links-from-twitter/ :

Excerpt from nickwilsdon post about twitter

and there’s another post someone did in March called “Gaming Twitter for Thousands Of Backlinks”:

Excerpt from nickycakes post about twitter

By the way, I totally support if Evan wanted to lift nofollow for real users in some way, but I figure that Twitter probably wanted to protect themselves against spam as a first step. Given that a month or so after I dropped them a note, Twitter hired a full-time spam person, I’m not surprised if Twitter was starting to see more spammers show up and wanted to take strong action to push back on spam as a first step–if Twitter got gummed up with spam that would be bad for everybody. Perhaps down the road they’ll look at ways to keep flowing PageRank to real users while not opening themselves to a spam attack. I would imagine that they have pretty good signals that would let them separate (most) real users from (most) bots/spammers. So Twitter could take steps such that most users would still get PageRank by removing the nofollow on sufficiently non-spammy users.

Best wishes,
Matt

That’s what I sent to John by email, minus the pictures.

Google Chrome user agent

It’s easy to find out what Google Chrome’s user-agent is. Using the same trick as I did with the iPhone, I searched for phpinfo HTTP_USER_AGENT in Google Chrome. Click on one of the results and search for HTTP_USER_AGENT on the page. Here’s the image that I see:

Google Chrome useragent

My exact user-agent is

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.A.B.C Safari/525.13

So “0.2.149.27” is the current version, but that will change over time. I wouldn’t be surprised if the “525.13” value for WebKit changed too.

Update: atul points out in the comments that using “about:” or “about:version” in the address bar works well too.

Update: Someone pointed out Google’s official documentation about the Chrome user agent as well.

Google does not want rights to things you do using Chrome

Alright, I’ve got another conspiracy theory misconception to dispel. 🙂 After reading through the Chrome Terms of Service, some people are worried that Google is trying to assert rights on everything that you do on Chrome. From one example story by Marshall Kirkpatrick:

The terms include a section giving Google “a perpetual, irrevocable, worldwide, royalty-free, and non-exclusive license to reproduce, adapt, modify, translate, publish, publicly perform, publicly display and distribute any Content which you submit, post or display on or through, the Services.” That seems pretty extreme for a browser, doesn’t it?

I knew that Google didn’t want to assert rights on what people did using Google Chrome, so I asked the Chrome team and Google lawyers for their reaction or to clarify (probably several other people pinged them too). Here’s what I heard back from Rebecca Ward, the Senior Product Counsel for Google Chrome:

“In order to keep things simple for our users, we try to use the same set of legal terms (our Universal Terms of Service) for many of our products. Sometimes, as in the case of Google Chrome, this means that the legal terms for a specific product may include terms that don’t apply well to the use of that product. We are working quickly to remove language from Section 11 of the current Google Chrome terms of service. This change will apply retroactively to all users who have downloaded Google Chrome.”

I hope that addresses the concerns that I’ve seen on a few places around the web. I appreciate that people pored through the Chrome license to find anything that looked unclear and then raised concerns so that Google could respond.

Update: I did this post quickly because I had to go to a meeting. Coming back to read what I wrote, I think I was too strident (both here and commenting a few places around the web) and I apologize for that. As a long-time Googler, I knew that Google wouldn’t want rights to everything that somebody did in Chrome. It also seemed like there had been incidents like this in the past and they always got cleared up quickly.

But it was clearly a mistake on Google’s part to include that language when it shouldn’t have been there, and I should have been grateful to the people that pointed it out. Instead of getting snippy with people, my reaction should have been more along the lines of “Oh crap, I don’t think that’s intentional. Thank you so much for noticing that and pointing it out. I’ll see if we can get an official clarification or reaction as soon as possible.” I apologize for that, and I appreciate the people who push Google to be better.

Update, September 3rd 2008: Earlier today, Google changed section 11 of the EULA to read as follows:

11. Content license from you

11.1 You retain copyright and any other rights you already hold in Content which you submit, post or display on or through, the Services.

I think that text is much better.

Update, September 4th 2008: Google provides a full explanation of how this happened and how they’ve corrected the Google Chrome End User License Agreement (EULA).

Answers to common Google Chrome objections

I spend a fair amount of my time debunking misconceptions about Google. So when I found out that Google Chrome was going to be released, I put on my thinking cap about what objections people would throw out about Google Chrome. Here are the questions that I came up with, along with my personal answers. I want to stress that this is my personal blog, so even though I believe all of this is accurate, these are not official answers.

Q: This browser is going to have AdSense hard-coded into a browser frame that I can’t delete, right?
A: No, there’s no ads built into Google Chrome at all.

Q: Hmm. Well, I bet you hard-code Google as the default search engine, don’t you? I’ll bet you can’t even select other search engines!
A: Prepare to be pleasantly surprised. By default, Google Chrome imports your default search engine from your default browser. When you first install Google Chrome, you get a dialog box listing the default search engine with buttons to keep or change the default search engine. In addition, you can change the default search engine at any point. There’s a couple easy ways to do it. For example, you can right-click on the Omnibox/address bar and select “Edit search engines…”:

Edit search engines

Then you can select from any of the major search engines (Google, Yahoo, Live, or Ask), or add a new search engine url:

Other search engines

Another way to change your default search service is to click the Chrome menu (it looks like a wrench), select “Options” and then in the “Basics” tab make your choice for the “Default search:” option.

But wait! It gets even easier to search on other engines. In the Omnibox/address bar, you can start typing “live.com” or “yahoo.com” until the Omnibox offers live.com or yahoo.com as an autocomplete option–for me, I only have to type the characters ‘l’ or ‘y’ to bring up these suggestions. Once you see “live.com” show up as an autocomplete option, just hit the Tab key and you will be offered to search for a query on live.com. So I can search on live.com just by typing the letter ‘l’, then pressing the Tab key, then entering my query. So it takes just 2-3 keystrokes to run your search on any search engine! Why not always two keystrokes? Well, should the letter ‘a’ bring up ask.com or amazon.com? Sometimes you have to type a few keystrokes to tell Google Chrome where you want to search.

Just in case you’re curious, the auto-search-discovery is triggered by search fields on the root page of the domain. I don’t have a search field on the root page of my domain (www.mattcutts.com), so if I go to www.mattcutts.com/blog/ and do a search there, Google Chrome won’t learn the autocomplete-with-the-Tab key trick. Instead, I would need a search box on www.mattcutts.com for the Tab key to learn to search (it’s also good to use standard POST HTML submits instead of anything too fancy like JavaScript). If that sounds too hard, there’s an easy solution for you: Google Chrome supports OpenSearch discovery too.

Lots of people would expect Google Chrome to hard-code searches to Google. I’m glad that Google Chrome
– tries to do something reasonable on installation based on your default search engine
– makes it easy to change the search service to any major search engine
– makes it trivial to run a query on different search engines with just a couple keystrokes, without lifting your hands off the keyboard, and without even changing your default search service.

And the criteria are clear and easy for a site to use this Tab-to-search ability in Google Chrome. Danny Sullivan has also written about how Google Chrome’s search service doesn’t bias toward Google.

Q: Okay, but this browser is tracking everywhere I go and sending that information to Google. Go on, deny my conspiracy theory!
A: Not only do I deny it, but in this post I broke down every piece of communication that I know of between Google and Google Chrome. The browser is also completely open-source, so by all means please verify for yourself or modify the browser however you want if you don’t like what Google Chrome does.

Q: Another browser? Geez, I’m a webmaster/search engine optimizer/front-end programmer and I don’t want to worry about another browser.
A: Google did not add another rendering engine. Google Chrome uses WebKit for rendering, which is the same rendering engine as Apple’s Safari browser, so if your site is compatible with Safari it should work great in Chrome. Personally, I do think creating clean code that validates and works on many different browsers will be an important skill for webmasters and web designers. These days a smart site owner thinks about how their web site looks to all browsers, from Internet Explorer to Safari to Opera to an iPhone.

Q: Clearly launching a web browser means that Google is losing its focus on core search quality, right?
A: As a member of Google’s search quality group, I can utterly and forcefully debunk that idea. 🙂 Google has more engineers working on core search quality and in-depth search research than ever before, and Google’s search quality group remains just as focused as we ever have been. (By the way, our search quality group has been opening up with more blog posts on the official Google blog recently.)

Q: Does this mean that you don’t love/support Firefox or the Mozilla Foundation?
A: This is a personal answer, because I don’t know Google’s official messaging. I *love* Firefox. I write about it all the time; heck, I wore a Firefox tattoo for a day earlier this year. Firefox was one of the first browsers to offer tabbed browsing, and Firefox’s extensions and flexible preferences let you do amazing things. But competition is good for everyone, because it will push web browsers ahead even faster. Google Chrome advances the state-of-the-art in web browsers in several new ways (read the comic book to see why). And since everything in Google Chrome is completely open-source, if we come up with a good technology such as V8, which is a blazing-fast, from-the-ground-up implementation of a JavaScript virtual machine, Firefox or anyone else is welcome to yoink it and use it themselves. It may be that innovations from Google Chrome are quickly folded into other browsers. If so, that’s great! That means that people will enjoy a faster, safer, more stable experience on the web.

For a more eloquent take on this issue, read the blog post that John Lilly, the Mozilla Corporation CEO did. While mentioning that Mozilla and Google are different organizations with different missions, Lilly also reiterated some of the ways that Mozilla and Google have worked together:

I think both organizations [Mozilla and Google] have done much over the last few years to improve and open the Web, and we’ve had very good collaborations that include the technical, product, and financial. On the technical side of things, we’ve collaborated most recently on Breakpad, the system we use for crash reports — stuff like that will continue. On the product front, we’ve worked with them to implement best-in-class anti-phishing and anti-malware that we’ve built into Firefox, and looks like they’re building into Chrome. On the financial front, as has been reported lately, we’ve just renewed our economic arrangement with them through November 2011, which means a lot for our ability to continue to invest in Firefox and in new things like mobile and services.

Q: But I can’t install extension X! Google Chrome is dead to me if I can’t use extension X!
A: Then you’ll have to use another browser for a while. Google Chrome currently doesn’t support browser extensions (it does support plug-ins, such as Flash). I’m sure that extensions/add-ons are something that the Chrome team would like to do down the road, but the Chrome team will be a bit busy for a while, what with the feedback from the launch plus working on Mac and Linux support. I’d suggest that you give Google Chrome a try for a few days to see if enjoy browsing even without extension X. A lot of really cool extension-like behaviors such as resize-able textareas and drag-and-drop file upload are already built into Google Chrome.

Q: Speaking of Macs and Linux, why no Mac/Linux support yet?
A: Again, this is just my personal take. I work on search quality and have no particular insight into this question. But personally, I think supporting Windows first is the right decision. My house has more Macs than Windows machines, but I am not a typical user. Windows machines are still the majority of the market, so it makes sense to start there. And I think it makes sense to launch the Windows version now so that the project starts getting the feedback that will make the Windows, Mac, and Linux versions of Google Chrome better.

Q: This is going to be some buggy, crashing piece of beta download, isn’t it?
A: No, I don’t really think so. Google determined one million pages that users are likely to surf to. Then a piece of software known as “ChromeBot” ran a torture test by loading those one million pages that we crawled for every new build of Google Chrome. This is a smart testing methodology because this real-world torture test quickly highlighted which bugs were most important to fix and helped determine priorities for the most important bugs. So Google Chrome has already been automatically “fuzz tested” on tons of web pages. Google Chrome is very robust against anything the web can throw at it. If I had to guess where the browser might crash, it would be because we haven’t seen Google Chrome run across all the weird, wild hardware that runs Windows. But I’ve been using Google Chrome for months with essentially no crashes; it’s been rock-solid for me.

Q: Google has some selfish motivation for doing this, right? I’m sure that there’s some angle here–there’s gotta be?
A: Here’s my personal take. If people like and use the web more, that’s eventually good for Google because they will do more searches. Therefore it’s in Google’s interests to make the web better, more accessible and more useful. And Google Chrome does that by making the web faster, safer, and more stable.

Q: Will Google stop actively working on Google Chrome and let it stagnate after a few months?
A: Again, just my personal take: I doubt it. Google Chrome feels at least as important to me as Gmail. And in all my interactions with the Chrome team, it’s clear that this project has been a huge labor of love. It’s been a labor of blood, sweat and tears as well 🙂 but mostly love. I have a huge amount of respect for the Chrome team, because they’ve accomplished something amazing and created a product that anyone would be proud of.

Update, Sept. 3rd 2008: Forgot one:
Q: Dude, this anonymous commenter said that Google claims that they own everything you touch when you run Chrome! Should I be worried?
A: No, of course not. I debunked that misconception last night in a Mashable comment and this morning in a ReadWriteWeb comment. Google does not want to claim the rights to everything you surf or do in Chrome, just like we didn’t want it the time before with Google Docs. 🙂 I’m sure that other Googlers will clarify that point more officially. It is good that people pore through the license and ask these questions though, because if something looks worrisome then we can use that opportunity to make it more clear.

If this post helped you feel better about Google Chrome, you can download it at http://www.google.com/chrome and try it out.

Preventing paranoia: when does Google Chrome talk to Google.com?

For better or worse, my blog is popular with the Google conspiracy-theorist demographic. 🙂 I knew that as soon as Google Chrome launched, some readers would ask tough questions about privacy and how/when Google Chrome communicates with google.com.

So I decided to tackle this issue head-on. I talked to the Chrome team to find out if there’s anything to worry about. The short answer is no. For the long answer, read on.

If you’re just surfing around the web and clicking on links, that information does not go to google.com.

If you are typing a search or url in the address bar, Google Chrome will talk to the current search service to try to offer useful query/url suggestions. I love this feature, but you can turn it off. Right-click in the Omnibox/address bar and choose “Edit search engines…”. Or click the Chrome menu (it looks like a wrench), then Options->Basics and then the “Manage” button. Either way, you’ll see this box:

Other search engines

Uncheck the checkbox at the bottom of the dialog box that says “Use a suggestion service to help complete searches and URLs typed in the address bar.”

By default, crash reports and other anonymous usage statistics (e.g. which features are used most often) are not sent to Google. The Chrome team would love if people opted-in to send crash report data though, because it would improve Chrome for everyone. To opt in, click on the Chrome menu (it looks like a wrench), then click “Options.” On the “Under the Hood” tab, check the box that says “Help make Google Chrome better by automatically sending usage statistics and crash reports to Google.” You can read more about this opt-in option on this support page.

I believe if Google Chrome sees a very short, stock 404 page (less than 512 bytes), it talks to Google in order to try to suggest other possible pages and options. My understanding that this is the same underlying technology that I talked about earlier this year. I think if you have a helpful 404 page (> 512 bytes), Google Chrome doesn’t modify that (this 404 page isn’t changed for example), but Google Chrome does try to help with very short/unhelpful 404 pages. If you still don’t like this feature, you can turn it off. Go to Chrome/Wrench menu->Options->Under the Hood and uncheck the box that says “Show suggestions for navigation errors.”

By the way, just as a quick plug: if you as a site owner want to improve your 404 pages, Google offers a nice snippet of JavaScript (well under 20 lines) to offer this functionality to site owners. See this 404 page blog post for more info.

Google Chrome checks for automatic updates every 25 hours. Other modern browsers check for updates as well, e.g. to plug security holes. Given today’s sometimes-hostile web, I think checking for updates like this is a very smart choice.

Every 30 minutes, Google Chrome downloads a list of 32-bit url hashes of urls thought to be dangerous (malware or phishing). That is a download of data from google.com, not to google.com. As you surf around the web, if you happen to hit a url whose hash is in the dangerous list, the 32-bit hash is sent to Google and Google replies with a full 256-bit hash of the dangerous url in question. Not only does this happen very rarely, but Google Chrome doesn’t send a url to Google, it sends a url hash, so Google doesn’t learn the url from this exchange. By the way, this is essentially the same protocol that Firefox 3 uses to protect its users from malware/phishing urls as well.

Update, September 5th 2008: A friend pointed me to this nice public description of Google’s Safe Browsing protocol in case you’re interested in more details.

When you choose your language in the user interface, Google Chrome downloads a spellcheck dictionary. Again, that is a download of data from google.com, not to google.com.

To the best of my knowledge, this is the only communication that happens between Google Chrome and google.com. I thought it would be better to write down all the communication that happens so that people wouldn’t invent conspiracy theories. As Louis Brandeis said, “Sunlight is the best disinfectant.” Luckily, you can double-check me because the browser is open-source. I hope this helps in case anyone has any privacy-related questions about Google Chrome.

Update: David Pogue writes this of Google Chrome in the New York Times:

Will Google ensure that its own services run better in Chrome than in other browsers? Is this part of Google’s great conspiracy?

That’s a no and a no. Chrome is open-source, meaning that its code is available to everyone for inspection or improvement — even to its rivals. That’s a huge, promising twist that ought to shut up the conspiracy theorists.

That’s a good way to put it.

css.php