Udi Manber interview

Eric Enge posted an interesting interview with Udi Manber of Google (Udi is a VP of Engineering at Google). Udi mentions that the goal of personalization is to improve overall search relevance (not to make SEO harder). Udi also talked about trying to make Google’s search algorithms more elegant:

[W]e have projects that their sole purpose is to reduce complexity. A team may go and work for two months on a new simpler sub-algorithm. If it performs the same as the previous algorithm, but it’s simpler, that will be a big win and people are encouraged to do that, and some of the improvements that we are making over time are those kinds of improvements.

It’s also nice to get more backup on the role of humans in Google search quality:

Eric Enge: What about the notion of the role of humans in search? There is another post by Matt Cutts where he talks a little bit about ways that Google is using human input. People can say they don’t want to see a particular site in their results, or they can vote on it if they have a Google Toolbar installed, and that sort of thing. Do you see Google looking for more ways to, scalable ways obviously to take advantage of human input?

Udi Manber: Absolutely. As Matt has said, we have done it from the beginning. If a website points to another website, that’s a signal, that’s a signal from a user and we use that signal. If somebody says they don’t like a particular search result that’s a signal. So, we’ve been using that for a long time, and we are working on new ways of using it.

You might also have seen that Dare Obasanjo attended Google’s Scalability Conference and wrote up an interesting session with Marissa Mayer:

Q: How do they tell if they [Google] have bad results?
A: They have a bunch of watchdog services that track uptime for various servers to make sure a bad one isn’t causing problems. In addition, they have 10,000 human evaluators who are always manually checking the relevance of various results.

At this point, if you think that Google doesn’t try to utilize human feedback in scalable, robust ways then you need to adjust your mental model. 🙂 See some of the previous discussion for more background.

40 Responses to Udi Manber interview (Leave a comment)

  1. I hear a heck of a lot of concern from SEO’s about the whole personalization issue.

    I think fundamentally it comes down to this one thing – people are very concerned that the searches they have worked so hard to dominate under the ‘old’ system will fall by the wayside under personalization.

    The simple fact that I see people failing to realize though is that the current system fails to adequately measure search INTENT. Sure, the algorithms seem to do a pretty good job at implying intent from search terms used (ie buy blue widgets is pretty likely to return blue widgets sales sites) but what about searches where the user just types ‘blue widgets’?

    How do you know whether that person wants to know about how blue widgets work or where s/he can buy blue widgets.. indeed, the searcher might even be looking for a site about their local footy team ‘the blue widgets’… it is so hard to measure intent.

    Because of this, in my opinion under the current system, the SEO often tends to try to be everything for everyone – you need to take a scatter gun approach.

    Sure, you could try educating searchers about how to search ‘better’ – but with personalization (if it’s done right) I see some huge possibilities, from the user and the site owner’s perspective.

    The user perspective is a simple one – more relevant results – If I’m a big football lover I’m likely to read about my beloved blue widgets team rather than having to trawl through loads of irrelevant results first off. Also, as a user, I find myself being more productive – using that back button a lot less.

    From the website owners perspective, particularly for a sales website, I should (again, if it is implemented right) get a much better conversion rate. Overall traffic may drop, but my sales will remain static (or even improve dramatically because I no longer have to compete with unrelated content). That has to be a positive..

    But this is sure going to be a rocky road with lots of debate – I know there are other equally passionately held perspectives .. I can see it being a challenging time for Google and SEO’s alike – the changes are going to be significant – perhaps the personalization approach may seem a bit paternalistic to experienced searchers.

    doc

  2. Yes, I agreed.Sometimes needs own to realize!

  3. Thanks for more behind the curtains kinda posts even if you’re just pointing us elsewhere. That algorithm must be an evergrowing monster (or many monsters).

    Yea, i was scared of personalization at first too, but anything that might shake up seo is probably a good thing 😉 .

    But really my concern about personalization, especially with regard to TV or radio ads is the loss of the communal experience. One great joy of tuning into any media is to see what other people are seeing and finding out what is going on. Even, in a small way, getting ads or search that are already tailored to my needs, gives me less insight into what others are seeing. And in a very direct way it prevents me from communicating with others as I can’t even direct them to do the same search. It will take a pretty slick solution to achieve the right balance.

  4. This is my own personal rant with search.

    My sites are doing very well in google thanks to following exactly the advice I have gotten at webmasterworld, Matt himself, pubcon and Google webmaster guidelines.

    The specific thing that I would most like to see improved is the serps really truly reflecting user experience. For example, I can add a page to my site such as the one we wrote about Danactive which shows up very high in the serps. The value of the page for the user is kinda o.k., but I think since the page is fairly keyword content rich it shows up well?

    On the other hand, we created a couple of cartoon books that are about 80 pages long each, and have very little content on each page, and cartoons. They are some of the best pages of content in our industry. Users love them, and they do have quite a few links.

    The cartoon books do not show up very high in the serps I think mostly because there is very little readable content on them. On the other hand if humans read them they love them.

    I love the idea of google toolbar keeping tabs of what people read, how long they stay there, and having voting or rating buttons on the toolbar. Perhaps we could even have google buttons on our pages where people could rank them, and have their ip address caught, and google receive the ip address and data?

    I am sure you guys have thought of all of these, but I am just putting in my vote!

    Matt, I also have never seen you make any comments about comment length specifically, and I know that I and a few others leave longer comments. Would you prefer we/I keep them a bit more abreviated or are these o.k. (Also I am dying to hear more about wordcamp and your thoughts! 🙂

  5. Dr David, I wonder whether the (relatively recent) acquisition of urchin (now known as Google analytics) might have been partially driven by the desire for that kind of data – altruistic intentions are cool but better data for Google is even cooler 😉

    doc

  6. It sounds to be that “signals” such voting using the google toolbar, returning to the same site repeatedly with the google toolbar installed, or reporting a site, are all factored into rankings (or play some role). I have been saying this for a while, and nobody seemed to agree. I don’t know if Udi’s comments validate this notion or not, but it seems likely that is what he is saying.

    While I would be a little scared to think that humans could manipulate rankings by manipulating things like these items. I also feel that it is data that cannot be ignored, as long as additional steps are taken to look for data irregularities.

  7. Google is updating with new algos.

    What is interesting about this update is that it is giving less of a priority to reference/ informational sites and is being more democratic about backlinks not to focussed on the trustrank of the baclinks or the site.

    Could this have been a result of the persistent posts by SearchEnginesWeb on their this blog about the overemphasis on certain informational sites in the SERPs, and the TrustRank over emphasis in BackLinks??????.???

    For example: Look at the Top 20 results for SEARCH ENGINE OPTIMIZATION: ( a very SEO’ed term)

    http://www.google.com/search?as_q=search+engine+optimization&hl=en&num=20

    Even Google’s Webmaster Guidelines which stayed at #2 for months has fallen to #12 – as well as another very white hat SEO who fell from page one after three years.

    Several SEOs who do engage very aggressively in link exchanges have actually risen a few spots after months of staying at the bottom of page one.

    There are also several changes in the SERPs for the term HOME – which is a good way to measure how effective INTERNAL BACKLINKS are on the SERPs

    http://blogoscoped.com/forum/102119.html

  8. Google is updating with new algos.

    What is interesting about this update is that it is giving less of a priority to reference/ informational sites and is being more democratic about backlinks not to focussed on the trustrank of the baclinks or the site.

    Could this have been a result of the persistent posts by SearchEnginesWeb on their this blog about the overemphasis on certain informational sites in the SERPs, and the TrustRank over emphasis in BackLinks??????.???

    For example: Look at the Top 20 results for SEARCH ENGINE OPTIMIZATION: ( a very SEO’ed term)

    http://www.google.com/search?as_q=search+engine+optimization&hl=en&num=20

    Even Google’s Webmaster Guidelines which stayed at #2 for months has fallen to #12 – as well as another very white hat SEO who fell from page one after three years.

    Several SEOs who do engage very aggressively in link exchanges have actually risen a few spots after months of staying at the bottom of page one.

    There are also several changes in the SERPs for the term HOME – which is a good way to measure how effective INTERNAL BACKLINKS are on the SERPs

    blogoscoped.com/forum/102119.html

  9. dockarl, I agree with you in general and I disagree with you in my specific case (and I know at least one other person agrees with me on my specific case).

    My specific case is that I’m concerned that any personalized data will be used in aggregate to generate default search results, which subjects them to mass manipulation by blackhats who may want to register multiple Google accounts and “personalize searches” that put their own sites at the top. As long as Google doesn’t do this with the default results, I’m fine. In other words, personalization needs to be user-specific and the non-personalized results need to remain as is.

    As long as personalization is kept separate from non-personalized results, I’m all for personalization, as it forces webmasters to worry more about the user experience.

  10. Heather Paquinas

    Personalization is yet another reason to use hittail, and build a multi-search engine long-tail marketing strategy.
    It’s great for longtail ppc campaign ideas too.

    Matt, your school got hacked for ‘buy viagra’:
    http://www.uky.edu/CampusGuide/
    http://www.uky.edu/CampusGuide/uk_navbar.js

  11. Does anyone know what Google means when it says that something is a “Supplemental Result”?

    i’m help me.

  12. As a die hard search fanatic I think personalization is a must in the search evolution process. Most SEO’s automatically freak when change is on the horizon , especially since most are not willing to adapt and are used to the same thing over and over again.

    I am even more excited to see more video and images mixed in with the results as of late. Cool stuff! I am urging my clients to get involved with the new web.

  13. Matt

    I would at any given time prefer serps based on algos and sub-algos than serps based on personal judgements of 10.000 school kids in Bombay and elsewhere hired for $10 a day as “human evaluators”.

    All that noise from the so called “human-edited serach engines” of involving human factors more and more hasn’t been justified yet by reliable large survey indicating that public Joe prefer more human-factors affecting the serps.

    IMO, the success of Google so far has mostly been due the intelligence, effectiveness and “nutrality” of Google’s algos, not because of the evaluation of school kids 🙂

  14. If I can take up a few more inches of screen space.

    As a user I would love to flip a few switches on my google toolbar to select the flavor of sites I am looking for tonight. Maybe heavier on images, a little cutting edge, not referenced with scholarly journals, with no worse than an R rating.

    Then on another day, it might be flipping the switches so Google delivers me only articles referencing peer reviewed research articles to back them up, text only, with a G rating.

    Then when looking at movies, I could flip to a search engine totally personally edited by tattooed ex punk rock skaters who also think Buckaroo Banzai is one of the greatest movies ever made. I can see how subparts, teams, etc. of google can be cross referenced by the user, so that we could adjust the weight of different aspects of the algo ourselves.

    So for certain things we could put more emphasis on back links, and for other searches put the emphasis more on the freshness of content.

    Cmon, we’ve got to raise the bar high for the google plex, otherwise they are going to run out of things to do there!

    I am sure many SEO’rs would not like this, because there would not longer be one algo to try to figure, but for the user, the more customization, or the option of having no customization would be awesome!

    One users opinion!

  15. I look at our clients and most of those clients are business men that know how to use a computer, but aren’t the type of google users that will use a google toolbar or report that this and that search is not relevant. They use google. But in a passive way.

    And the interests of these business men is not the same as a group of teenagers or young adults, that still go to school, or young employed individuals, that work in the IT industry. I think that younger people have such a natural relationship with the internet. It’s part of their life. And they will be in the front line to help google improve results.

    How do we help these business men and other social groups that aren’t internet lovers like ourselves to participate in this personalization of google? Is google going to create tutorials and videos explaining the importance of that personalization? Or are these groups excluded from the process, due to their own limitations? And how will this affect them?

  16. Indeed involving the human factor in evaluating sites implies introducing a very subjective component to a highly objective tool. But it is not a black and white situation and I agree with Dr. David Klein opinion. If we are talking about customization why not go all the way and let the customer decide how much personalization to use and when, in the process of finding the information he/she is looking for.

  17. Fruit Helmet Cat

    Dear Matt,

    Google elegant…. now that’s sexy. Hey, how can I be a human evaluator? I am just a little furry but I love my Google and can type with both paws. – Fruity

  18. Matt – If you report someone gaming Google and give is a few weeks and see the person still having his/her way in your algorithms does this say that it is ok to do the same type of SEO to get the same results?

    It gets a little tiring out here waiting on the sidelines…

    Please revisit the guidelines for multiple site ownership and linkage, lots of questions on the subject going unanswered in Google Groups and blogs.

    Thanks

  19. Dr. Klein, a couple of points:

    1. On using more time-spent and similar user interaction variables — not so fast. This is still search, and SERP’s must be in relation to a query involving keywords with intent and meaning. In your comic-book example, if there is very little readable text, then the search engine may well have the ability to measure the user interaction with the content, but that doesn’t tell us what it’s about. What queries would you expect it to rank well for? So here the only solution would be for the SE to be able to read graphical text, which shouldn’t be terribly difficult, but this still doesn’t translate well into determining the meaning of other kinds of non-textual content.

    2. Personalizing your own algorithm? You mean something like this?:
    http://www.traffick.com/article.asp?aID=158
    http://www.traffick.com/2005/05/yahoos-slider-makes-commercial-vs.asp

    Music to my ears… I’d love to see them do more stuff like this. Apparently there is no mass market for it, however.

  20. In my opinion, personalization is a big plus for SEO. Not only does it help even the playing field by making their be potentially multiple #1positions in the SERPs, but it also, in theory, should make targeting the right searcher much easier.

  21. Dockarl said:

    How do you know whether that person wants to know about how blue widgets work or where s/he can buy blue widgets.

    Google does already know (well kinda), for example:

    I sell rain barrels, I can see in my stats that people search for the phrase “make a rain barrel” often so guess what shows up now in the search results for “rain barrel”? A old outdated .pdf file with a broken link and directions on how to complete this task. (For me to offer directions would lose many sales)

    The issue:
    I have found that those who search for “how to make” are actually happier having me make a barrel when they find my site so “how to make” is actually a search done by people who know nothing about my product but I do not rank for “how to make”. Google also has the search data I gave them from using analytics on my site for a year or more (you are welcome, I didn’t even use it, it was for you). BUT does Google know that “how to make” actually means, “tell me about”? Nope!

    Now advance this thought a year and you got marketers hitting Google with HUGE amounts of media to show up in many places, (which is ok) but you can not measure this type of intent.

    The above is also an example of how personalization threatens “mom and pop” but who listens, links or responds to us?

    Give me a “phrase” and I will show you a pack of wolves advancing quickly.

    Of course smart SEOs like personalization, it is a gift.

    Adopt a bunny

  22. dockarl

    “I hear a heck of a lot of concern from SEO’s about the whole personalization issue.

    I think fundamentally it comes down to this one thing – people are very concerned that the searches they have worked so hard to dominate under the ‘old’ system will fall by the wayside under personalization.”

    White hat SEO (which is usually based on solid knowledge of search, how search engines operate and how to meet the quality guidelines of search engines etc..) doesn’t need to have any concerns regarding personalization.

    Its mostly the back hat SEO (most correctly should be called spam :)) who should be very worry of any developments from the search engines side.

  23. Regarding the fear of personalization. I only hope Google doesn’t try a “one size fits all” approach to it. For instance, the demographics of my market might behave in an intirely different manner than the demographics of another market. Just for example, the “SEO Crowd” is usually into all the internet “bells and whistles” while the typical purchaser of a handgun holster might be happy with an outdated version of IE and a dial up connection.

  24. I have a bit of a hard time understanding how the personalized searches are working. Take for example this simple scenario:

    I check to see where my page is in the SERPS and from time to time will click through to my website from a search. It would seem to me that this would be a good indication that this page should resolve higher in the results when in fact the opposite happens. In fact, sites that show up below me in the natural SERPS will show up above me in the personalized search. This seems counter to what would, to me, seem like the best indication of what I am looking for in a personalized search result. Perhaps I am wring about what I am looking for or perhaps the algorithm could use a few tweaks!

  25. I’ll probably get hammered for this thought – but why is it so important for Google to spend those resources turning complex algorithms into simple algorithms? Does an increase in simplicity = and increase in efficiency or what?

  26. Maybe Matt failed to mention simple but MORE algorithms for specific tasks?

    So if you have one algorithm and it is really messing things up the others keep search relevancy going. I have a website that is currently getting really messed up, still feels the same on my end even if I am a victim of one algorithm.

    Interesting question by E Lawrence Welch.

    I am talking as much as igor, must shutup now…

  27. igor

    And I thought you left to China and was just about to declare Aaron Pratt as our new igorrrrrrrrrrrrrrrrrrrrrrrr 🙂

  28. E. Lawrence Welch – The reason for investing in simplifying algorithms is that writing code is a very labor intensive process. Equally difficult is learning someone else’s code.

    Since lots of engineers touch the Google code, they are better off if it’s simpler. It takes less time to learn, is easier to maintain, and developers are less likely to accidentally break something when they make changes. It also makes testing less expensive.

  29. igor

    I see a kind intelligent man behind igorrrrrrrrrrrrrrrrrrrrr. Provided that you leave Adam Lasnik and Matt Cutts in peace 🙂

  30. Could the human editor be the reason that my site keeps dropping after making it to the top page of google for certain keywords?

    I get a ton of links because we run a news site that is tops in the industry, so the site climbs, then drops… The cycle continues every couple of weeks.

    Just the past two weeks my staff and I have really become disheartened because after two years of building the best site we could, after doing everything perfectly and really making a name for ourselves in our industry, we had no answer to the question, what else can we do?

    What a waste of time! Two years of my life, and stopped at a certain point because an editor at google doesn’t like our site.

  31. I can’t wait until the day where people take full advantage of these search input features and beat down into search oblivion everyone who’s getting high rankings currently on sheer numbers of links, link schemes, age of domain, etc. Not to say there aren’t a ton of high-quality sites out there, some well-deserving of their top positions, but increased human input into the search results will continue to quality-control the already pretty accurate SERPs.

    The only thing that kinda scares me about it though; at some point, popular sway could begin to have too much influence over search results. And then we’d end up with sites that are popular getting the high positions, regardless of whether they contained the most relevant (or accurate) information.

  32. Rishi Lakhani

    How do you know whether that person wants to know about how blue widgets work or where s/he can buy blue widgets.. indeed, the searcher might even be looking for a site about their local footy team ‘the blue widgets’… it is so hard to measure intent.

    Valid comment, but shouldnt there also be an onus on educating searchers on refining results?

    One thing that words great are the annotated results: http://www.google.co.uk/search?hl=en&q=gallbladder+surgery&meta= (refining results and advice on top of SERPs)
    and related searches: http://www.google.co.uk/search?hl=en&q=british+holidays&meta= (below the SERPs).

    These sort of touches enble the use of long tailed aproach to reaching the right target and a great step in the right direction… (solely IMO)

    @Dr. David Klein

    On the other hand, we created a couple of cartoon books that are about 80 pages long each, and have very little content on each page, and cartoons. They are some of the best pages of content in our industry. Users love them, and they do have quite a few links.

    Please fogive me if this is a stupid question (i havent seen the pages in question), but do you make use of relevant alt tags on the comics? Maybe descriptions of the story they portray may help increase text content. These should help with SERPs, and failing that, would atleast add a factor of usability for those using screenreaders, and maybe show up results on related image searches.

  33. Rishi Lakhani

    I get a ton of links because we run a news site that is tops in the industry, so the site climbs, then drops… The cycle continues every couple of weeks.

    again maybe am asking a stupid question, but does your site drop because a large number of other sites are reporting/talking about the same issues? If these sites take longer to publish these articles / stories, they would probably force you from the top positions if they are stronger.

    A human editor is highly unlikely to target one site – there are various of other SEO factors, but without seeing the present state of your site, it would be hard to advise…

  34. Dr David, the Google toolbar already allows voting. Go to Settings >> Options >> More and enable it.

    And to others, don’t feed the trolls. The only way to kill them is to ignore their comments altogether.

  35. Rishi,

    Nope. It was not a stupid question! I am still finding plenty of pockets of things I don’t understand in this. The pages in question have NO alt tags!

    I will clean those pages up and add those and see what happens. Thank you so much!
    dk

  36. Personalization is a good thing as long as I have a way to delete my cookies to opt out.

    For instance, one of the thing I most hate, is to enter http://www.google.com and getting the results from http://www.google.at although the page still seems to be google.com.

    When I enter a term on http://www.google.fr, I still want to have the pages in France and the page from French and again it is only possible through deleting the cookies.

    Personalization may be good but in my personal case, it is making my researches harder so I am better off without it.

  37. Hi!

    My way to personalize would some optional buttons:

    While searching my Stats-files
    after the behaviour of a special crawler-bot (and his targets)
    I thought about the best way to find him and his brothers one way. So I came to a further idea for the Searchulator’s button-tide.

    This * button could be the joker in my search queries.

    Greetings Karl

  38. #
    E Lawrence Welch Said,

    July 10, 2007 @ 11:26 am

    I’ll probably get hammered for this thought – but why is it so important for Google to spend those resources turning complex algorithms into simple algorithms? Does an increase in simplicity = and increase in efficiency or what?

    Actually, this is a very good question, and there are lots of reasons simplicity is preferable. One person already pointed out that it’s easier to code.

    The other reason is that Google is basically trying to develop systems based on estimates of what is probable. So, they are essentially doing probability and statistics but based on massive amounts of noisy data.

    When ever you try to develop a probabalistic model of some sort, there is a strong likelihood that some features in your model are data artifacts that come from imputing meaning to noise.

    If you find a simple 1 parameter model that fits the data as well or better than a 2 parameter model, it is generally the case that the second parameter gave no additional data. Simpler models are often more robust.

  39. Why does Google not simply redirect people to their regional google when searching, like Yahoo and MSN? If I search for something in Google.com from outside the US I get a mainly US dominated set of results. If I search within the US it’s all US dominated. If I search the same thing from, say, google.ca I get a Canadian dominated set of results. But loads of users automatically revert to Google.com wherever they are. So a Canadian user goes to Google.com and sees *some* Canadian results and thinks that’s it. Google doesn’t tell him to go to google.ca, nor does it redirect them to google.ca.
    Can you possibly stop buying companies and inventing stuff and make what you already have – work properly? Just redirect people to their regional google – is that so hard?

  40. Hi,
    having researchers checking pages is in general a good idea as long as it is not “censorship”.
    But I see the danger, it becomes the same as it is with DMOZ: an aggregation of manipulation and self-overestimating persons (hopefully, G. has better internal guidelines and quality-education and audits) which does not exclude the bad boys and pages, but safe them from competitors…

    A human should sort out obviously wrong and negative results and (useless) pages which reached good positions using unwanted methods only and that should be audited, because without it, many pages would become victims of personal preferences. (see DMOZ, where this takes place!).

    I wonder sometimes, why endless advertisement pages, catalogues and contentless fakepages and worse pages with faked content but in rality bad content cover the best places, and not only for a short ime, over months and longer… nobody seems to check this…

    More: scriptresults like “buy North pole at Ebay” , when someone searches for North pole, are numerous… stupid results… will humans do something against this? Or is the financial Adwords-aspect finally the benchmark of correction…? If the bad page makes a lot of clicks, no matter, what´s real content? Sometimes, it seems not only to me so…

css.php