Two Search Interviews

Popular Mechanics asks 20 questions of Udi Manber, who is a VP of Engineering at Google on core search quality. My favorite:

There have been a lot of fads in search of late, such as Human Assisted Search and contextual search. Do those get folded into search as a whole? What are real trends in search and what are fluff?

So let me first tell you about Google. At Google we do not manually change results. For example, if we find for a particular query that result No. 4 should be result No. 1, we do not have the capability to manually change it. We made that decision not to put that capability in the algorithm—we have to go and actually change the algorithm. That is, we have to find what weakness in the algorithm caused that result and find a general solution to that, evaluate whether a general solution really works and if it’s better, and then launch a general solution. That makes the process slower, but it puts a lot more discipline on us and makes it more unbiased.

That’s the right answer for a general/Popular Mechanics audience. For the nitpicking search junkies that read here, I’ll just add that we are willing to take manual action on a small number of issues like webspam and removals for legal reasons. The rest of the interview was also interesting.

One other interview: A few months ago I talked with John Jantsch for the Duct Tape Marketing blog, but I don’t think I pointed the interview out to folks here. The MP3 is up on the site, if you want your search engine optimization (SEO) interview fix. I think this is a pretty good interview if you’re a small business owner or a little newer to SEO.

Speaking of MP3s, I’m caught up on my Daily SearchCast and I’m looking for maybe one other podcast to add to my listening rotation. What podcast would you recommend that I try? I don’t mind something outside of search — in fact, a non-search podcast might be nice to listen to something different.

40 Responses to Two Search Interviews (Leave a comment)

  1. By the way, please don’t take Udi’s “we do not manually change results” comments and blow them out of proportion. We already went through that kerfluffle in 2004, when David Krane noted “Our search results are not manipulated by hand. We’re not able to make any manual changes to the results.” (see here). Then folks on WebmasterWorld got hot and bothered because Google has also been clear that we are willing to take manual action on spam (no one wants to get off-topic porn when they search for their own name, for example).

    After a little while, GoogleGuy stopped by and said this:

    I walked over to see David Krane and asked him about it, because I had a hunch that David was talking about the results for this particular search (the word “jew”) and not our overall system. And that’s the correct explanation.

    To give some background: people write us all the time to say that they dislike or disagree with a particular set of search results. For example, at one point someone wrote in and claimed that one of the search results for Martin Luther King was a revisionist history and wasn’t accurate. Should Google go and remove that result by hand? Who gets to decide whether a result deserves to be in the top 10? You can see where the slope gets slippery really quickly when you start bringing value judgments about the content of the site into the mix.

    So historically Google has very strongly tried to follow a policy of letting our algorithmic search results stand as they are; we put our efforts much more into improving search by writing better algorithms instead of trying to fix a smaller set of searches by hand. We have a quite small set of circumstances that can result in taking manual action: things like a valid legal request (e.g. a DMCA complaint), spam and things outside our quality guidelines (e.g. off-topic porn for a person’s name), and a very small amount of security-related stuff (e.g. credit card numbers on a web page). Other than that, we do our best to let our algorithms work out the results on their own. I think that’s the right approach, and I think most of our users would prefer that instead of lots of hand-editing.

    Does that mean every search is perfect? Of course not. With 200+ million searches a day, there will be some searches that aren’t as good as they can be. But when a bad search is pointed out to us, we look to how to improve our algorithms instead of doing some one-off change. That’s the principle that’s coming into play here.

    If you go back and read the article, you can see that idea underlying it. I did a double-take at the second paragraph of the article: “Weinstock has launched an online petition, asking Google to remove the site from its index. He said if Google receives 50,000 requests to remove the site, it will comply.” I have to wonder why Weinstock would say that (if he did). That did not ring true to me at all–I can’t imagine anyone in a position of responsibility at Google ever saying anything like that. I don’t like the first result for this search either, but we’re not going to tweak the results for “jew” by hand. Now go back and re-read the fifth and sixth paragraphs of the news.com article. I think knowing Google’s philosophy and a little more background puts the quote into context. It’s hard to communicate all of these ideas that I’ve mentioned here with complete specificity and absolutely no ambiguity in three sentences, but I think David did a great job; he was saying that we won’t tweak the results for this search by hand. I asked David to make sure that’s what he meant, and it was.

    That GoogleGuy comment, even though it’s 4+ years old, has aged pretty well in my opinion. And if you’re really, really interested in this subject — which I am; put a Sprite in my hand and I’ll happily discuss the tradeoffs of algorithms and humans for hours — then you can read more about this in my post on the role of humans in Google search. Given that Udi was giving a brief answer to a general audience, I thought his answer was spot on.

  2. Wanna bet how many people scan the first sentances seeing “At Google we do not manually change results” and scan the rest of the post without reading it?

    My gut was like whoa, that’s not what I was told. I do like how you emphasise it’s the right answer for a general audience. I just listened to audio of me talking to some non-bloggers and thought “OMG, search folk would think I’m nuts.” Some already do. 🙂

    Going to check out the duct tape podcast.

  3. Podcast Recommendation:

    Marketplace – American Public Media

    Keeps you updated with US business day

  4. Nicely preempted Matt!

    You should definitely check out TWIT if you haven’t already. Leo’s on holiday right now, but should give you time to get up to speed with the previous episodes. 🙂

  5. Merrick, I think I’ve listened to that on the radio and it was pretty good..

  6. “…at the bottom of many pages, you’ll see query refinements. These are suggestions from us about what your next query should be. And we put it at the bottom because that’s where you run into problems…”
    actually not entirely true, half of the related searches (3) are placed at the top of the page

  7. A few important points here. Thanks for the clarification.

    I have read or heard a few times where someone claims that someone at Google will mark a site as “the authority” which will cause that mark will to supersede the algorithm. I am glad that no one can ever finally win the game in their sector.

    Also, please note, do not offer Matt a Mountain Dew. I tried this at Pubcon since the vending machine was out of Sprite. He politely declined. In the future I am carrying an ice chest of Sprite to all future Search events, just in case.

    dk

  8. TWIS science radio show. A humorous and irreverent take on the world of science.
    http://www.twis.org

  9. Hi Matt,

    A couple on non-search podcast suggestions you might like (if you aren’t listening to them already) are:

    Kevin Smith’s SModcast –
    http://www.quickstopentertainment.com/category/smodcast/
    Really funny ramblings from everyone’s favorite slacker/filmaker.

    The Ricky Gervais Podcasts –
    http://www.rickygervais.com/podcast.php
    More stream of conciousness comedy from that bloke from the english version of The Office. I laughed so hard a little bit of wee came out 😉

  10. HI Matt:
    Podcasts:
    1.HBR Idea Cast
    2. TWIT
    3. Boag World
    4. Manager- Tools

  11. Hi!

    Ignored by every one
    I feel me a little bit crazy today.

    If YOU/Gooogle don’t react on suggestions like mines (comment-125958 here)
    the newest German one-liner will become 100% percent true:

    Google isn’t a search engine but a misapprehension-finder

    in German it’s sounds still better:

    Goggle ist keine Suchmaschine sondern ein IrrtumsFinder.

    Greetings Karl

  12. I am trying to learn spanish by podcast … not quite going as well as can be hoped…. but my wife is a native spanish speaker and her family doesn’t speak english, my spanish is practically non-existant, but oddly enough we get on really well… so maybe learning isn’t the best idea….

    I am listening to – http://www.myspanishconnection.com/ on my mobile phone …

    I tried stock market tips – but didn’t understand a word of it and i have figured out that playing the stock market should only be done by people who have some idea of what they are doing …

    I totally do reccomend the Daily Search Cast, and its amazing how many people have started listening to it in my office (AARF UK) recently.

  13. Matt, looking for more podcasts? Subscribe to my video podcast at http://www.seroundtable.com/subscribe.html#video (via RSS, iTunes, or YouTube). 😉

  14. The only problem with changing the algorithm because of a few bad sites, is that good sites suffer also.

    So in essence a few bad apples can ruin it for many innocent ones.

    This may explain why good Webmasters were mystified by sudden drops in rankings when they did not do anything bad

  15. I don’t know if this will turn into a podcast series but Jeff Atwood and Joel Spolsky are starting something at stackoverflow.com. I am sure the first podcast is worth a listen…

  16. Best digital photography podcast to be found. Based in Germany, but the podcast is in English. Has both audio and video episodes. I’ve been following it for years. If you’re not already listening, you’ll get hooked quickly; and all previous episodes are archived so if you want to find out a particular topic, you can easily go back. Suitable for the beginner to the professional.

    http://www.tipsfromthetopfloor.com

  17. Hi Matt,
    When you say “we are willing to take manual action on a small number of issues like webspam and removals for legal reasons.”, you’re talking about penalties that Google can impose upon a site violating GQG. These penalties include manual lowering of the site’s rank in serps, right?

  18. I recommend ‘Search Engine’ podcast. It is not about SEO.

    http://www.cbc.ca/podcasting/index.html?newsandcurrent#searchengine

  19. Podcasts –

    This American Life – Ira Glass – NPR

    Informative, funny, touching, like the one on the nature of love and my all time favorite – and a must listen, Fiasco!

    Unconditional Love (very moving with some surprises)
    http://thislife.org/Radio_Episode.aspx?sched=1204

    Fiasco! (very very funny)
    http://thislife.org/Radio_Episode.aspx?episode=61

    There is a tiny orange icon that says “Full Episode” –
    you need to click on that to hear the full, free,
    show.

    And podcast subscription link:
    http://feeds.thisamericanlife.org/talupdates

    You will not hear the two above in the current feed – those are classics that are not to be missed.

    No, I’m not affiliated with the show in any way. 😉

    Enjoy!

  20. Off to make a “I bribed Matt with a Sprite” t-shirt. Maybe Threadwatch needs revived. Too many good tshirt ideas to go threadless 😉

  21. in terms of podcasts I recommend Danny Sullivan do more Daily Search Casts!!! Can’t we make this show daily!?!? perhaps everyone in the search space should agree to sponsor the show for like $500 each a month (20 folks) so there is a budget to keep it going day in and day out.

    the DSC is amazing and it kills me that it is not daily. i think a second host to fill in for danny would be killer.

    best j

  22. Hey I found your blog on Jason Calicanis recommendation. This is a great blog I love the content and now you are in my google reader rotation.

    I also requested to follow you on twitter if you are interested in accepting that invite

    Here are my podcast suggestions:
    Marketing Edge – non search focused
    Web 2.0 show – interesting interviews
    Science Friday – NPR this is a good one for geeks, and nerds

    Audible audio books – for me these are supplementing a lot of my podcast listening time – right now I listening to “On Intelligence” Jeff Hawkins book (the palm guy). He is a big nerd (when did that become a compliment?) doing some very cool things and it is so interesting.

  23. I’ve been enjoying escape pod – it’s almost like an audio version of a sci-fi mag. The short stories make for a nice addition to a business or tech podcast.

    http://escapepod.org/

    It also sometimes has a broader definition of what qualifies as sci-fi, which makes it all the more interesting.

  24. geekbrief.tv
    webbalert
    loaded – cnet
    they are all short < 5 minutes and video

  25. Podcast suggestion: not directly search-related, but deals with search a lot: Marketing over Coffee, with Christopher Penn and John Wall. Two pretty brilliant guys sit in a Dunkin Donuts and talk about online marketing.

  26. purposeinc, no, we can’t mark a site as “the authority” in the same way that we can’t force a site to be #1 for a query. You might ask some of the other search engines if they do that though. 😉

    Claudiu, I’m sure this interview was a few weeks/months ago. I believe you’re right that we’re sometimes willing to show query refinements at the top recently.

    Karl, I didn’t respond, but I did read your comment. On average I get 65+ comments for every post I write, so I can’t write a response to all of them.

    Gerry, I do like the idea of a free set of MP3s to learn a new language. That’s a pretty cool idea for a niche. Anyone want to suggest existing resources for that?

    Andy, yes: we can manually remove or otherwise penalize sites on incidents that we consider to be webspam.

    Gideon, suresh, netmeg, and graywolf: thanks for the suggestions! Barry Schwartz, I need something that I can listen to in my car, so video cramps that a little bit. 🙂

  27. Morris Rosenthal

    Matt,

    A friend of mine who works for a large Jewish newspaper asked me years ago what could be done about the Google results for “Jew”. I told him to contact his colleagues at other Jewish publications and to put out the word that linking to the site in question, in order to have a news story about how it’s bad, was a large part of the problem.

    Unfortunately, it turned out that most journalists lack that sort of discipline or think the the moment in the limelight is worth the longterm effects caused by linking. And the idea of adding “NOFOLLOW”
    tags just draws a blank stare.

    Morris

  28. Radiolab. http://www.wnyc.org/shows/radiolab/ best podcast and radio show (other than This American Life or WaitWait Don’t Tell Me) on NPR. They talk about music, the brain, laughter, sleep, zoos, space, from a non-science, science approach.

    My favorite (and a great place to start) is Season 2 Musical Language
    “What is music? How does it work? ”

    Excellent listening.

    -Marshall

  29. You really love podcats, dont you? About tech things you can check out http://www.techpodcasts.com/

  30. Matt,

    “Given that Udi was giving a brief answer to a general audience, I thought his answer was spot on.”

    Udi is an algo freak. He is 100% academic when he talk. Problem has always been: once an academic simplify explaining academic matters to general audience, he/she gets in trouble with the prof in that field.

    I guess thats exactly what happening.

    Solution..keep academic matters at academic level, regardless of the audience 🙂

  31. This is one of my favorite:

    All in the Mind, presented by Natasha Mitchell, is Radio National’s weekly foray into all things mental – a new program about the mind, brain and behaviour.

    http://www.abc.net.au/rn/allinthemind/default.htm

  32. I go with NPR podcast. They are often easy listening and something that deals nothing with my job. It is kind of like putting an RSS headline ticker in my ears. If I am really wanting an entertaining listen I go with a comedy podcast.

  33. Dave (original)

    Matt, why try and educate those who refuse to listen to facts? Also, why is common sense no longer common?

  34. Matt,

    What were Emmy´s and OZ presents to you today 🙂

  35. Hi Matt,

    Udi said “at the bottom of many pages, you’ll see query refinements. These are suggestions from us about what your next query should be”

    These are supposed to be aids to help you to find “the right answer” or at least the best web sites on that topic but in the UK, for certain terms they have exactly the opposite effect. They take people down blind alleys and waste their time. I wonder if this is to do with the US orientation of Google and the fact that UK English semantics are different to US English semantics. I would be happy to send you a case study if you are interested.

    On the main SERPS in our market I’ve noticed a number of sites rise to the top by breaking a number of your guidelines. You say “we are going to seek out paid links and penalise sites that contrive their way to the top” or words to that effect but in our market the top 3 have recently risen to the top by doing exactly the thing you are saying you are campaigning against. I wonder if this is because it is difficult to have different thresholds in your algorithms for the US and UK market. Perhaps a side effect of your actions on the US market is having an opposite effect on the UK. Again I would be very happy to send you the example.

    On a general point, some things regarding Google are simply brilliant but it seems to me that Google is so huge and the amount of noise that you get thrown at you every day must make it virtually impossible to identify which feedback is worth considering. If I ever write to Google if I get a reply at all it is very bland and generic which makes me think that the person replying has not read my message and couldn’t care less if they have. It’s so frustrating. It kind of makes the Christmas card and gift you send me a worthless PR exercise because it just makes me think, if you really cared you would listen to me through the year, not just send me a card at Christmas.

    Cheers

    Phil

  36. Hi!

    AHA, he could not respond to all …….
    ?dementia is the elide word every one reads on comments like yours

    I hope that’s the same strategy, as while my ?dementia suggestions towards WebmasterConsole
    first: ignore, than make and afterwards delete

    than I have a wish: delete it as fast you can

    every who can search on Internet
    could know that I have a disease named fibromyalgia.

    I said and say it once more:

    I new (by 50%) the reasons of this illness.
    I am a man, and its mostly a women’s problem,
    I can only experiment with my body.

    Which doctor can come this illness nearer than me, having it for decades?

    Doctors say only: we don’t know anything about this illness than the words we find 4
    but your thoughts must be wrong or better them ignored me 99%
    because the truth maid be too destructive to there fame.

    Before analysing saying: this bacteria is normal on humans skin if
    There are no impetigo or acne and that some effects of this bacteria
    causes pains like cramps or dysentery (also often effects by fibromyalgia)
    are no indication only coincidences

    I could show them more indications which point in the same direction
    but no chance to be heard on places which could analysing them.

    I have had the chance to save mankind for its worst moments this decades
    and I failed (not alone). I think, the thanks 4 it is, that my name is on another of this special lists,
    only because the truth maid be too destructive to there fame.

    All this destiny collides with my ambitions
    This winter the wrists and fingers become problems
    while lifting a couple of pounds
    And soon I will be unable to work with fine motor skills

    With nearly all what I wrote here (despite this comment)
    I wanted to help us all
    I need no THANKS 4 it.

    Only equitable action.

    Greetings Karl

  37. If you want a non-SEO podcast, try Scientific American’s 60-second Science and 60-second Psych. Both are fun and informative.

  38. That means you manually change results for a spam website which has manipulated your great algorithm to show up in search results?

  39. Harith, the same presents as every day: warm, furry hugs. 🙂

    “These are supposed to be aids to help you to find “the right answer” or at least the best web sites on that topic but in the UK, for certain terms they have exactly the opposite effect. They take people down blind alleys and waste their time.”

    Phil Holden, that was one of the big reasons why we didn’t do query refinements for a long time — it can actually slow users down as they get off on a tangent. Regarding the sites you mentioned, I would use the mechanism that I’ve mentioned before to report those. We do actively still prowl through that feedback. Tell ’em Matt sent you in the report. 🙂

  40. Is Bing A Better Search Engine?

    We have created a very simple logical test that shows which search engine provides better search results. Google or Bing? I will explain the test on this page.

    First, I would like to make the test concept more clear with several examples:

    Say we take a series of Titles to search on Google and Bing for comparison.

    Here are several example: (all the tests are at rssfeedrss.com/index2.html)

    Title 1) Patients are willing to undergo multiple tests for new cancer treatments

    rssfeedrss.com/test2.html

    Title 2) Conference on composite materials for structural performance: Towards higher limits

    rssfeedrss.com/test4.html

    Now, I explain the way this test works.
    Each title is about two or three main keywords.
    For example Title 1 is about cancer treatment.
    Title 2 is about composite material.

    I propose a logical test that uses Google, and also Bing search results that extracts the main keywords in a logical manner. The better search engine will provide a better and more relevant extraction based on this logical test. I like to emphasize logic.

    Now what is this logical test?
    The better search engine provides search results that contain higher number of main keywords in the search page results (usually in bold).

    For example, if we take title 1 to either Google or Bing and make a search on the whole title and then count the number of times the main keywords appear in the search results (usually in Bold), the better search engine will give us cancer treatment and not other words. That means if you count the number of times the keywords cancer treatment appear in search results in both Google and Bing, Bing provides a higher quantity.

    I used both Google and Bing for the test on the page rssfeedrss.com/index2.html and Bing provided a better search. You can do this test in-house.

    I will propose this test in search engine conferences. It is a valid test.
    I can email you the perl file that performed the test. Call 949-500-8638 or email info@katir.com.

    In fact, if you continue the test to second page results, it also shows which search engine provides better search results for the second page or third page or….

    Why is this test valid?

    It is not very complex to prove why this test is valid. If you type a sentence that contains several main keywords, you prefer more information about those main keywords. The higher quantity of those main keywords prove the page is more relevant and the search engine has delivered more relevant results.

css.php