Review: Custom Search Engine

Google just announced something that I’m really jazzed about: Google Custom Search Engine. Several people mentioned that Google’s Accessible Search was built by using Google Co-op under the hood. Co-op has opened much of that power up to the public, so that anyone can build a custom search engine.

Most custom search engines (whether it be Google’s free sitesearch or Yahoo! Search Builder) only let you select one site to search, or you can offer websearch. Even Rollyo only lets you search over 25 sites.

This new offering lets you easily add hundreds (thousands?) of urls. You can search over ONLY the sites you choose, or (my favorite) you can apply a boost to the sites you choose, with regular websearch as a backfill. That’s really nice, because if your chosen urls talk about a subject, you’ll often get matches from those urls, but if the user types something completely unrelated, you’ll still get web results back. So it’s a true custom search engine, not just an engine restricted to showing matches from some domains.

You can also choose to exclude results from different sites. As far as I can tell, this happens in pretty close to real-time, even for complex url patterns. For example, I added the pattern “google.com/*” and started to get results from the Google directory, so I excluded “google.com/Top/*” and the Google directory results went away immediately.

The look and feel is also customizable. Search results can open up on google, or you can do an iframe to open results on your own site. The former option lets you pick some colors and a logo to customize; the latter option lets you fully integrate the search results into the look/feel of your site because you just wrap your preferred chrome around the iframe. If you’re a power webmaster, you’ll want to play with the iframe option.

Lots of other nice features are tucked away under the hood. For example, there’s a bookmarklet (Google Marker) so that if you’re surfing the web and find a site you’d like to add to your search engine, you just click and that site is instantly added to your search engine. And it wouldn’t be based on Google Co-op if you couldn’t choose to allow volunteers to edit your search engine and add new sites if you want. :)

When I played with the first version, I wanted to avoid the standard stuff where you plug in 1-2 sites and get a custom search engine that isn’t blood-pounding-ly exciting (“Oh, a search box, and it searches. Great.”). So what I did was take my feeds (I was using Bloglines at the time) and exported it as an OPML file. Running a command like
cat export.opml | grep "title=" | cut -d'"' -f6 | grep -v '^$' | sort | uniq
was enough to get the blog urls that I was reading (not the feed urls), and I threw those urls into the custom search engine.

And just like that, *BOOM* I had a search engine that covered 70+ blogs in the search/SEO industry. If I searched for [bug], it would return search engine bugs, not bugs in general. OPML-import was so much fun that the Co-op folks promised to support it (I know that importing from Bloglines works; importing from Google Reader might still need a tweak to the OPML parsing). It’s nice that every blogger can have a custom search engine that is centered around their interests.

There’s one tidbit that didn’t interest me much, but readers will probably be interested to hear. If you build a custom search engine, the Co-op folks provide a way to share revenue if people click on your search engine’s ads. Offering revshare is a great way to take a useful tool and get even more people interested in it.

Which leads to my personal take: there’s companies that tackle search for specific verticals (Trulia for real estate, Truveo for video, Kosmix for health and other topics, Powerset for natural language, Guruji for India). Those companies work hard to bring something special to their vertical search. But there’s a tier below “I want to get VC money to do vertical search,” and I think this product could enable that. In the same way that AdSense enabled a lot of very good content creation in different niches, Custom Search Engine could help a lot of people who want to make a search engine, but would be happy doing it not-as-a-VC-funded-startup. Anyway, this whole personal take is wild speculation, but it would be neat if it turned out to be true.

I do think that this launch will kick off a lot of opportunity that not everyone will see or understand at first. For example, the first person to make a truly kick-butt search engine about biking will likely start to attract volunteers and traction and first-mover attention, and could very well become the authority search for that niche. I think that this launch could kick off a wave of search over a long tail of niches; rather than a big vertical like “health,” someone could make a search for the much much smaller “health at every size” movement. Or juggling. Or insurance companies. Imagine your favorite niche, which could be as specific as a small-town or as broad as you want.

Give the Google Custom Search Engine a try. It’s easy enough that you can make a search engine quickly, but there’s a lot of power under the hood.

121 Responses to Review: Custom Search Engine (Leave a comment)

  1. Hi Matt,

    I first saw this over at the Google Blogoscoped forums. I’ve been developing a website recently, so my first thought was to make one for web developers. I’ve also thrown in some SEO stuff so now it is sort of a web developer/webmaster custom search engine. If anyone wants to volunteer to help me, I’d be happy to accept them :). Here’s the URL:

    http://google.com/coop/cse?cx=012209517088559882238%3Agthoahad-m8

    And no, I won’t put ads on it.

    Andrew

  2. tclimb

    Thanks for the post, very informative. Have you considered putting your custom search engine on your blog site so your readers can use it as a resource?

  3. piikol

    Go Google, another great piece of software. Google Custom search engine opens so many doors and so many oppertunaties. IS the Googles way of overtaking the vertical search market?

  4. Matt are you just the new product reviewer for Google now or are we gonna get some proper posts?

  5. It will be interesting to see the results (if they are published) of this new endeavour. I imagine it could help Google improve their search results as they are going to be supply (hopefully) more relevant results to the user who is searching.

  6. Unless you’re a 501(c)(3) non-profit, university, or government agency you gotta have ads, right?

  7. Matt -

    Very nice however the order of the results changes depending on the order in which I enter the urls to be searched. Why is this? Surely the relevancy of a given search should be the same for a given batch of urls no mater what order they were entered. Or am I missing something?

    - Michael

  8. David

    @Michael It doesn’t respect the order of urls with my setup. Everything is ranked like it is on the regular seach.

  9. Great tool much better than before, Matt can you pass on some feedback to the COOP team – there are no geographic or advanced search options :-( surely they can AJAX these into the code.

  10. ps. From experience with a previous but of R&D here, to make COOP even simpler can they make it so that when someone adds .mysite.com – notice the dot at the start, it means all sub domains of that domain. Having the dot ensures that it doesn’t match similar domains I.E. http://www.allaboutmysite.com

  11. Although now I’ve checked that doman (included as example before looking) I probably would want to add it :-)

  12. Upon checking this out, three features stood out:
    1) You can exclude sites that you do not want in the results
    2) You can easily do so using the Google Marker
    3) Anyone can volunteer to help

    So we decided to throw up an experiment to encourage everyone to mark spam sites to be excluded from search results.

    Working together as a community we may be able to radically improve the quality of the search results (or perhaps just get in a blacklisting war?)

    The result is Putch – http://www.putch.com

  13. rob

    Nice.

    In the “Search the entire web but emphasize included sites. ”

    Does this mean place the emphasis on included sites, as in show them first, or does it mean just highlight them (eg bolded).

    I found that by selecting this option, my included sites were nowhere to be found even for the most relevant of search terms.

    Im assuming that this is a consequence of some other general rank -30 penalty or similar ‘sandbox’ phenomenum.

    Shame really, especially if I want to give my users an option to search my site via google using google results as a backfill for no results returned.

    Maybe they can append the param &draconian=0 at the backend :D

  14. This is the missing link I have been looking for. Previously I have played with the opensource Nutch search engine to create my own verticle specific search engine – this is much better.

    I don’t imagine that this will become official, but it is begging to be integrated with del.icio.us or similar, so that as you tag the entries are indexed by your search engine (previously this was only possible with offline tools such a Cocoa.licio.us), and the tags themselves become ‘refinements’.

    In the meantime a few regex’s later on my OPML export from NetNewsWire and an export of del.icio.us links and I have my own web development domain language search that can be added to by my colleagues.

    All I have to do is remember to ‘Google Marker’ a page when I tag it. Very much appreciated to the guys over at the Co-op.

  15. I wish there were something between “anyone” and “invited individuals” for collaboration. The former opens the door to web spam, which will soon come in. The latter is too simple a model for building a large community.

  16. Works perfectly well with Google Reader too. I just needed to change the position from 6 to 10: cat export.xml | grep “title=” | cut -d’”‘ -f10 | grep -v ‘^$’ | sort | uniq

  17. I am surprised you didn’t mention the feature “Search engine keywords” – this adds a background label, which enables one to manipulate the search on whatever keywords you add in this section. I am really glad they included this similar option (a background label) on removing completely URL’s not wanted. For those who are interested in more “under the hood” detail on how this works see this link: http://www.indigoguide.com/implementcoop.htm which I did include on your blog some time ago. It is incredibly powerful stuff!!

  18. Hi Matt,

    nice idea about the OPML export :) That and the Google Marker will make it a usable product for sure.

  19. Jarid

    And just like that, *BOOM* I had a search engine that covered 70+ blogs in the search/SEO industry

    Hey Matt, how about releasing this custom search? It sounds pretty useful..

  20. Geo

    What i most liked, was the ability to put ads on the cse. Because, yes, i do want to make some money but not on my blog interface.

  21. Matt, I am not planning to do so but can this Google Co-op be used for SEO experiment to verify which of two different sites is preferred by Google’s top ranking on certain queries?

  22. loki

    hm.
    you forgot to say that LiveSearch search macros does it too, with maybe the difference that it doesn’t ask to have adsense shayte linked to it.
    Gigablast offers free custom forms to create sort of rollyo engines too, for quite a long time.

    Anyway… would be nice if it was accessible through a webservice, and not monetize-my-content-ffs oriented.

    look, read the FAQ : http://www.google.com/coop/docs/cse/faq.html
    out of 19 question, question 8 (!!) is : “How do I make money from my Custom Search Engine?”

    Then, question 3 :

    With a Custom Search Engine you can:

    * Apply your website’s look and feel to the search results page.
    * Make money from your Custom Search Engine by participating in Google’s AdSense program.

    etc..

    Come on.. what kind of priorities is that ? Google places the money part before the features ?

    …enough said. another great Google product. good luck.

  23. Having not tried it yet, there’s one thing that concerns me somewhat about it (and I can see this becoming one of the next SEO myths):

    Let’s say I build a custom search engine based on the sites I’m primarily interested in. And let’s say I put in a.com and b.com as my two favourite sites, the ones I want to see first if they contain results that are relevant to my query.

    Are the domains a.com and b.com going to receive any kind of boost in terms of rankings because I added them in? Personally, I would hope not simply because it would be far too easy to manipulate this type of thing.

  24. This tool is pretty cool. Very easy to set up. It’s funny though: I set up my custom search engine, put in the sites I wanted to highlight, and they still showed up at the bottom of the results!

  25. And here we have another Google service that fails in Opera out-of-the box. Sigh.

  26. This is great stuff. I just implemented it over at my site about the famous Macbook Shutdowns – http://www.macbookrandomshutdown.com/search/

    I also ‘hooked up’ my adsense pub-id to my coop implementation, but I don’t see any ads on the search? Any ideas?

    Thanks in advance!
    Matthew

    Are you AS hijacked? – http://www.adsensevalidator.com

  27. I have just created a custom search engine on all things sharepoint.
    Go ahead and do a search on it and help me improve my search engine listings.

    Visit http://www.sharepointbuzz.com

  28. matt,
    nice overview. it will be interesting to see how google unfolds his vertical search strategy. personally, i think this is just the opening move.

    mark

  29. I actually saw my first implementation of this today http://www.blogsearchengine.com/ from search engine journal.

    I will definitively be giving it a run through. My fear is that this will turn into another dmoz with the same content on thousands of sites. I guess in the end it’s all about who can leverage the information in new and unique ways to provide useful information to the user. Same as it ever was :)

  30. My own first implementation, hope you like it, I kinda reversed the model to make it entirely exclusion based: http://www.myserp.com

    Basically a small number of sites take up a vast amount of real estate in the SERP, the old 80-20 rule, so by filtering most of them out, suddenly you can find really cool stuff much quicker, and if I wanted to use them I just turn off the filter, it’s that simple. Try it I think you’ll like the experience.

  31. Another great idea. I created an NFL search engine for those of us into football and fantasy football. Feel free to contribute:

    http://www.google.com/coop/cse?cx=014720192563721879933%3Axkugvfhuseg

  32. SEOidiot, Google Reader and Custom Search Engine were two products I’ve really enjoyed, and I said to myself “Hmmm. I guess I can talk about whatever I want, up to an including things that might bore SEOidiot.” ;) So I decided to talk about them. If the idea of making your own search engine isn’t interesting, just skip this post.

    Jeremy Luebke, thanks for the pointer!

    Elmer W. Cagape, I don’t think you’d learn anything from that. CSE is applying a boost to the sites you ask for. As David points out, it’s just “boost or non-boost”; the order of the sites doesn’t matter.

    Jarid, I built it on an interal prototype of CSE, but as soon as I get a chance I’ll recreate it and post a link.

    esoos, I believe that’s correct (which I think is fair since you’re getting a free search engine). But if you have an AdSense account (or want to sign up for one), you can get revshare.

  33. /pd

    MAtt, what is really needed is RSS feeds for search term. I can export my OMPL, create a trigger (search term) and then suck this back into my reader via rss , whenever that term is found on my CES..

    Any plans on how to do this ?

  34. I am really enjoying this and the potential for great things is enormous. To start things off I am attempting to form a collaboration for searching only WIKI sites in an attempt to find information quickly. Of course I don’t know all the good Wiki’s so I have put the search engine open to volunteers who wish to add sites.

    The URL is here: http://google.com/coop/cse?cx=006775555251158006122%3Anxp0gaipa40
    and it currently has 27 sites included in the search and is quickly adding more.

  35. I was so excited… until I saw the ads.

    I cannot put it on my site since that will have all of my competitors ads.

  36. /pd, good suggestion. I don’t think that’s on their list right now, but I’ll pass the feedback on.

  37. How about specifying whether to use news search, blog search, normal search, etc. or a combination of search services?

  38. I would love to see and option that lets me define the width of the search box.

  39. How does one import a list of url’s into the product?

    The search box width can be defined in the HTML that they give you (not as nice as a fill in the blank but adequate)

  40. Never mind my question in the last post…I was using Safari which doesn’t show the add site pop ups correctly

  41. Nice nice nice. But… Now we need a directory for all the new Custom Search Engines. The Featured Google examples page at http://google.com/coop/cse/examples/
    is a good start, but my hunch is within a month there will be hundreds, and soon enough, thousands. I created one and would have loved it if one of the options at the end was to “Include this engine in Google’s Directory of Engines” but alas, nada…

    Sounds like an opportunity to me.

    Eric

  42. Joe

    Hi Matt,

    This is the best thing since page-rank. I’ve got a ton of ideas about it. For instance, you could create an add-to-CSE addin for firefox/IE.

    In any case, I hope you put aside another 1.6 billion for the girl/guy who came up with this (this is much better then YouTube).

  43. Matt Cutts Sad

    Sounds useful, not sure if it’s already been done, but wouldn’t it be good if your list could be published, and other peoples too and then users could mine those lists to extend your own.

    e.g. Suppose I then had a list ‘political astroturf sites’ and it listed say 100 URLs.

    Suppose person B also has 100 sites on his ‘I hate fake news sites’.

    Suppose 50 of those sites are common to both lists.

    Now if I say ‘search my list and similar lists within 55%’, since persons B’s list is within 55% of my list, it also searches the 50 sites on his list that are not on mine to widen my searches.

    The aim would be to apply collective knowledge to narrow search, sort of Friendster meets Google. So that the result set isn’t static and doesn’t require manually updating, it grows based on how other people mine similar datasets.

  44. Fantastic new feature. I initially looked at this for site search purposes, but then I tried building a vertical search engine for UK car reviews.

    Using URL Patterns, it’s so easy to filter out just the reviews from reputable sites, without spurious navigation pages, non-UK sites, or webspam getting into the results.

    See http://www.carfilter.co.uk

    I second the request for an RSS feed, or a way to see the more recent results by date. Extending the functionality to other Google searches such as Google News would also result in some great little niche seach engines.

  45. Superb. The ability to add many URLs was a big problem with LIVE and Yahoo’s equivalent. Can’t wait to use this for regional travel search, where trusted and informative URLs are often hard to find yet very important to quality.

  46. Eric, (and Matt),

    This is just too funny. I just emailed you directly about our directory of custom search engines. You can see it at: http://www.customsearchguide.com/.

    Includes more than 30 CSEs, user voting, and will soon include the ability for users to make their own suggestions too.

    Visitors welcome!

    Cheers,

    Eric

  47. Joe

    matt all I can say is after spending the day on this looking playing tinkering thinking I am fired up as I have found a kazillion things I can do with it to make me some mo money add content help my visitors bring them back and make it fun to search.

    This is gonna be fun.

  48. LOL ok Matt it just shows i care thats all ;)

    I actually like Reader apart from the oops an error occurred message. Id try my own search engine but im sure youd just point out thats what i had at SEOidiot.com before the blog… no hiding eh?

    IMO you should be very aware of who reads the blog though – most of us have been around a long time and we get our news about new products from places independant of the company releasing them, thats all. We are a very sceptical and paranoid audience…. well the sneaky people are – im not saying I do anything that could make me paranoid though ;) All my stuff is high quality original content…

  49. Thats great Matt, Im loving the CSE. Im making my first CSE all about cats and kittens since that is the 1st main area of my blog, i will likely make another for human rights issues etc, because that is the other part of my blog.. Thanks again, this is deffinatley one google product im liking. My CSE isnt on my blog yet but it will be soon.

  50. Tim

    Just a quick question. Would this allow me to create a custom search engine that searches urls from different domain types. Eg. all .edu sites or all .info sites?

  51. Joe, glad you like it. It definitely has a ton of stuff you can tune and tweak to your heart’s desire, and I think Google will keep adding more.

    Eric, thanks for replying on the thread; I emailed you this morning, but hadn’t seen your reply yet.

    SEOidiot, hope I didn’t jump off the handle. I was actually thinking “Will I get ‘where are the SEO posts?’ comments about this?” And I feel like sometimes I need to branch out a little bit, whether it be gadgets or whatever.

    Very cool, Auliya. It’s neat to take stuff that you know about and bundle it up into a search engine. :)

  52. Really like the new tools for webmasters that you guys have put up for us for more precise info. The storelink to Ebay is great too as is the batch upload for Froogle(Google Base now I guess). I have been using these for awhile, along with the analytics.

    New issue for us is how to move our listings that we have had for several years with our Yahoo Store to a new ecommerce site and not lose the rankings we have built up over the years. I understand the 301 concept but am stumped by how to properly implement this through the Yahoo store page by page and not just a domain 301. They seem to only have a 301 available at the overall domain level and we can’t seem to get to the server to do it there apparently. We can get to head tags and meta tags but that seems to be all.

    Any thoughts???

    Thanks

  53. Hi Matt,

    Sheesh! Just emailed you again. I hate it when email does not go through for some reason. Let me know if you don’t get it.

    People may also be interested in the article about How to Build a Quality CSE at:

    http://www.customsearchguide.com/creating-quality-cses.shtml

    Eric

  54. Gadgets is good Matt, maybe you can do a few more site reviews with the consent of the site owners… things we can all get far too worked up reading things into that you really had no intention in saying…. ;) you know how we are…

  55. Harith

    Matt!

    “And I feel like sometimes I need to branch out a little bit, whether it be gadgets or whatever.”

    In fact you haven’t branched out gadgets for ages. I mean the real kentuckian gadgets, you know :)

  56. I made a custom information architecture search engine here: http://iasearch.net/

    It’s more customized than the default look and a good example of what you can do in 2 hours with this tool. Pretty damn powerful. Google’s definitely shooting at the vertical search engines with this one.

  57. Matthew:
    I do see ads, I searched on “macbooks” in your search box.
    The only problem I have is on the resulting page I have random characters “t6d3f2cvofm” proceeding my text box and as one of the radio option below the box. The search box is at the very bottom of http://www.othersingles.com/quicksearch.cfm

  58. > This new offering lets you easily add hundreds (thousands?) of urls

    Yes, thousands!

    A total of 2000 and 5000 “annotations” are allowed per annotation file and per coop profile, respectively – according to the Advanced tab for a custom SE’s control panel. (There’s a 1-1 correspondence between an annotation and a site you specify for a custom SE.)

    I can’t seem to determine the maximum limit of annotations though, if there is one! But I guess I can worry about that when I hit the limit. *grin*

  59. Matt, from what I gather the system can be compared to a “simple” search content filter, correct?

    It looks useful as a personal tool, for instance, if I wish to restrict the results of my queries to certain groups of sites.

  60. Matt, this is simply great. I did one for Oregon Travel in a few minutes and can improve/add sites via the *excellent* Google Marker feature. I’ll drop the URL here but feel free to delete it: http://www.google.com/coop/cse?cx=018196336978593658282%3Audekr-ngfc4

  61. It would be better if the search form had a recently searched cloud function.

    eg if recently searched was wooden dummy or kung fu or jeet kune do on my site I would like the search feature to show this in a good looking way.

  62. I played yesterday with the custom search, too and I like it because I think community based search is much better. Just an example: a search buildt by an existing “Internet marketing” community, for each new user an important start to get important information about this item.

    I hope they will connect the Google bookmarking service with this search feature, This way I don’t need to add websites twice. :D

  63. Hi Matt,

    My only problem concerning this new feature from Google is that somebody can create a new “search engine” just based on his websites and since the term “powered by Google” appears it can make people (less informed ones) think that his web site is the best one about that keywords or issue.

    Can this be some type of black hat SEO ?

    Regards

    Manuel

  64. Mistah

    It’s a great idea. I’ve set up a custom search engine using the code generator.

    The only problem is that I’m getting a lot of blank lines under the search results. I notice a few posters at WMW have the same problem.

    Any chance of posting a fix?

  65. It’s definitely a great news from Google. Will it be a big blow to search engine companies like Trulia or Guruji that tacke specific verticals ? They have invested lot of money and research in building such engines but now anyone can make similar engines free using the Google co-op custom search engine.

  66. Hi Manuel,

    You could, of course, produce a CSE with black hat intentions as you outline. However, most users will recognize that all the results are coming from one site, and see little value in the search engine. Unfortunately, some people are easily fooled, and that’s just the reality of it.

    Eric

  67. Hello Matt,
    I pretty much went with your suggestion and fed the 300+ sites we know to be published locally from our region, that we already feature on our service and built a custom search with it:

    http://www.phillyfuture.org/searchphiladelphia

    I haven’t finished the refinement filters yet, but it’s very very powerful for what it is as it stands. If you’re searching for Philadelphia related subject matter, especially subject matter that is being discussed about *right now* – well the results are mind blowing.

    Great work Google :)

  68. How come I cant add my CSE box to my blog?

  69. Greyhound

    Hey Matt,

    CSE looks to be rich fodder for lots of exploration and experimentation.

    I was an early adopter and fan of co-op, and while I acknowledge that it was a little too abstract and difficult for most users, I do mourn the demise of some of it’s nifty features.

    Co-op Topics had one feature which was both great and damning at the same time. You could subscribe to a topic, and once that was done (*and you were logged in* and that was admittedly a big hurdle for the non-geek crowd), any Topic that you subscribed to was *automagically* invoked for appropriate queries. If I subscribed to an SEO Topic, and did a search on “cloaking” in my Google toolbar, I’d get the Topic results in my normal Google results page. By contrast, with CSEs, I must go to the site hosting the search box to run this search.

    I was hoping the “must log in to a google account” thing would be relaxed, as it is with CSE, and that the subscribe process would be made easier, but it’s too bad you can’t retain the transparency of Topics.

    Another thing I was hoping to see was better support for generic “blacklist” topics. For example, it’s easy to exclude certain URLs and URL patterns from a CSE, but if you maintain multiple CSEs, you have to maintain multiple lists of exclusions. Even more to the point, why not put the community to work at identifying spam (i.e., creating an “exclusion Topic/CSE”, and allow any CSE to “include” the blacklist Topic/CSE in its definition? Results for that CSE would then automatically filter out the stuff listed in the community-maintained blacklist.

    Overall, I’m really impressed with CSEs. They’re built on the incredibly powerful Topics framework (though user-defined Co-op Topics seem to have gone away! 8-( ) which means you have an extraordinarily powerful set of features lying just below the covers. The ability to do rewrites, redirects, background labels, custom boosts, custom score values, etc. means you ought to be able to very finely tune your CSE results. To me, this is where alot of the power really lies and will be unearthed over the coming weeks and months.

    For example, while it’s great to include/exclude a whole site, and with URL patterns this can get pretty specific, in my opinion, a REALLY strong CSE is going to want to manage things down at the page level. I may want to include seven articles from a site, exclude all other pages , and want to rank them in some order. Doing that with crude URL pattern matching is difficult at best. But if you’re willing to get down to the “code” level, you can fine-tune those rankings as much as you like. Of course, the next question is – will there be tools for doing this, or do you have to be an XML God? ;-)

    Lots of fun stuff to play with and think about for sure.

    -greyhound

  70. Mike

    Matt,

    It’s an interesting and somewhat spooky product…

    “could help a lot of people who want to make a search engine, but would be happy doing it not-as-a-VC-funded-startup”

    Or someone who’s somewhat unhappy and already doing something very similar as a result of physically encountered time restraints. I honestly wouldn’t mind picking your brain some day. Have you been picking mine?

    Mike

    P.S. Perhaps this is a kooky request, but say hi to “Marcia-rissa” at SEW.

  71. This is great, there are many directions to go. I don’t know what to start with.

    I started compiling a list of some of the customized searches in order to reduce redundancy.
    http://www.grimmthething.com/other/cse.htm

  72. Matt,

    Good review of a really interesting twist on the Co-op story !

    Wondering how far Google will go in allowing users to define/share their own intent based search?

    To me this looks like Yin to Google Base’s Yang. This will come full circle once Google starts allowing programmtic control over how site owners show results.
    I guess its a great start and a day is not very far when this will become a Web data source for many new online applications.

    For example if user search on keyword “campaign” on this CSE

    http://www.openapp.org/search

    I would like to show results in a tabular format based on different vendors.

    Is it possible?

    Brij

  73. Matt, will Google provide stats for search queries and/or click-thru on search results?
    Thank you :)

  74. Ian

    Are Google’s trademark policies on using the term “Google” (in site name, URL, etc) going to stay the same?

  75. Not sure Greyhound is correct – this feature is still active – its been separated however on the “old” context file to uploads on subscribed links. As far as I am aware – all the features are still active and there is nothing new whatsoever – all that has changed is that Co-op put together really good tools to create the CSE’s – I must say they have done a good job.

    SORRY one exception – there is one thing that is new I think. That’s the “search results include Subscribed Links from the following xx people” you see on the profile pages. For example if I subscribed my profile http://www.google.com/coop/profile?user=014531838225797937547 to another profile – the subscribed links on the profile I subscribed to would trigger on everyone who is also subscribed to my profile. Hope that makes sense.

  76. I LOVE IT! …….. But I cannot use it for our corporate site. :-(

    There is no way I could get approval to implement it because of the ads.

    And since we are not a non-profit, Google says “no we cannot use the ad filter.”

    If there were a subscription fee associated for companies who wanted the no-ad version, I could definitely get approval for that.

    The only other realistic options are the enterprise google search server starting at $30,000.00. I think thats fworth it, but no way I’m gonna sell that to the people who create the budget.

    We do not want the headache of installing/maintaining the server/software.

    The web based solution would be PERFECT!!! but it’s a shame that it’s not available to us.

  77. Greyhound

    Steve – if this is true (that you can still upload the old context files, create new user-defined topics, and let users subscribe to them, and offer the refinement interface for search results on google.com), I’d sure love to get a pointer to where and how that’s done. Google has removed all references to user-defined topics from the docs.

    Have you used the CSE control panel advanced tab to upload an “old style” user defined topic context file (with querytriggers and all that good stuff, and had it work? Since you can’t create a topic with the UI anymore, how the heck do you get anyone subscribed to your topic?

    Certainly if this is still possible, it’s been made even more obscure than the original topics stuff was, and it’s hard to believe you can rely on it continuing to work in that way.

    But I’d love to find out I’m wrong and to hear more about your experience with this.

    Cheers,
    -greyhound

  78. I added one of my own sites to my CSE, searched, and selected one of the results from my site. When I checked my site logs to see what the referrer is; it takes me to the search results and says in tiny print “Google search results customized by using Google Co-op”.

    There is no link provided to my CSE page or the page of my site where I included the search box and results. Is there a way to change that or make it more obvious in the referrer link that while this is a Google tool, my site also has a role to play?

    Whether on the receiving end or sending end of these searches, it would be nice to have a better way to track where it is going or coming from.

  79. this is another one, focusing on content relevant to prediction markets:
    http://gtziralis.googlepages.com/search

    I agree that there is a need to create a directory of custom search engines.
    are there any running attempts/plans towards this direction?

  80. First off, I like that your spam protection is an easy math problem.

    Next, this is the sweetest thing ever! I’ve been telling this guy at work that i wish Google gave users more control over what gets displayed in SERPs.

    One thing I would like to see Google experiment with is letting users set algorhyms. EX: I want to be able to decide how important links are, how important keyword density is, etc when my personal search engine returns results.

    Great start googlers!

  81. Matt,

    Great news this has happened. Shame about the ads thing though. I am sure a lot of corporates will exclude this service as a result. Never the less, I commissioned development of a similar thing based on Nutch a few months ago. Do you think that the power of Nutch will impact on Google’s ideas?

    This is still a great advent for Google and the more they open their search services the more they will gain that vertical search market.

    I am about 3/4 months away (lack of time) from putting my Nutch thingy live. So, if you want a sneak preview. let me know.

    Regards,
    Christian

  82. After playing with this a bit more, a few observations/issues…

    Search for hat yields no sites, hats does including a site with a description including the word hat. I realize the difference could and should perhaps impact the order of the results, but one would think that hat would return results for both hat and hats and searches for hats would only return results for the plural version.

    I’m disappointed that despite listing a series of keywords and the additional refinement of the search term that the ads presented are only based on the search term. I think most folks who use this tool are going to put specific phrases and not the bigger picture terms. For example, if my CSE contains ski oriented sites, someone looking for a ski hat would probably only search for hat since the ski qualifier would be most likely assumed. This is going to greatly reduce the relevancy of the ads shown.

    Due to the placement on the page of the message Google search engine results customized by… the first organic result looks like an ad rather than a result. Would love to see that laid out a bit differently.

    Finally, a question… Does the order the sites are listed impact the results? At least one of my CSE’s does seem to be doing that.

    I think this can become a great asset and tool for one of my sites and look forward to it’s continued growth.

  83. I’m in two minds with Google.
    They do realease some very good products and are allowing developers to use some of their technology for their own use, but how long will it be before the questions are asked as to when their dominance of the internet will stop. or where is it going. Same questions of Microsoft and Not so very long about Oracle if your memeories go back that far. Still lets enjoy the fact that they are releasing some good software for niche developers to get their teeth into. If you have that type of mind set I’m sure they will release some very tasty code over the next year, John

  84. Matt,

    I’ve just created my own Google custom search engine for heavy metal at http://find-metal.com.

    I’ve noticed that ads only appear on my inital results page. If a site visitor clicks on a refinement link Google doesn’t serve any ads. Is this standard behavior?

    Ken

  85. And, to add to a list of comments that seems to go on forever (and yet has been quiet for a few days because this is old news already), my domain finally registered.

    So, here’s my attempt at this Custom Search Engine + AJAX Search API:

    http://findingfunnies.com

    A search engine for jokes. It was fun learning to use these 2 “technologies” combined…

  86. Matt,

    I was blown away when I learned about this and jumped on the opportunity right away. I made a CSE geographically within and near the state of Massachusetts on the home design industry. So you could search ‘residential architect’,’tile’, ‘contractors’, etc and all your results are businesses located in the Massachusetts area. I already have over 1000 sites in this CSE and it kicks butt!

    http://www.mahomedesign.com

    Cheers,

    Brendan

  87. If you are interested in publicising a CSE you may want to be an early bird submitter at our CSE links directory at http://www.cselinks.com – your submission will include as long a description of your CSE as you wish together with relevant keyword lists, both of which you will be entitled to edit and keep up to date if you register as an Editor. You can also add new CSEs as you build them. All Editors will also have permission to rate and comment on other CSEs thereby constantly improving the overall quality of our visitors’ experience. The site will officially go live on Thursday.

  88. Matt et. al., I had this idea that CSEs could be used for charity and non-profit fundraising purposes, though I had some concerns about having employees of the organization click on the ads. Luckily, a quick query to AdSense support answered that question. I’d be curious to see someone actually do this with one of the organizations they support.

  89. Hey Guys,

    I’m creating a blog with a list of all the custom Google Search engines. Leave a comment or email me if you want to be listed!

    hyrulianavenger@gmail.com

  90. Matt,

    I think the real use of this search engine is coming into play. Today one of the best community review based directory got converted to a google powered search engine. They have some requests as well. Google has entered into the right domain now, the human power is going to show now.

    http://www.hedir.com/introduction/about17734.html#48889
    “We wish Google had
    As suggested by one of our member, a weighted tagging facility. …”

    I think a directory getting converted to google powered search engine is a great achieved for web as a whole. WELL DONE GOOGLE AND TO THE MIND WHO THOUGHT OF IT.

  91. Is there a way we can add the sites dynamically through some APIs. Also I will like to know more about weight.

    Week 1
    Site A – weight 0.9
    Site B: weight 0.8
    Site C: weight 0.7

    Week 2
    Site A – weight 0.7
    Site B – weight 0.9
    Site C: weight 0.5

    Is there a way I can update my next weeks through some programming, APIs? Also as I mentioned in my last comment a tag based weight will be more useful for many sites.

    I am loving this CSE stuff.

    Thanks,
    AjiNIMC

  92. This is awesome.

    I have been working on my own CSE over the last week, and one of the most useful parts to me has been the ability to exclude sites. I do numerous test searches on possible search terms, and then go through the results excluding non-pertinent sites. This is a great tool.

    Here is my custom Golf Search:

    http://www.google.com/coop/cse?cx=001964964372795679491%3Aksitu5rddc0

  93. Hi Matt, just came across this and am going to start some experimenting.. Some great stuff from Google again!

    Cheers!

  94. I think Google has really created a great thing here. I have been playing around with it creating a wedding search engine. It has been a lot of fun find all the sites to add; I have almost 2,000 now.
    And, the integration into your own website makes it even more powerful. Mine is at http://www.dhbphotography.com/weddingsearchengine.html. I am thining about buying a new domain for it and marketing it.

    THANKS GOOGLE

  95. My experiment:
    http://searchforbooks.googlepages.com
    A search engine to find books and book reviews.
    I think it could be useful.

  96. TCR23

    Matt, I’m curious, how do you think custom search will affect Google paid search? Will it make the ads more or less relevant? I’ve seen some people write that it will reduce the value of paid search ads because the natural results will be more relevant. I think maybe it’s the opposite — for broad match ads it will make them more relevant because of the context of the sites being searched. Thoughts?

    Thanks for the great overview of this cool tool.

  97. Well, it can be ued as a site search I think :)

    I need to think it over how to use it better.

  98. Hi Matt,

    When Google Custom Search was first announced I quickly created a custom search engine for the community I care about, that of food blogs – http://www.foodblogsearch.com, promoted it on my blog, and encouraged many in the community to create their own custom food blog search engines.

    Since using the custom search for a few months, it has become less and less useful. The reason? What I care about is searching 200-300 BLOGS, and the recency of a blog post is much more interesting to me and my readers than the page rank of the articles. The few sites that have high page rank (including mine) unfortunately dominate the results. If I have chosen to include a site in the search results, it by default is important to me. There is no reason to display the search results using the normal Google method which accounts for page rank.

    Ironically Google Blog Search function is not very useful either. The problem? Too many non-blogs masquerading as blogs.

    Lastly, I created my custom search engine on my Mac, using the Safari browser. About a month ago I was no longer able to make any changes to the search engine, like uploading new sites to search. Apparently the GCS edit window is no longer compatible with Safari or Firefox on a Mac.

  99. Am I allowed to click on sponsored results in my SE?

  100. Is the revenue share in existence ? was it something that is not included anymore ? Where can I find more info on it Matt ?

    thanks
    Giorgos Kontopoulos

  101. Andrew Mitton

    I recently created a custom search engine for map mashups. The search engine is at mapshark.com. Like Matt said, “there’s a lot of power under the hood” and mapshark is a good example of that. The results page presents links that take you directly to a page with a Google Map or a page that’s at least one link away from a map. No more results pages with hundreds of millions of results; instead you get 10 to 100 targeted pages.

  102. I created a search engine for russian web-developers – Codavr.Ru. It searches only best russian websites with information about web-technologies such as PHP, AJAX, Ruby etc. I like Google CSE. I think it is an excellent, one of the best of Google’s services.

  103. Loving CSE so far, it’s definitely my favorite recent offering from Google.

    I just launched a co-op health CSE, http://www.healthfind.com. Spent a lot of time finding high-quality health & medical sites. Search results are quite good so far.

    For example: Yesterday a co-worker had bluish hands, and she was worried (obviously). A few of us started googling “blue hands”, and similar terms. Top 10 results from a standard Google search showed:

    -3 pages about a plotline in the TV show Firefly called “Hands of Blue”
    -Some decent info from wrongdiagnosis.com & other sites
    -Blue hand towels from Amazon.com
    -This crazy page – tinctoris.com/archives/2007/01/19/blue-hands-and-a-torch

    The results on my co-op CSE were much better for “blue hands”. They were relevant to the symptom, and from respected medical authority sites.

    Anyways…. by this time someone had already driven her to the hospital, but I was very happy with the performance of my CSE. (after an examination, she was rather surprised when the nurse was able to remove the blueness from her hands with an alcohol swab. The blue color had bled from a new sweater that got wet, weird… Her and the ER docs had a good laugh about it.)

  104. I just want to know whether im aligned properly with google TOS.
    I have created a custom coop search engine. And also i have attached my google adsense account with Google coop search engine.

    Now my question is.
    There is already Two Blocks of Google Ad code which is default for the search results. My question Can i add one more google adcode manually or Referral code from adsense on the footer does this allowed by google tos. Anybody who is really knows the answer. Help me

  105. Eric and Matt,
    there is also http://www.coopdir.com which is a niche directory for cse created only with google co-op.

    That makes them two.

    Waiting for all you to submit for free…

  106. Yes you can post this question on Google Coop search engine discussion forum.

  107. This is another interesting tool from Google. Comparing with live search macro and yahoo pipes, I think allowing public contribution of adding good sites to cover and a handy “marker” tool is a big plus. This helps take advantage of the community. But some more powerful operators (like those on live search macro and yahoo pipes) are missing. Another idea I have is to provide integration with bookmark tools like del.icio.us, so all the sites I bookmarks are added automatically to the custom search site list. One stone two birds.

  108. Rich

    Can someone help?

    My friend and I are in the process of building a custom search engine using google. We are uploading different websites into the index etc thus creating a vertical search engine. I have two questions. 1. Using Adobe GoLive, how do we get the results to appear in a Webpage that we created? 2. How do we change the URL for the results page? (example http://www.mysite.com/google_results…etc.etc..)

    Thanks!

  109. Rich,

    If you go to the ‘code’ section of your CSE you can scroll down and expand the box that lets you return results on your own page.

    Is there yet a list of custom search engines?

    http://dark.alfredfox.com

  110. This CSE is new stuff to me. But I am excited to implement this into my “family friendly” niche. Originally I didn’t want to use a standard google search box because of the adult related keywords, but this should work out nice. better than a filter i’ve seen.

    thanks

  111. Very good idea and usefull, it would help visitors to easy search and they will recoqnize the Google engine wich gives confidence!!!

    thankx
    Frank aka Mr Lhasa Apso

  112. Thanks for the insights into CSE possibilities. It’s the next wave for search for sure.

    I’m into indie music and here’s one CSE door that could end the current era of the music industry as we know it…

    Algo: Only urls (+filters) that can virtually ensure users reaching subject relevant free music downloads “a click from landing”. That’s my touchstone.

    With these kind of to-die-for quality search returns (for indie explorers at least), I’m now trying to figure out how CSEs could be more easily shared and distributed as a plugin.

  113. I am jazzed as well. we are in the process of customizing the first Real Estate CSE: http://www.google.com/coop/cse?cx=000747579154309164948%3Annakvu69iqy

    Thanks

  114. For some reason it doesnt seem to work for me. I get a “custom search url not valid” error.
    http://greatindiandiscoveries.blogspot.com

  115. Hi,

    Can you suggest ways to promote a custom search engine? I edited a CSE to point out towards free book download sources. What are the other areas possible?

    Regards,

  116. Even though this post is over two years old, it is still relevant. And surprisingly, there still isn’t a great resource for finding custom search engines, and little from Google to make it easier. Strange.

    But then it hit me–this is a perfect job for a CSE. So I humbly submit a Custom Search Engine for Custom Search Engines. ;)
    http://tr.im/30nf

    Hope someone out there finds this useful. Let me know if you want to contribute. I’m still waiting for CSE’s to take off, as I find them incredibly useful.

  117. hi matt,
    I recently created a CSE for the VR panorama world, works pretty fine,
    you might want to take a look at it: Thanks for your great post, all the best, your marco

Leave a Comment

Your email address will not be published. Required fields are marked *

*

If you have a question about your site specifically or a general question about search, your best bet is to post in our Webmaster Help Forum linked from http://google.com/webmasters

If you comment, please use your personal name, not your business name. Business names can sound salesy or spammy, and I would like to try people leaving their actual name instead.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

css.php