Politics

Tools for Transparency - Build a Transparency Search Engine

By Scott Stadum

Google Custom Search is a simple service that lets you create your own search engine based on links that you choose. This is incredibly useful if you regularly need to search through a defined list of sites and want to exclude outside sources.  For example, if I want to search across all Sunlight and partner sites, I could easily set that up. In fact, as you'll see below, that's exactly what I did.

When you head over to the Custom Search Engine site and select "Create a Custom Search Engine," you will be asked to name and describe what it will do. You are then asked to list the sites you would like included in the search, one URL per row. The asterisk after each URL indicates that Google should search all pages in that URL. (There are some more options to play with under the 'Learn more' link.) Here's my example set up for Sunlight:

Once you have accepted the terms and clicked through to the next step, you are given options to customize your search engine and to test it out.

You'll have the option to further customize your search engine after you have finished all of the steps.

The last step gives you a code snippet to put on you website or blog.  Here you can see the results for "Pelosi" on the Sunlight Search:

Depending on your focus, you can develop a very comprehensive search engine that excludes unrelated results. One caveat: this tool doesn't include deeper levels of data that you would otherwise retrieve from querying our databases.  This basically means Google only retrieves content easily accessible on the main pages.

To get a feel for what I mean, you can test out the Sunlight Search in the search bar below:

Full post as published by Sunlight Foundation on March 03, 2011 (boomark / email).