Archives for September 2005

Fun with zip codes

Noticed via Tara Calashain’s excellent ResearchBuzz, here’s a great site (maps.huge.info) that lets you view zip code outlines in Google Maps. It’s pretty addictive to type in zip codes (assuming you’re in the U.S., and you have the particular bent of mind that makes you enjoy stuff like Google Maps). For the lazy among you, here’s how it looks:
Zip codes in New York
Notice that it’s doing something cool: after you’ve typed in the first zip code, you can type in more zip codes and it will draw multiple zip codes at once.

Suppose you wanted to do stuff like this yourself. First you’d want the Google Maps API. Next, you’d need a list of all the zip codes in the U.S. This is harder than it should be. From http://www.usps.com/ncsc/faq/:

Q: Where can I get a database or directory of all ZIP Codes and/or ZIP+4 codes along with the corresponding city, state, county, etc.? Is there an FTP site for downloading?
A: The information you are seeking is not available via download but is provided through our National Customer Support Center at (800) 238-3150 ….
[and then later] The Postal Service does not maintain any ZIP Code maps. The only related product we have at this time is the TIGER/ZIP+4 File.

And “TIGER/ZIP+4 File” is a link to a 404 page. Oy, thanks a lot! Grrr. Okay, so pop on over to the U.S. Census Bureau. For example, this page has a link to many good resources, including http://www.census.gov/tiger/tms/gazetteer/zips.txt which has a list of zip codes along with the city, state, latitude, and longitude of each zip code (the lat/lon is for the zip code center).

Technically, the Census Bureau calls their data Zip Code Tabulation Areas or ZCTAs. Why? As they put it,

This new entity was developed to overcome the difficulties in precisely defining the land area covered by each ZIP Code.

For example, zip codes represent postal routes and can’t always be represented with polygons. You can almost feel the contempt seething from the Census Bureau toward the U.S. Postal Service. I can believe that when Census and Postal personnel get in the same room, arguments break out and tempers flare. Sort of like Herbert Kornfeld, the Accounts Receivable Supervisor at The Onion that is always rumbling with Accounts Payable.

Okay, where were we? Postal Service: not helpful at all. Census Bureau: a great, easy-to-parse file. Okay, how about the boundary info with the shape of a zip code/ZCTA? How about this: http://www.census.gov/geo/www/cob/zt_metadata.html Oh, snap! Census Bureau 2, Postal Service 0!

Shall we delve a little deeper? Let’s do it; zip codes are fun. By the way, if you’re not a complete and utter nerd, or you don’t have hours to kill digging into files, the huge.info site sells a DVD with cleaner data in an easier format.

Okay, let’s examine the zip code for 94043 (where Google’s headquarters are located). Go to http://www.census.gov/geo/www/cob/z52000.html#ascii and save the file for California and decompress the zip file. The files are available in three formats, but we’re sticking with straight ASCII. Inside the zip file there are two files, a tiny one and a big one. In the tiny one, look for the zip code you’re interested in (94043 in this example):

2465
“94043”
“94043”
“Z5”
“5-Digit ZCTA”

This entry means that in the big file, the zip code 94043 is represented with an ID of 2465. If you look in the big file, you’ll find the data for that polygon in latitude/longitude format:

2465 -0.122065371904532E+03 0.374225573395062E+02
-0.122077973000000E+03 0.374482390000000E+02
-0.122077373000000E+03 0.374500390000000E+02
…. ….
-0.122077973000000E+03 0.374482390000000E+02
END

The first coordinate appears to be the center of the ZCTA, so chop off the first line and the END line and put the resulting lines in a file called 94043. Now fire up gnuplot and type these commands:

set terminal png
set output "94043.png"
set nokey
set title "Zip code: 94043"
set xlabel "Longitude"
set ylabel "Latitude"
set grid
plot '94043' with lines

Alrighty, let’s see how we did. Here’s the zipcode on the original application:

Original map of 94043 zip code

And here’s our Gnuplot map:
Gnuplot plot of 94043 zip code

There’s just too many fun things to do in the world. 🙂

Update: If you enjoy playing with zip codes, Gary Price pointed me to http://www.melissadata.com/Lookups/index.htm where you can look up neat things like demographics and business counts.

UI fun: Remove result

One request we sometimes hear is for the ability to modify Google results, especially to block unwanted sites. A few eagle-eyed people may have noticed a user-interface experiment on Google that adds the ability to remove results. Here’s what you’d see. Imagine that you did the search [lynx paw clipart], and you notice one particular result that looks spammy:
search snippet
You check the cached page, and you notice that if you turn off Cascading Style Sheets, there’s a bunch of spammy text:
spammy text

At that point, your options would normally be to 1) ignore that result, or 2) report the url to Google via our spam report form.

But if you’re in this experiment, you’ll have newfound powers. Click the “Remove result” link and with one click you can drop that url from your search results. It looks like this:
blocking a result
By default, it will only block that url for that particular search. If you’re really annoyed, you can click “More options” and you’ll get two more choices: block this url from all future searches and (my personal favorite) the ability to block the entire host from all future searches. Here’s what it looks like:
More blocking options

Before I tackle some questions, I want to send mad props to the people who worked on this in our New York office. This is one of my favorite UI experiments; it’s neat to give users the ability to modify some parts of Google, and I especially like the Ajax-y aspects of it. Okay, let me try to anticipate some questions.

Q: Why don’t I see “Remove result”? I want it! I want it baaad!
A: Dude, did you not read the last paragraph? It’s an experiment; if we showed it to everyone, it would be a beta. 😉 Oh, alright. Here’s a tip. Go sign up for My Search History/Personalized Search. You don’t have to store your searches, but you will need a Google Account.

Q: How exactly do I try this out?
A: If you have an email address like johnpublicuser@gmail.com, you should be able to go to http://www.google.com/psearch and sign in with “johnpublicuser@gmail.com” and your password–make sure to click “Remember me on this computer.” If you don’t have a Gmail account, you can create a Google Account at https://www.google.com/accounts/ . Once you are logged into your Google Account, click on “Personalized Search” on the left-hand side:
personalized search
Once you are in Personalized Search, you should see the Remove Result link any time that you’re signed in.

Q: Are you going to use this data to improve general search?
A: It’s too early to say. It’s still an experiment that may not even launch; it depends on how people like it. The form or format could also change as well.

Again, thanks to the people who worked on this. I’ll be curious to see whether people enjoy it.

css.php