I got a USB Missile Launcher for Christmas. The manufacturer, Dream Cheeky, provides software–but only for Windows XP. And I thought to myself, “wouldn’t it be fun to practice some USB reverse engineering skills?” Because another Christmas present was a USB protocol analyzer from Total Phase. I should note that plenty of other people have apparently already written drivers/software for USB missile launcher toys, but I wanted to poke around myself.

Total Phase makes a high-speed USB 2.0 protocol analyzer for $1200, or a regular-speed USB protocol analyzer for $400. Here’s a trick someone mentioned: if you get the cheaper protocol analyzer and need to work with a high-speed USB device, you may be able to plug the high-speed device into a low-speed USB hub to slow the device down.

I decided to start with ladyada’s excellent guide to hacking a Kinect by reverse engineering USB packets. So here’s what I did.

Step 1. Make sure the device works. It would suck to attempt to reverse engineer a broken device. I keep a Windows XP computer lying around, so I downloaded the software for it, installed the program, and plugged in the USB rocket launcher. After the install, XP wanted to restart, so I restarted the XP computer (unplugging my USB rocket launcher after the computer was off), then started the rocket launcher software back up, then plugged in the USB device. Sure enough, everything worked fine. The controls are: pan left/right, tilt up/down, and fire. Tip: the rocket launcher uses bursts of air, so don’t jam the foam rockets down hard on the launcher.

Step 2. Probe the device. I plugged the USB rocket launcher into a Linux machine running Ubuntu 10.04 (Lucid Lynx). I ran the command sudo lsusb -vv and the relevant info from the list of USB devices on my system was this:

Bus 002 Device 045: ID 0a81:0701 Chesen Electronics Corp. USB Missile Launcher
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               1.10
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0         8
  idVendor           0x0a81 Chesen Electronics Corp.
  idProduct          0x0701 USB Missile Launcher
  bcdDevice            0.01
  iManufacturer           1 Dream Link
  iProduct                2 USB Missile Launcher v1.0
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           34
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xa0
      (Bus Powered)
      Remote Wakeup
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 No Subclass
      bInterfaceProtocol      0 None
      iInterface              0 
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.00
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength      52
         Report Descriptors: 
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              20
Device Status:     0x0000
  (Bus Powered)

Note that my Vendor ID = 0x0a81 and my Product ID = 0×0701. Also note that bNumEndpoints = 1. An endpoint is a channel for USB data communication. Then we get the Endpoint info:

      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              20

According to ladyada’s write-up, the “IN” means that data goes IN to the computer from the device, and the “Interrupt” transfer type is good for sending large amounts of small data quickly (e.g. a USB mouse).

Step 3. Prepare your Linux system to talk to the device. First, let’s review ladyada’s steps, which is for Windows. She installs libusb-win32 and then runs a program called inf-wizard to make a driver shell. Then plugging the device into Windows will attach the LibUSB-win32 device driver. Next, she installed Python and PyUSB.

I wanted to stick with Linux. I didn’t need libusb-win32 or inf-wizard.exe, and I already had Python installed. So my next step was to download PyUSB, extract the zip into a directory, change into that directory, then run sudo python setup.py install in that directory to install PyUSB. Since you’re installing PyUSB system-wide, you do need to run that command with “sudo” to run it as root.

Step 4. Write a short program on your Linux machine to talk to the device. I made a file missile-launcher.py and executable access with “chmod ugo+rx missile-launcher.py” next. Here’s the short program I ended up with:

#!/usr/bin/python

import usb.core
import usb.util
import sys
 
# find our device
dev = usb.core.find(idVendor=0x0a81, idProduct=0x0701)
 
# was it found?
if dev is None:
    raise ValueError('Device not found')

# Linux kernel sets up a device driver for USB device, which you have
# to detach. Otherwise trying to interact with the device gives a
# 'Resource Busy' error.
try:
  dev.detach_kernel_driver(0)
except Exception, e:
  pass # already unregistered
 
# set the active configuration. With no arguments, the first
# configuration will be the active one
dev.set_configuration()
 
print "all done"

Note that my “idVendor=0x0a81, idProduct=0×0701″ parameters use the values I found from lsusb -vv. If you compare against ladyada’s short program you’ll notice one major difference. My code has these lines:

# Linux kernel sets up a device driver for USB device, which you have
# to detach. Otherwise trying to interact with the device gives a
# 'Resource Busy' error.
try:
  dev.detach_kernel_driver(0)
except Exception, e:
  pass # already unregistered

Ladyada’s PyUSB program for Windows didn’t have anything like that. But when I ran the program under Linux, I got the error message “usb.core.USBError: Resource busy”. It turns out that the Linux kernel tries to use a default kernel driver, and that prevents my program from talking to the device. Detaching the kernel driver lets me talk to the device just fine. I picked up this tip Ken Shirriff’s post about a USB panic button with Linux and Python. In theory you could also unbind the USB device from a command-line, but I prefer to do it right in my PyUSB program directly.

Note that you will need to run the python program as root, e.g. “sudo ./missile-launcher.py” or else you’ll get a warning message like “usb.core.USBError: Access denied (insufficient permissions)”.

At this point, you have a small working program that opens up a connection to the USB rocket launcher. If the USB rocket launcher isn’t plugged in, you’ll get a “Device not found” error, and if the USB device is plugged in, you’ll get an “all done” message and the program exits gracefully.

Step 5: Try to read from the USB device. In ladyada’s guide, she tried sending 0b11000000 = 0xC0 (“Read Vendor data from Device”) to a Kinect. I got no response from that, but I did get a response sending 0b10000000. That corresponds to sending:
- a ’1′ to read from the device
- a message type of ’00′ = standard. Ladyada got a response sending to ’10′ = vendor
- ’000′ (reserved bits, so always 0)
- ’00′ to say the recipient of the message is the device.
Then sending a request of 0 got back a result of two zero bytes:

bRequest  0
array('B', [0, 0])

Interestingly, running the same program again would get a “usb.core.USBError: Unknown error” response. At that point, I would unplug the USB device and then plug it back in to reset it. I didn’t get any other responses from trying to send message types of class or vendor (as opposed to standard), nor did I get any responses from try to send messages to the interface or endpoint (as opposed to the device). See ladyada’s guide for more details about fuzzing the device and what all the various bit fields mean.

Step 6: Set up the Linux computer to use the Total Phase Beagle. The CD worked nicely with HTML documentation on it. First, you copy some udev rules so that the device is writable by anyone when the Beagle is plugged in:


cd /media/Total\ Phase/drivers/linux/
sudo cp 99-totalphase.rules /etc/udev/rules.d/
sudo chmod 644 /etc/udev/rules.d/99-totalphase.rules


If you’ve already plugged in the Beagle, you’ll need to unplug it and plug it back in for these rules to fire. Next, you’ll need the Data Center software. You can get it off the CD, but I’d recommend downloading the latest software and user’s manual from the website instead. My CD had software version 4.20 for example and the website was up to 5.01. Extract the software zip file (either from online or the CD). Then follow the directions in the online manual (or user manual PDF). The directions according to the manual are

1. Install the USB drivers and Data Center software. Copying the udev rules is enough for USB drivers on Linux. Unpacking the zip is all you need for the Data Center software, because the executable is self-contained.
2. Plug the Beagle analyzer into the analysis machine. This was my Linux machine.
3. Plug the Beagle analyzer into the bus to be analyzed. In this case, this was my XP computer. Don’t plug the USB missile launcher in yet though.
4. Start the Data Center software. Run the program “Data Center” in the directory you extracted from the .zip file. Follow the rest of the instruction in the Quick Start section.

But I’m also a big believer in picking out really big goals too. In 2010 I trained for a few months so I could climb Kilimanjaro, for example. For 2011, I went skydiving and ran my first marathon in San Francisco. 2012 ended up being a blur: lots of work, plus trips and time with family. I still managed to run a couple more marathons and finish a Tough Mudder race back in Kentucky.

Which brings me to my big goal for 2013. On April 6th, 2013, I’m going to try to run 50 miles for the American River 50. It’s an ultramarathon from Sacramento to Auburn with flat ground for the first half and single-track dirt trails on the second half. The cutoff limit for finishing is 13 hours. As a running friend put it, “You really just need to make the 32 mile cutoff in 7.5 hours or so; after that you could walk the rest of the way!”

In theory, if you run the American River 50 in 11 hours then you can qualify for the Western States 100-Mile Endurance Run. But I’ve gotten a chance to run the trail half of the course a couple times now, and I don’t think there’s any way I can finish that fast. So I’m just shooting to finish in 13 hours. By the way, if you’re interested in ultramarathons (any race longer than a marathon) or trail running, there’s a great documentary about the 2010 Western States race called Unbreakable.

So that’s what I’ll be doing this weekend. Wish me luck, and if anyone want to suggest running music, please leave a comment pointing to a good YouTube video for running. Thanks!

My five-point plan for the Mitt Romney Halloween costume went like this:

1. Start with a suit and tie.
2. Put a little silver in my sideburns. My wife used some silver and white eye shadow.
3. Swoosh the hair up a little bit and lock it in with hairspray.
4. Take off the glasses, and
5. Add a flag. Boom!

This was a fun, easy, comfortable costume. I practiced a few of Mitt Romney’s catchphrases and I think people really enjoyed seeing “Matt Romney” around the Googleplex.

My wife and I are also trying something new for Halloween. I remember as a kid when I got a full-size candy bar for Halloween–it blew me away! I still remember those neighbors as really cool. Recently someone reminded me of that experience.

So this year, the first couple dozen kids who stop by our house are in for a nice treat:

Happy Halloween, everyone! You might also enjoy seeing some of my other Halloween costumes.

To keep myself honest, here I am, clean-shaven, with a today’s issue of USA TODAY on November 1st:

By the way: Kim Kardashian is getting divorced‽! I really thought they had a love for the ages.

This is going to be an extra-fun 30 day challenge because I’m joining up with Duane Forrester of Bing and we’re both raising money for the cause. I have to warn Duane though: he might not realize how fast my moustache grows. For example, here I am just an hour after I shaved clean:

That’s right Duane, I can grow a moustache in an hour, so look out! By the way, thanks to Mike Halvorsen for asking a timely question about Movember during our recent live Q&A on YouTube.

This should be quite a fun month:
- If you’re a woman (or a man who can’t or won’t grow a moustache), you can still help out the cause by donating to our team.
- Men, I hope you will grow a moustache this month. A great cause like Movember only comes around once a year–why not join us? All you have to do is not shave. It doesn’t get any easier than that.
- The Twitter tag is #seostache if you want to root us on. I hope to see some serious stache from fellow people of search!

That sounded like a good challenge. I finished the costume and taped a video, but unfortunately I didn’t have enough time to get the video to the quality level that I wanted. Here’s the video:

A couple quick things: 1) I know the video is cheesy and lo-fi, but that’s what I was going for. 2) With a little more time and more work on the video (e.g. better lighting, turning off automatic brightness on the video camera, a little more time spent on the greenscreen key) I think it could have been pretty great. But sometimes you run out of time, and October was a busy month.

Maybe I’ll write a bit more about the costume and the video later. Happy Halloween everyone!

To make the glowing red eye, I punched a small hole in the pumpkin, taped an LED to a battery, and pushed the LED from the center of the pumpkin through the small hole. I thought about doing an actual red laser, but 1) I don’t want to blind any kids walking by my pumpkin, and 2) laser beams aren’t visible unless there’s fog or smoke or something for the beam to hit.

To make the side “gun” sections of the pumpkin, I had a small dowel rod lying around, so I just cut two small stakes from the dowel rod and jammed them into the pumpkin sides. It’s a bit rickety, but it works. For the video, I played some turret sound effects and moved my red laser over the pumpkin as if another turret was locking onto it.

I didn’t want to take long to carve my pumpkin. I finished it in maybe an hour, but check out this really elaborate pumpkin that someone else did. And it turns out that they work at Google too!

I like how my pumpkin came out overall:

Happy Halloween, and I hope you’re getting creative with your pumpkin carving!

I’ve gotten some great suggestions for good books to read in previous years. Here’s how I expressed my summertime reading preferences early on:

Okay, I’m looking for fun, light reading for my vacation. I don’t want search stuff, I don’t want heavy reading, I don’t want geopolitics or history.

Things like The Curious Incident of the Dog in the Night-Time. Or Terry Pratchett. Or early William Gibson. Cheesy cyberpunk if they don’t get the computer stuff too wrong. Neil Gaiman. Transmetropolitan.

Based on that, what books would you recommend for fun summer reading?

When I was in grad school, I would have loved to have a device like the Kinect. So I decided to run my own contest:

The first $1000 prize goes to the person or team that writes the coolest open-source app, demo, or program using the Kinect. The second prize goes to the person or team that does the most to make it easy to write programs that use the Kinect on Linux.

It’s time to announce the prize winners. There’s been so many cool things going on with the Kinect that instead of two winners, I ended up declaring seven $1000 winners.

Open-source Application or Demo

I picked two winners in this category.

• Tomoto Washio for Kinect Ultra Seven. This program lets you transform into Ultra Seven with lots of different powers. It’s a really fun demo–check out the videos:
  
• Tiago Serra and the SenseBloom team for XBox Kinect OSCeleton. The application sends 3D tracked body skeletons through the OSC protocol. Essentially, it’s a do-it-yourself motion capture system:
  

People that have made it easier to write programs for the Kinect

A ton of people have made the Kinect more accessible on Linux or helped the Kinect community. I ended up picking five winners.

• Hector Martin
• Arne Bernin
• Kyle Machulis
• Brandyn White
• Joshua Blake

All of these individuals pushed things forward so others can develop great programs on the Kinect more easily. Congratulations to all the winners, and to everyone doing neat things with their Kinect!

Wha wha wha?? That’s right: 2009. As 2008 drew to a close, I jotted down some predictions, but I never hit the “publish” button. Some of them weren’t fully-formed (or even complete sentences!), so I won’t print all of them. But some are interesting. Here are a few:

- “Market share for IE falls below 50% in some countries in Europe.” TRUE according to StatCounter. Here’s a graph:

- “Cuil will be acquired by Baidu.” Definitely a MISS. This was an educated guess on my part. I didn’t think Cuil would do well, and someone named Greg Penner was on the board of Cuil and also a board member of Baidu.

- “More people will realize the inevitable truth that Bill Gates saw years ago and that Apple has chased since the introduction of the ROKR: of all the devices in your pocket, the only one you’re not willing to give up is your phone. Therefore, all personal gadgets will eventually be subsumed by your phone. Camera? Already part of your phone. Pen and notebook? Quite close. Video camera? Almost there, give it a couple more years. Car keys, wallet? It will come. In five years, your phone will have fingerprint authentication and be able to start your car or pay for groceries with contactless/RFID chips. It’s all coming. In 10 years you’ll use your phone to authenticate yourself at the doctor, authenticate prescriptions, and store your personal health history, not to mention all your desktop preferences, bookmarks, browser add-ons, and keys to which music you have permission to stream or download from the cloud.” I call this TRUE. Most people now agree that your phone is a personal computer in your pocket. Back in 2008, not everyone realized this.

- “By the way, if anyone figures out how to lick the problem of satisfactory output/input, e.g. heads-up displays or retinal lasers and a virtual keyboard or something with as high a bandwidth as normal typing (and they will), your computer will migrate into your phone. Solving the input/output problem is one of the most important problems for the next decade. Witness the efforts that companies have put into pen-based computing and voice recognition, for example. Re-examine the success of some major products in the new light of better input/output mechanisms:
- Google Maps: direct manipulation
- iPhone: touch and multitouch
- Wii: accelerometer and optical sensors
Smart computer scientists and engineers should be paying as much attention to sonar, inertial, and optical sensing technology as they do to the change from hard-drive platters to solid-state storage.” Um. This one was more of a rant then a prediction.

- Here’s what I wrote back in 2008: “Canny self-promoters plus a few genuine believers will jump on the Facebook backlash early, either for privacy/personal information or for keeping Facebook’s data to itself. But unless the site makes a Beacon-level mistake, Facebook will experience massive growth in 2009, and the Facebook backlash won’t begin in earnest until 2010 or 2011.” Right now it feels true. Maybe we’ll check back after 2011 to see whether that’s accurate.

- “Semantic web technology won’t take off, at least not in the generally-accepted ways.” I’m going to call that TRUE. Remember how Web 3.0 was going to be the semantic web? You don’t hear that meme as much anymore.

- “The most interesting “savviness” test for me will be which of Yahoo, Facebook, or Microsoft let you freely retrieve your email with them into Gmail. Odds are that none will allow it, but if I had to pick one, it would be Yahoo.” A double MISS, because Hotmail did allow POP3 access in 2009. Meanwhile, I believe Yahoo still wants $20/year for POP access. I don’t think you can fetch Facebook mail using POP or IMAP.

- “Vanity iPhone apps.” This was a prediction that individuals would commission their own personal iPhone apps that collected their blog posts, tweets, photos, etc. The other part of the idea was that conferences would commission conference-specific apps. While a few savvy conferences have done this, I wouldn’t call either trend widespread. So it’s a MISS.

- “In the same way that millions of people dropped their land-line for a cell phone connection, thousands of people will attempt to go digital by scanning their photos, ripping their CDS, digitizing their old VHS tapes, and scanning their papers.” MISS, I did a lot of those things in 2009, but average people didn’t. The brunt of the analog to digital migration will happen as data is “born digital” from the beginning. Millions of people will have physical photos stuck in boxes that they don’t look at much, plus digital photos on Facebook, Flickr, or Picasa.

- “Google Chrome becomes one of Google’s most successful products.” I think this is TRUE. (Remember, this prediction was less than four months after Chrome was introduced in September, 2008. Back then I was going out more on a limb.)

- “Breakthroughs in green technology push renewable energy closer to the mainstream, but not into the mainstream. As many people focus their attention in this area, unexpected surprises abound, such as the do-it-yourself solar roof installation or practical solar shingles. Expect to see more green technology scams as well though.” Meh. MISS.

- “Apple will weather the 2009 recession much better than most people expect as people continue to buy with their heart, not always with their head. More people will realize that ‘Once you go Mac, you never go back.’ ” I’m calling it TRUE. Apple radically outperformed the Dow in 2009. Apple has done very well competing against MSFT in operating systems.

- “Obama’s administration mandates that all federal buildings much use Compact Fluorescent Lightbulb (CFL) technology before his presidential term ends.” I’ve still got at least two more years on this prediction. If the mandate is for LED lightbulbs, I’m still counting it.

- I thought some public officials would start lifestreaming as the ultimate endorsement of transparency. That’s a MISS. Chalk it up to believing that politics would be different after the 2008 election. Oh well.

- “Microsoft will demo a snazzy new phone operating system, but it won’t get much traction, either because a) it’s not as snazzy as the iPhone or b) people don’t trust it to be as open as other mobile operating systems.” Given that this was a prediction for 2009, I’ll call it TRUE. Microsoft didn’t get a lot of phone traction in 2009.

- “Someone not officially associated with Android will work on an “iPhone skin” for Android to make Android look like an iPhone.” TRUE. Looks like that did happen in 2009.

- “Twitter’s 2009 business model consists of premium features that either individuals can pay for (at a nominal rate such as $1/month or $10/year) or that businesses can set up for self-contained twittering. The premium features include richer API access as well as some new features, such as the ability to save tweets to be posted in the future or periodically.” That, my friends, is a MISS. Although Twitter is now charging for richer access to Twitter data, so maybe not a horrible miss.

- “Hacking of PC clients will decrease, but hackers will shift their sites to web server software. Massive databases of cross-site scripting attacks will be traded in the underground. Script kiddies will launch dictionary attacks. Anyone that writes their own web server software will probably be at a high risk of being hacked.” Overall I’m going to call this TRUE. As individual PCs become more secure, there’s a multi-year trend toward hacking webservers instead. I fear we’re just at the beginning of this.

- “When Android opens premium apps, one of the big controversies will be developers who take great premium-app ideas for the iPhone and rewrite the ideas behind the app for Android, but without the permission of the iPhone developer. Expect flashlights, lighters, beer, farting applications, goldfish, etc.” I’m not sure if this is TRUE or a MISS. I haven’t read a lot of complaints from iPhone app developers about Android app developers stealing their idea and writing their own apps.

- “Multiple people write a bit of JavaScript that site owners can add to their page to guilt IE6 users to upgrade to a new browser. Any browser, just something that was released after 2001 (IE6 was released in 2001?!?). The JavaScript code becomes a viral sensation and sweeps across the net. Whoever authors or hosts this JavaScript gains status around the web. Through the collective work of a bunch of savvy webmasters, SEOs, and site owners, IE6′s market share drops to 5% in some markets by the end of the year.” IE6 is slowly riding into the sunset, but call it a MISS. A few businesses will probably be running IE6 a decade from now for their internal apps.

- “eBay introduces spell correction for search queries which delivers a small boost for their profits.” I’m going to call this TRUE because the search [ipod touc] returns “Did you mean: ipod touch?” at the top of the eBay’s results now.

Okay, it’s your turn. Which of your predictions for 2009 or 2010 were spot on or off-base?

Before I joined Google, I was a grad student interested in topics like computer vision, motion self-tracking, laser scanners–basically any neat or unusual sensing device. That’s why I was so excited to hear about the Kinect, which is a low-cost ($150) peripheral for the Xbox. The output from a Kinect includes:
- a 640×480 color video stream.
- a 320×240 depth stream. Depth is recovered by projecting invisible infrared (IR) dots into a room. You should watch this cool video to see how the Kinect projects IR dots across a room. Here’s a single frame from the video:

What’s even better is that people have figured out how to access data from the Kinect without requiring an Xbox to go with it. In fact, open drivers for the Kinect have now been released. The always-cool Adafruit Industries, which offers all sorts of excellent do-it-yourself electronics kits, sponsored a contest to produce open-source drivers for the Kinect:

First person / group to get RGB out with distance values being used wins, you’re smart – you know what would be useful for the community out there. All the code needs to be open source and/or public domain.

Sure enough, within a few days, the contest was won by Héctor Martín Cantero, who is actually rolling his reward into tools and devices for fellow white-hat hackers and reverse engineers that he works with, which is a great gesture. Okay, so where are we now? If I were still in grad school, I’d be incredibly excited–there’s now a $150 off-the-shelf device that provides depth + stereo and a lot more.

It’s time for a new contest

I want to kickstart neat projects, so I’m starting my own contest with $2000 in prizes. There are two $1000 prizes. The first $1000 prize goes to the person or team that writes the coolest open-source app, demo, or program using the Kinect. The second prize goes to the person or team that does the most to make it easy to write programs that use the Kinect on Linux.

Enter the contests by leaving a comment on this blog post with a link to your project, along with a very-short description of what your project does or your contribution to Kinect hacking. The contest runs until the end of the year: that’s Dec. 31st, 2010 at midnight Pacific time. I may ask for outside input on who should be the winner, but I’ll make the final call on who wins.

To get your ideas flowing, I’ll offer a few suggestions. Let’s start with the second contest: making the Kinect more accessible. In my ideal world, would-be hackers would type a single command-line, e.g. “sudo apt-get install openkinect” and after that command finishes, several tools for the Kinect would be installed. Maybe a “Kinect snapshot” program that dumps a picture, a depth map, and the accelerometer values to a few files. Probably some sort of openkinect library plus header files so that people can write their own Kinect programs. I would *love* some bindings to a high-level language like Python so that would-be hobbyists can write 3-4 lines of python (“import openkinect”) and start trying ideas with minimal fuss. To win the second contest, you could write any of these libraries, utilities, bindings or simplify installing them on recent versions of Linux/Ubuntu (let’s say 10.04 or greater).

Okay, how about some ideas for cool things to do with a Kinect? I’ll throw out a few to get you thinking.

Idea 1: A Minority Report-style user interface where you can open, move, and close windows with your movements.

Idea 2: What if you move the Kinect around or mount it to something that moves? The Kinect has an accelerometer plus depth sensing plus video. That might be enough to reconstruct the position and pose of the Kinect as you move it around. As a side benefit, you might end up reconstructing a 3D model of your surroundings as a byproduct. The folks at UNC-Chapel Hill where I went to grad school built a wide-area self-tracker that relied on a Kalman filter to estimate a person’s position and pose. See this PDF paper for example.

Idea 3: Augmented reality. Given a video stream plus depth, look for discontinuities in depth to get a sort of 2.5 dimensional representation of a scene with layers. Then add new features into the video stream, e.g. a bouncing ball that goes between you and the couch, or behind the couch. The pictures at the end of this PDF paper should get you thinking.

Idea 4: Space carver. Like the previous idea, but instead of learning the 2.5D layers of a scene from a singe depth map, use the depth map over time. For example, think about a person walking behind a couch. When you can see the whole person, you can estimate how big they are. When they walk behind the couch, they’re still just as big, so you can guess that the couch is occluding that person and therefore the couch is in front of the person. Over time, you could build up much more accurate discontinuities and layers for a scene by watching who walks behind or in front of what.

Idea 5: A 3D Hough transform. A vanilla Hough transform takes a 2D image, looks for edges in the image, and then runs some computation to determine lines in the image. A 3D Hough transform finds planes in range data. I’ve done this with laser rangefinder data and it works. So you could take a depth data from a Kinect and reconstruct planes for the ground or walls in a room.

Idea 6: What if you had two or more Kinects? You’d have depth or range data from the viewpoint of each Kinect and you could combine or intersect that data. If you put two Kinects at right angles (or three or four Kinects around a room, all pointing into the room), could you reconstruct a true 3D scene or 3D object from intersecting the range data from each Kinect?

I hope a few of these ideas get you thinking about all the fun things you could do with a Kinect. I’m looking forward to seeing what cool ideas, applications, and projects people come up with!