I got a USB Missile Launcher for Christmas. The manufacturer, Dream Cheeky, provides software–but only for Windows XP. And I thought to myself, “wouldn’t it be fun to practice some USB reverse engineering skills?” Because another Christmas present was a USB protocol analyzer from Total Phase. I should note that plenty of other people have apparently already written drivers/software for USB missile launcher toys, but I wanted to poke around myself.

Total Phase makes a high-speed USB 2.0 protocol analyzer for $1200, or a regular-speed USB protocol analyzer for $400. Here’s a trick someone mentioned: if you get the cheaper protocol analyzer and need to work with a high-speed USB device, you may be able to plug the high-speed device into a low-speed USB hub to slow the device down.

I decided to start with ladyada’s excellent guide to hacking a Kinect by reverse engineering USB packets. So here’s what I did.

Step 1. Make sure the device works. It would suck to attempt to reverse engineer a broken device. I keep a Windows XP computer lying around, so I downloaded the software for it, installed the program, and plugged in the USB rocket launcher. After the install, XP wanted to restart, so I restarted the XP computer (unplugging my USB rocket launcher after the computer was off), then started the rocket launcher software back up, then plugged in the USB device. Sure enough, everything worked fine. The controls are: pan left/right, tilt up/down, and fire. Tip: the rocket launcher uses bursts of air, so don’t jam the foam rockets down hard on the launcher.

Step 2. Probe the device. I plugged the USB rocket launcher into a Linux machine running Ubuntu 10.04 (Lucid Lynx). I ran the command sudo lsusb -vv and the relevant info from the list of USB devices on my system was this:

Bus 002 Device 045: ID 0a81:0701 Chesen Electronics Corp. USB Missile Launcher
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               1.10
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0         8
  idVendor           0x0a81 Chesen Electronics Corp.
  idProduct          0x0701 USB Missile Launcher
  bcdDevice            0.01
  iManufacturer           1 Dream Link
  iProduct                2 USB Missile Launcher v1.0
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           34
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xa0
      (Bus Powered)
      Remote Wakeup
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 No Subclass
      bInterfaceProtocol      0 None
      iInterface              0 
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.00
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength      52
         Report Descriptors: 
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              20
Device Status:     0x0000
  (Bus Powered)

Note that my Vendor ID = 0x0a81 and my Product ID = 0×0701. Also note that bNumEndpoints = 1. An endpoint is a channel for USB data communication. Then we get the Endpoint info:

      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              20

According to ladyada’s write-up, the “IN” means that data goes IN to the computer from the device, and the “Interrupt” transfer type is good for sending large amounts of small data quickly (e.g. a USB mouse).

Step 3. Prepare your Linux system to talk to the device. First, let’s review ladyada’s steps, which is for Windows. She installs libusb-win32 and then runs a program called inf-wizard to make a driver shell. Then plugging the device into Windows will attach the LibUSB-win32 device driver. Next, she installed Python and PyUSB.

I wanted to stick with Linux. I didn’t need libusb-win32 or inf-wizard.exe, and I already had Python installed. So my next step was to download PyUSB, extract the zip into a directory, change into that directory, then run sudo python setup.py install in that directory to install PyUSB. Since you’re installing PyUSB system-wide, you do need to run that command with “sudo” to run it as root.

Step 4. Write a short program on your Linux machine to talk to the device. I made a file missile-launcher.py and executable access with “chmod ugo+rx missile-launcher.py” next. Here’s the short program I ended up with:

#!/usr/bin/python

import usb.core
import usb.util
import sys
 
# find our device
dev = usb.core.find(idVendor=0x0a81, idProduct=0x0701)
 
# was it found?
if dev is None:
    raise ValueError('Device not found')

# Linux kernel sets up a device driver for USB device, which you have
# to detach. Otherwise trying to interact with the device gives a
# 'Resource Busy' error.
try:
  dev.detach_kernel_driver(0)
except Exception, e:
  pass # already unregistered
 
# set the active configuration. With no arguments, the first
# configuration will be the active one
dev.set_configuration()
 
print "all done"

Note that my “idVendor=0x0a81, idProduct=0×0701″ parameters use the values I found from lsusb -vv. If you compare against ladyada’s short program you’ll notice one major difference. My code has these lines:

# Linux kernel sets up a device driver for USB device, which you have
# to detach. Otherwise trying to interact with the device gives a
# 'Resource Busy' error.
try:
  dev.detach_kernel_driver(0)
except Exception, e:
  pass # already unregistered

Ladyada’s PyUSB program for Windows didn’t have anything like that. But when I ran the program under Linux, I got the error message “usb.core.USBError: Resource busy”. It turns out that the Linux kernel tries to use a default kernel driver, and that prevents my program from talking to the device. Detaching the kernel driver lets me talk to the device just fine. I picked up this tip Ken Shirriff’s post about a USB panic button with Linux and Python. In theory you could also unbind the USB device from a command-line, but I prefer to do it right in my PyUSB program directly.

Note that you will need to run the python program as root, e.g. “sudo ./missile-launcher.py” or else you’ll get a warning message like “usb.core.USBError: Access denied (insufficient permissions)”.

At this point, you have a small working program that opens up a connection to the USB rocket launcher. If the USB rocket launcher isn’t plugged in, you’ll get a “Device not found” error, and if the USB device is plugged in, you’ll get an “all done” message and the program exits gracefully.

Step 5: Try to read from the USB device. In ladyada’s guide, she tried sending 0b11000000 = 0xC0 (“Read Vendor data from Device”) to a Kinect. I got no response from that, but I did get a response sending 0b10000000. That corresponds to sending:
- a ’1′ to read from the device
- a message type of ’00′ = standard. Ladyada got a response sending to ’10′ = vendor
- ’000′ (reserved bits, so always 0)
- ’00′ to say the recipient of the message is the device.
Then sending a request of 0 got back a result of two zero bytes:

bRequest  0
array('B', [0, 0])

Interestingly, running the same program again would get a “usb.core.USBError: Unknown error” response. At that point, I would unplug the USB device and then plug it back in to reset it. I didn’t get any other responses from trying to send message types of class or vendor (as opposed to standard), nor did I get any responses from try to send messages to the interface or endpoint (as opposed to the device). See ladyada’s guide for more details about fuzzing the device and what all the various bit fields mean.

Step 6: Set up the Linux computer to use the Total Phase Beagle. The CD worked nicely with HTML documentation on it. First, you copy some udev rules so that the device is writable by anyone when the Beagle is plugged in:


cd /media/Total\ Phase/drivers/linux/
sudo cp 99-totalphase.rules /etc/udev/rules.d/
sudo chmod 644 /etc/udev/rules.d/99-totalphase.rules


If you’ve already plugged in the Beagle, you’ll need to unplug it and plug it back in for these rules to fire. Next, you’ll need the Data Center software. You can get it off the CD, but I’d recommend downloading the latest software and user’s manual from the website instead. My CD had software version 4.20 for example and the website was up to 5.01. Extract the software zip file (either from online or the CD). Then follow the directions in the online manual (or user manual PDF). The directions according to the manual are

1. Install the USB drivers and Data Center software. Copying the udev rules is enough for USB drivers on Linux. Unpacking the zip is all you need for the Data Center software, because the executable is self-contained.
2. Plug the Beagle analyzer into the analysis machine. This was my Linux machine.
3. Plug the Beagle analyzer into the bus to be analyzed. In this case, this was my XP computer. Don’t plug the USB missile launcher in yet though.
4. Start the Data Center software. Run the program “Data Center” in the directory you extracted from the .zip file. Follow the rest of the instruction in the Quick Start section.

Suppose you have a device and want a Linux device driver for it. There are a few steps you’ll need to take. One of the heroes in this area is Greg Kroah-Hartman. Greg wrote USBView, which is a Linux tool to enumerate a list of USB devices. He’s also done a massive amount of documentation as we’ll see below. One of his more eye-catching tricks is to walk a classroom through the process of writing a Linux driver for a USB thermometer live and in real-time. In addition to all the work he does for Linux in general, he recently announced a program to work with manufacturers and provide Linux drivers for new devices for free. That’s right, manufacturers get a free driver. From the original announcement:

All that is needed is some kind of specification that describes how your device works, or the email address of an engineer that is willing to answer questions every once in a while. A few sample devices might be good to have so that debugging doesn’t have to be done by email, but if necessary, that can be done.

In return, you will receive a complete and working Linux driver that is added to the main Linux kernel source tree. The driver will be written by some of the members of the Linux kernel developer community (over 1500 strong and growing).

That is majorly good karma for Linux and Greg. But if you’re not a manufacturer, here are the steps that you’d look into.

1. Get documentation of the USB protocol, or reverse engineer the protocol. It’s far easier if you can get documentation of the protocol. If you do need to reverse engineer the USB protocol, here are some tools that might help:

Windows tools: USB Snoopy let you do actions with your device and log the stream of USB information going downstream/outbound to the device, or upstream/inbound back to your computer. Snoopy Pro is a variant of the same code that evidently has some improvements. It appears that the preferred location for Snoopy Pro is here.

Linux tools:
- The previously mentioned USBView will show you devices that are currently plugged in.
- The usbutils package includes a bunch of handy console tools for USB, including lsusb, which shows you the USB devices that are currently plugged in. The output of lsusb looks like this:

Bus 001 Device 006: ID 2222:3061 MacAlly
Bus 001 Device 002: ID 0557:7000 ATEN International Co., Ltd
Bus 001 Device 003: ID 045e:00db Microsoft Corp.

2. Write the driver.

- It seems that the process of writing drivers in Linux is getting easier over time. The Linux Journal has documented this well. Compare this 2001 article by Greg Kroah-Hartman to Greg’s 2004 article on controlling a simple USB lamp device. Then see Greg’s follow-up article on writing a linux driver in user space. It turns out that you can use the libusb library to read/write with USB devices without ever mucking around in the kernel. This is possible because Linux provides a USB filesystem (called USBFS) that automatically mounts USB devices into the Linux directory tree. Note that libusb also works on BSDs, Mac/OSX computers, and that there is a Windows libusb port.

If you really want to delve into this deeply, there’s an O’Reilly book on Linux device drivers that you can buy as well.

I recently stumbled across this post and it inspired me. I decided to try to reverse engineer the USB protocol for my Omron pedometer, which can upload your step data, but only to a Windows computer.

There are two parts to writing a Linux driver for a new USB device: reverse-engineering the USB protocol, and writing the Linux program.

Reverse-engineering the USB protocol

Typically your problem is that a device only runs under Windows. Like it or not, that means that you’re going to need something that runs Windows. It can be a Windows computer, or you can get fancy and run Windows as a “guest” operating system using something like VMWare to do virtualization. That is, you’d install Linux, then install VMWare, then install Windows to run under VMWare. But let’s start simple.

Step 0. Find the Vendor ID and Product ID of your device

Every USB device should have a Vendor ID plus a Product ID (sometimes called a device ID) that identifies it. You’ll need to discover this information before you can talk to the device. I plugged my Omron pedometer into a linux machine and typed “lsusb”. You’ll get a lot of information back. I saw a line like

Bus 002 Device 018: ID 0590:0028 Omron Corp.

That tells me that the vendorid is hexadecimal value 0×0590 (which is 1424 in decimal) and the productid is hex value 0×0028 (which is 40 in decimal). For other operating systems, this post tells you how to find your vendor id and product id under Mac and Windows. For Windows XP, it looks like you can run “msinfo32.exe” and then look under “Components” and then “USB” and look for “VID_” (vendor id) and “PID_” (product id).

1) The simple approach: a dedicated Windows computer

In the beginning, it’s easiest to just use a Windows computer and run some software to sniff on the USB packets as they go back and forth. The wild part is that the best open-source/free program I found is five years old (SnoopyPro). It still worked fine on Windows XP though. SnoopyPro is the program you want. There’s a whole long history of how it forked from USBSnoopy (evidently also called “sniff-bin“), and there’s another program called sniffusb which is related but different (I think both sniffusb and SnoopyPro are forks off of the original USBSnoopy/sniff-bin program). It’s all very confusing. I went with SnoopyPro and it worked fine for me.

Further reading on SnoopyPro and related USB sniffer programs:
Some documentation on how to use SnoopyPro
If you’re willing to shell out for a book, it looks like USB Complete, now in its third edition, is one of the best.
http://www.piclist.com/techref/usbs.htm – mentions all the different sniffers
http://hackspire.unsads.com/USB_Protocol#USB_traffic_analysis – talks about how to convert SnoopyPro (and SniffUsb) logs/traces into hexadecimal data.

Are there other options? Sure. USB Monitor from HHD Software is $85 and runs on Windows. Or you could spend $850-950 to buy a hardware USB protocol analyzer. Since I have only a casual interest, that’s a bit steep for me.

One last option is to run Windows as a virtual “guest” on a Linux system running something like VMWare. VMWare can let programs interact with USB devices. As the virtual version Windows interacts with the USB device, the Linux computer gets to see everything that happens, because it sits between Windows and the USB device. In fact, Eric Preston presented a method that could log all the the output of the Linux usbmon program as binary data. Eric changed usbmon to use relayfs so that large amounts of data could be quickly relayed from kernel space to user space, then wrote a user space program to dump that binary data to disk. Eric also wrote a dissector for ethereal so that he could view the USB data in real-time. Unfortunately the PDF of his slide presentation have disappeared from http://download.linuxmontreal.com/projects/usb/reveng/ where they used to be. In fact, all of linuxmontreal.com appears to be gone now.

By the way, Ethereal is now known as Wireshark, and it is a protocol analyzer that runs on many platforms and apparently supports USB traces. It looks like Wireshark has supported USB since version 0.99.4:

Wireshark now supports USB as a media type. If you’re running a Linux distribution with version 2.6.11 of the kernel or greater and you have the usbmon module enabled and you have a recent CVS version of libpcap (post-0.9.5) installed you can also do live captures. More details can be found at the USB capture setup page on the wiki.

Follow the link in the quote to find Wireshark’s USB wiki page.

On Ubuntu 7.10 (Gutsy Gibbon), I was able to do these commands:

sudo mount -t debugfs none_debugs /sys/kernel/debug
sudo modprobe usbmon
ls /sys/kernel/debug/usbmon
0s 0t 0u 1s 1t 1u 2s 2t 2u

General USB Reading:
USB in a NutShell is a pretty good overview of how USB communication goes.
This Java and USB tutorial starts with a good overview of USB.
This USB and Linux tutorial starts to get into the nitty gritty of USB on Linux.

You can easily make an internet-connected scale out of a Wii Balance Board and a Linux machine:

First, find a Bluetooth dongle and configure your Linux machine to talk to the Wiimote.

Next, apply a few extra patches so that your Linux machine can talk to a Wii Balance Board.

Finally, use some Python code to upload your weight to a Google Spreadsheet.

If you’d like to hear me describe how to hook it everything together, you can watch me give a 7-8 minute talk about it (more info in that post), or you can watch it here:

Special thanks to Kevin Kelly and Gary Wolf for kickstarting the Quantified Self movement and encouraging me to talk about this project.

Over the years, I’ve accumulated lots of bookmarks.html files. I’d love someone to write an App Engine program that would let you upload bookmarks.html files and would merge them all into one master file. After that, you could prune/remove useless bookmarks, especially any bookmark items that are installed by default on a new browser but are useless.

Why do it on the Google App Engine?

Because it would be an easy way to get started. Essentially you want to upload a small set of files to one web location from several different computers, and then do something interesting with that data. App Engine is perfect for that kind of thing.

Can App Engine’s version of Python parse bookmarks.html files?

The Mozilla/Firefox bookmarks.html file format is a little strange, but not too strange. I found a few programs to parse bookmarks.html files. For example, one fellow wrote a Python program to merge bookmarks using sgmllib, which I’m guessing would work on App Engine.

Digging into it more, it looks like several people like Beautiful Soup as a parser. First off, you can download it as a single Python file to work in App Engine. It also looks pretty easy to use. I like this short example of extracting favicons to .ico files from a bookmarks.html file using Beautiful Soup. At least one other person has released tools to manipulate bookmarks.html files with Beautiful Soup.

Can you upload files to Google App Engine?

Yes! There’s evidently a limit of 10MB on uploaded files, but my biggest bookmark file was about 500K, and I suspect most people have much smaller bookmark files. Stack Overflow has a good example of file uploading in Google App Engine, plus there’s official examples as well as people helping other people to the point of showing live examples.

Plus browsers are getting better about uploading files to the web easily. Google Chrome supports really easy drag-and-drop file upload. I think Safari supports drag-and-drop file upload as well? And I know Firefox has the dragdropupload extension that eases uploading files to the web.

What about uploading Google Chrome bookmarks files?

Ah, a person after my own heart. The short answer is that Google Chrome can export bookmarks in a format that looks like Firefox to me. Click on the Wrench, then “Bookmark manager,” then Tools->Export Bookmarks… to get a bookmarks.html file. The more fun answer is that “C:\Documents and Settings\{$USER}\Local Settings\Application Data\Google\Chrome\User Data\Default” appears to have a “Bookmarks” file, and it appears to be in JSON format. Can Python parse JSON? It can; Yahoo mentions that simplejson is a great library to use, and it turns out that Google App Engine supports simplejson very easily. Just say “from django.utils import simplejson” to use simplejson. So it wouldn’t be hard to upload raw Chrome bookmark files either.

Aren’t there existing websites to do this?

Maybe, but I don’t know of them. I thought that Foxmarks might be able to do this. Foxmarks (like the now-defunct Google Browser Sync) can synchronize bookmarks across multiple computers. And Foxmarks provides a my.foxmarks.com web interface that lets you manipulate and export your bookmarks, but you can’t upload a raw bookmarks.html file to Foxmarks; instead, you have to upload/sync bookmarks via a browser extension. If Foxmarks added the ability to upload bookmarks.html files (vote for that idea here), that would be pretty sweet.

TimeTrax was a program that allowed XMPCR owners to listen to XM Radio on their computer. Even nicer, the program would “time shift” the recording by taking the XM meta data and recording the raw audio of a channel to an MP3 with the artist/title of the song.

TimeTrax is not really viable anymore (see the Wikipedia page), but another program called SXRecorder will do much of the same thing.

If you’re using an XMPCR 100, you’ll need to install USB drivers for your device:

http://www.ftdichip.com/Drivers/VCP.htm

From the page: “Virtual COM port (VCP) drivers cause the USB device to appear as an additional COM port available to the PC.” So if you have the FTDI chip in a device (FTDI = Future Technology Devices International) like the XMPCR, it makes that USB device look like a serial port device, so that you can talk to COM ports.

Searching on Google for SXRecorder finds www.backpocket.com/sxrecorder/ . The software is free, but you can donate $25 or buy extra plugins for SXRecorder for $35.

You’ll need to activate your XMPCR receiver, but you can refresh your radio receiver at xmradio.com.

When I was in grad school, I would have loved to have a device like the Kinect. So I decided to run my own contest:

The first $1000 prize goes to the person or team that writes the coolest open-source app, demo, or program using the Kinect. The second prize goes to the person or team that does the most to make it easy to write programs that use the Kinect on Linux.

It’s time to announce the prize winners. There’s been so many cool things going on with the Kinect that instead of two winners, I ended up declaring seven $1000 winners.

Open-source Application or Demo

I picked two winners in this category.

• Tomoto Washio for Kinect Ultra Seven. This program lets you transform into Ultra Seven with lots of different powers. It’s a really fun demo–check out the videos:
  
• Tiago Serra and the SenseBloom team for XBox Kinect OSCeleton. The application sends 3D tracked body skeletons through the OSC protocol. Essentially, it’s a do-it-yourself motion capture system:
  

People that have made it easier to write programs for the Kinect

A ton of people have made the Kinect more accessible on Linux or helped the Kinect community. I ended up picking five winners.

• Hector Martin
• Arne Bernin
• Kyle Machulis
• Brandyn White
• Joshua Blake

All of these individuals pushed things forward so others can develop great programs on the Kinect more easily. Congratulations to all the winners, and to everyone doing neat things with their Kinect!

Before I joined Google, I was a grad student interested in topics like computer vision, motion self-tracking, laser scanners–basically any neat or unusual sensing device. That’s why I was so excited to hear about the Kinect, which is a low-cost ($150) peripheral for the Xbox. The output from a Kinect includes:
- a 640×480 color video stream.
- a 320×240 depth stream. Depth is recovered by projecting invisible infrared (IR) dots into a room. You should watch this cool video to see how the Kinect projects IR dots across a room. Here’s a single frame from the video:

What’s even better is that people have figured out how to access data from the Kinect without requiring an Xbox to go with it. In fact, open drivers for the Kinect have now been released. The always-cool Adafruit Industries, which offers all sorts of excellent do-it-yourself electronics kits, sponsored a contest to produce open-source drivers for the Kinect:

First person / group to get RGB out with distance values being used wins, you’re smart – you know what would be useful for the community out there. All the code needs to be open source and/or public domain.

Sure enough, within a few days, the contest was won by Héctor Martín Cantero, who is actually rolling his reward into tools and devices for fellow white-hat hackers and reverse engineers that he works with, which is a great gesture. Okay, so where are we now? If I were still in grad school, I’d be incredibly excited–there’s now a $150 off-the-shelf device that provides depth + stereo and a lot more.

It’s time for a new contest

I want to kickstart neat projects, so I’m starting my own contest with $2000 in prizes. There are two $1000 prizes. The first $1000 prize goes to the person or team that writes the coolest open-source app, demo, or program using the Kinect. The second prize goes to the person or team that does the most to make it easy to write programs that use the Kinect on Linux.

Enter the contests by leaving a comment on this blog post with a link to your project, along with a very-short description of what your project does or your contribution to Kinect hacking. The contest runs until the end of the year: that’s Dec. 31st, 2010 at midnight Pacific time. I may ask for outside input on who should be the winner, but I’ll make the final call on who wins.

To get your ideas flowing, I’ll offer a few suggestions. Let’s start with the second contest: making the Kinect more accessible. In my ideal world, would-be hackers would type a single command-line, e.g. “sudo apt-get install openkinect” and after that command finishes, several tools for the Kinect would be installed. Maybe a “Kinect snapshot” program that dumps a picture, a depth map, and the accelerometer values to a few files. Probably some sort of openkinect library plus header files so that people can write their own Kinect programs. I would *love* some bindings to a high-level language like Python so that would-be hobbyists can write 3-4 lines of python (“import openkinect”) and start trying ideas with minimal fuss. To win the second contest, you could write any of these libraries, utilities, bindings or simplify installing them on recent versions of Linux/Ubuntu (let’s say 10.04 or greater).

Okay, how about some ideas for cool things to do with a Kinect? I’ll throw out a few to get you thinking.

Idea 1: A Minority Report-style user interface where you can open, move, and close windows with your movements.

Idea 2: What if you move the Kinect around or mount it to something that moves? The Kinect has an accelerometer plus depth sensing plus video. That might be enough to reconstruct the position and pose of the Kinect as you move it around. As a side benefit, you might end up reconstructing a 3D model of your surroundings as a byproduct. The folks at UNC-Chapel Hill where I went to grad school built a wide-area self-tracker that relied on a Kalman filter to estimate a person’s position and pose. See this PDF paper for example.

Idea 3: Augmented reality. Given a video stream plus depth, look for discontinuities in depth to get a sort of 2.5 dimensional representation of a scene with layers. Then add new features into the video stream, e.g. a bouncing ball that goes between you and the couch, or behind the couch. The pictures at the end of this PDF paper should get you thinking.

Idea 4: Space carver. Like the previous idea, but instead of learning the 2.5D layers of a scene from a singe depth map, use the depth map over time. For example, think about a person walking behind a couch. When you can see the whole person, you can estimate how big they are. When they walk behind the couch, they’re still just as big, so you can guess that the couch is occluding that person and therefore the couch is in front of the person. Over time, you could build up much more accurate discontinuities and layers for a scene by watching who walks behind or in front of what.

Idea 5: A 3D Hough transform. A vanilla Hough transform takes a 2D image, looks for edges in the image, and then runs some computation to determine lines in the image. A 3D Hough transform finds planes in range data. I’ve done this with laser rangefinder data and it works. So you could take a depth data from a Kinect and reconstruct planes for the ground or walls in a room.

Idea 6: What if you had two or more Kinects? You’d have depth or range data from the viewpoint of each Kinect and you could combine or intersect that data. If you put two Kinects at right angles (or three or four Kinects around a room, all pointing into the room), could you reconstruct a true 3D scene or 3D object from intersecting the range data from each Kinect?

I hope a few of these ideas get you thinking about all the fun things you could do with a Kinect. I’m looking forward to seeing what cool ideas, applications, and projects people come up with!

On the childlike-sense-of-wonder-scale (as fake Steve Jobs would say), the iPad is better than the Macbook Air but not as stunning as the iPhone when the iPhone first came out. I played with my wife’s iPhone for just a few minutes before I knew I had to have an iPhone. But I never really cared about the Macbook Air, mainly because the screen resolution was worse than my current laptop. The iPad fits between those two products in the spectrum of desirability for me.

The form factor is… weird. You’re going to feel strange carrying one of these into the grocery store, in the same way you felt weird using your cell phone in the grocery store at first. Leave it to Apple to blaze a trail of coolness though; the iPad will make this form factor acceptable, so you won’t feel quite as strange carrying a tablet into a meeting in a few months. The form factor fundamentally is awkward though: the iPad is book-sized, but much more delicate than a book. A screen this big with no protection will get scratched or scuffed. I’d expect to see plenty of articles about dropped iPads like you did about Wiimotes getting thrown into TVs and windows.

The gadget lover in me wants one, but the part of me that cares about open source and tinkering is stronger. I’m with Cory Doctorow on this one. The iPad is gorgeous, but it’s still not worth it for me. Yesterday, I also bought two books at the bookstore to read on a trip. Walking back to my car with “paper media” felt a bit dorky–why am I buying books on paper in 2010? If I could buy a book digitally and really own it (not just obtain a license to read a book, where the license could be revoked), I’d quickly switch to buying my books digitally. But the success of the Kindle shows that a lot of people care more about the convenience than completely owning what they’re buying digitally.

I think the iPad will be a huge hit. Non-tech-savvy consumers will love it because of the user experience, the simplicity, and the lack of viruses/malware/trojans. It’s like a computer without all the hassles of a typical computer (pre-installed crapware, anti-virus software, inconvenient software upgrades). Lots of tech-savvy consumers will love the iPad for the same reasons, and especially for the polish and user experience. The current iPad lacks a few things (such as a camera), which ensures that future generations of the iPad will also be a huge hit.

But the iPad isn’t for me. I want the ability to run arbitrary programs without paying extra money or getting permission from the computer manufacturer. Almost the only thing you give up when buying an iPad is a degree of openness, and tons of people could care less about that if they get a better user experience in return. I think that the iPad is a magical device built for consumers, but less for makers or tinkerers. I think the world needs more makers, which is why I don’t intend to buy an iPad. That said, I think the typical consumer will love the iPad.

At any point, I could go to a web page to view a map of where I’d been. The page would show a “heat map” of signal strength for each carrier or frequency band. Maybe I could also slice/dice by time or see the total number of readings in each location. I’m pretty sure you could rig this up out of 2-3 cell phones running Android in the worst case.

So far, I’ve found:

Android

- RF Signal Tracker is a nice app to collect and map signal strength data. It looks like it can upload to OpenCellID, which is a project to create an open database of cell IDs (numbers that correspond to cells).
- Antennas is a pretty cool free app to show you nearby antennas and signal strength. It can even export some data in KML for use with Google Maps/Earth, but it doesn’t seem to make a heat map that could be easily grokked.
- Sensorly has a free Android app, but they seem to want you to pay to zoom in closer than city level. I’m willing to do that, but didn’t see the for-pay addon in the Android Market.

iPhone

- I also found an iPhone app called Signals that will continuously collect signal data and upload it.
- AT&T offers an iPhone app called Mark the Spot to report dropped calls, no coverage, etc. I have to admit that I don’t understand why this is manual though. Personally, I’d want my phone to ping my carrier with its location every time the phone dropped a call.

Web

- SignalMap is a website to (manually!) submit the number of bars for a location. It doesn’t appear to have any mobile app to back it up. Likewise, Dead Cell Zones and Got Reception? appear to rely on manual reports. I don’t think manual reports is the best way to tackle cell phone coverage maps though — you really want an app for this.
- http://www.cellreception.com/ has the standard manual reports data, but also will map the location of cell phone towers based on the location of cell phone towers registered with the FCC.
- Root Wireless powers the cell phone signal strength maps that CNET uses, but I didn’t see any apps I could download or install on a phone. I registered to be a beta tester a long time ago, but no one ever contacted me.

That’s what I could find. Do you know of any good Android (or iPhone) programs to collect, map, or upload cell phone strength measurements? If so, let me know in the comments.