Compile a simple USB program in Linux

Here’s another “hairball” post about USB devices and drivers on Linux. I wish some expert would write the definitive “here’s how to reverse-engineer a USB device and write a new USB driver” guide. I am definitely not that expert.

Once you reverse engineer a Windows USB device enough to know how it works, you’re ready to try talking to the device under Linux. Here are a few tips.

lsusb to list devices from a command-line.

On Ubuntu 7.10, usbview (use “sudo apt-get install usbview” to install it) is broken. It says “Can not open the file /proc/bus/usb/devices Verify that you have USB compiled into your kernel, have the USB core modules loaded, and have the usbdevfs filesystem mounted.” It turns out that Ubuntu decided to drop support for /proc/bus/usb devices. You can still find USB devices under /dev/bus/usb/ on Ubuntu, but they are special binary files.

usbmon is a Linux kernel module that lets you monitor USB events: . The text file format is pretty terse (see the PDF for some details), but if you’ve installed Linux kernel source code, check Documentation/usb/usbmon.txt for more info. Note that USBMon is a tool with a graphical user interface (GUI) that is based on usbmon. I believe that usbmon operates by using instrumentation hooks in the usbcore Linux kernel module. usbmon normally outputs only text, although Eric Preston reported adding a binary interface to usbmon.

You can trace how talking to a USB device under Linux has gotten easier over the years by reading the sequence of articles that Greg Kroah-Hartman has written for Linux Journal over the years. Let’s examine a few of the articles:

October 2001: How to Write a Linux USB Device Driver. Greg walks through nitty gritty details for using a program usb-skeleton.c that he wrote. Aspiring writers of drivers have to learn quite a bit of kernel hacking.

April 2004: Writing a Simple USB Driver. Greg writes a simple driver for a USB lamp. The driver still sits in kernel space, but exposes some functionality to the user through sysfs. What the heck is sysfs? According to the Wikipedia page, “Sysfs is a virtual file system provided by the 2.6 Linux kernel and it replaced the old deprecated devfs which use in 2.4 series of stable kernels. Sysfs exports information about devices and drivers from the kernel device model to userspace, and is also used for configuration.” Pretty handy. This article is also notable because Greg mentions a couple ways to reverse-engineer an unknown USB protocol: 1) sniff the USB events with Windows software such as SnoopyPro, or 2) run Windows on top of Linux using virtualization software such as VMWare. When you plug in a USB device, VMWare can make the device available to the Windows instance. Windows happily talks with the USB device, unaware that because Linux is sitting between Windows and the USB device, Linux gets to see (and can dump) all the communication that is happening between Windows and the USB device.

June 2004: Writing a Real Driver — In User Space. Greg has now left the kernel behind for the ease of talking to a USB device from user space. This article covers three things that make like easier.

1. usbfs. Originally called usbdevfs, usbfs is a way for user space programs to read/write data to USB devices. The method of talking to the USB devices are ioctl (“input/output control”) function calls to special files, where each file represents a USB device. If you’ve use ioctl before, you’ll know that this is still a pretty down-to-the-metal way to talk to hardware.

2. libusb. This is where life really starts to get good. libusb is a library radically simplifies talking to USB devices. Even better, it runs on many types of UNIX besides Linux (including Mac OS X). You can read the official libusb documentation, but it lacks a good “talk to a simple device” example. I also enjoyed this overview of libusb.

3. Example source code. Is there anything better when you’re delving into a new library than example source code? Greg gives a short program that echoes his previous article. In just a few lines of code, he shows how to turn on an LED in a USB lamp.

There are a couple wrinkles with the sample code. First, there are at least a few errors in the code on the Linux Journal website. A C programmer can look at code like

usb_handle = usb_open(usb_dev);
if (usb_handle == NULL) {
goto exit;
usb_handle = usb_open(usb_dev);
if (usb_handle == NULL) {
fprintf(stderr, "Not able to claim the USB devicen");
goto exit;

and say “Hmm. That first fprintf doesn’t have a closing parenthesis. For that matter, why call usb_open() twice and overwrite usb_handle the second time?” So something got scrambled in the code listing. I recommend checking out this simple usbradio program as another short example of how to use libusb.

Second, Greg doesn’t really say how to compile the program. On Ubuntu 7.10 (Gutsy Gibbon), I had to install a few packages. I think I had to do “sudo apt-get install gcc build-essential libusb-dev” to get everything I needed. Then if your program is called talk-to-usb.c, you would compile the program with “gcc -o talk-to-usb talk-to-usb.c -lusb”. (The “-lusb” tells the compiler to use the libusb library when linking in code.)

August 2004: Snooping the USB Data Stream. This is a really fascinating article. Greg is given an unknown USB device and told to make it work on Linux. It turns out to be a crypto key and comes with working Linux code — but the license prohibits Greg from using the USB device as freely as he would like. This puts Greg back in the position of reverse-engineering a USB protocol. In the same way that Windows-running-atop-Linux-in-VMWare provides a way to monitor USB communication that takes place in usbfs, adding usbfs logging to Linux helps to see what commands are being sent from the too-restrictive Linux library for this USB device.

If you understood that last sentence, congratulations for paying such close attention. 🙂 To continue, Greg walks the reader through a patch to the kernel to enable kernel-based logging of any USB communication through usbfs. He even adds a variable (controllable through sysfs) so that users can turn this USB logging on and off at will.

At this point (looking on the web, not in the kernel source), I noticed several reports of additional USB logging in Linux. There was Greg’s article. There’s usbmon, which is “a facility in kernel which is used to collect traces of I/O on the USB bus.” I noticed a patch by Paolo Abeni to add binary dumps to usbmon. Both usbmon and Paolo’s patch utilize debugfs, which is another Greg Kroah-Hartman invention to allow reading/writing of data to user space, especially debugging/binary data. As Greg puts it, “one line of code and you have a variable that can be read and written to from userspace.” And Eric Preston evidently added binary dumping of USB data to Linux as well, and then started a USB protocol dissector to view USB communication in ethereal/Wireshark, which normally is used to sniff on network communication.

Now we’ve gotten a lot of background, so we’re ready to look at an example actually monitoring USB communication in Linux. The best write-up I’ve seen is this one on USB snooping on Linux. It’s a little over a year old, but check out one of the comments:

The cool thing I have spotted is that the debugfs and usbmon module is already present under ubuntu. You will not need to compile or configure the kernel, just mount the debugfs, load the usbmon module, and the usbmon directory should appear under the /sys/kernel/debug directory.

That means that a stock Ubuntu 7.10 (Gutsy Gibbon) install can just run these commands:

sudo mount -t debugfs none_debugs /sys/kernel/debug
sudo modprobe usbmon
ls -l /sys/kernel/debug/usbmon/
total 0
-rw——- 1 root root 0 2008-02-17 00:59 0s
-rw——- 1 root root 0 2008-02-17 00:59 0t
-rw——- 1 root root 0 2008-02-17 00:59 0u
-rw——- 1 root root 0 2008-02-17 00:59 1s
-rw——- 1 root root 0 2008-02-17 00:59 1t
-rw——- 1 root root 0 2008-02-17 00:59 1u
-rw——- 1 root root 0 2008-02-17 00:59 2s
-rw——- 1 root root 0 2008-02-17 00:59 2t
-rw——- 1 root root 0 2008-02-17 00:59 2u

After that, you’re ready to cat those files, exercise the USB device, and observe the resulting USB traces to figure out what commands and data passed back and forth. In case you’re wondering, the filenames seem to be a number (0,1,2) followed by a letter (s,t,u). According to the usbmon documentation, the number corresponds to the bus numbers of the USB devices. The documentation also mentions that the ‘u’ files are a superset of the ‘t’ files with more detailed information. I don’t know what the ‘s’ files are.

Just as an aside, it looks like /proc/bus/usb/ is deprecated in Ubuntu 7.10 and is being replaced with /dev/bus/usb. Unfortunately, that breaks a few programs (usbview, virtualbox, and qemu). Instead of usbview, you can use lsusb to find the particulars of a specific USB device. Unfortunately, it sounds like /proc/bus/usb/devices is going away entirely, which is a shame because the file was easy-to-read for regular users. The bug report I linked to gives a workaround if you really want that file.

How to strip JPEG metadata in Ubuntu

If you want to post some JPEG pictures but you’re worried that they might have metadata like location embedded in them, here’s how to strip that data out.

First, install exiftool using this command:

sudo apt-get install libimage-exiftool-perl

Then, go into the directory with the JPEG files. If you want to remove metadata from every file in the directory, use

exiftool -all= *.jpg

The exiftool will make copies, so if you had a file called image.jpg, when you’re done you’ll have image.jpg with all the metadata stripped plus a file called image.jpg_original which will still have the metadata.

Announcing the winners of the Kinect contest

When the Kinect launched, Adafruit Industries ran a contest for the first person who released open-source code to extract video and depth from the Kinect. Adafruit also ended up donating to the EFF after the contest was over.

When I was in grad school, I would have loved to have a device like the Kinect. So I decided to run my own contest:

The first $1000 prize goes to the person or team that writes the coolest open-source app, demo, or program using the Kinect. The second prize goes to the person or team that does the most to make it easy to write programs that use the Kinect on Linux.

It’s time to announce the prize winners. There’s been so many cool things going on with the Kinect that instead of two winners, I ended up declaring seven $1000 winners.

Open-source Application or Demo

I picked two winners in this category.

People that have made it easier to write programs for the Kinect

A ton of people have made the Kinect more accessible on Linux or helped the Kinect community. I ended up picking five winners.

All of these individuals pushed things forward so others can develop great programs on the Kinect more easily. Congratulations to all the winners, and to everyone doing neat things with their Kinect!

Open Kinect Contest: $2000 in prizes

I’m starting a contest for people that do cool things with a Kinect. See the details below.

Open Kinect Logo

Before I joined Google, I was a grad student interested in topics like computer vision, motion self-tracking, laser scanners–basically any neat or unusual sensing device. That’s why I was so excited to hear about the Kinect, which is a low-cost ($150) peripheral for the Xbox. The output from a Kinect includes:
– a 640×480 color video stream.
– a 320×240 depth stream. Depth is recovered by projecting invisible infrared (IR) dots into a room. You should watch this cool video to see how the Kinect projects IR dots across a room. Here’s a single frame from the video:

IR Projection

but you should really watch the whole video to get a feel for what the Kinect is doing.
– the Kinect has a 3-axis accelerometer.
– the Kinect also has a controllable motor to tilt up and down plus four microphones.

What’s even better is that people have figured out how to access data from the Kinect without requiring an Xbox to go with it. In fact, open drivers for the Kinect have now been released. The always-cool Adafruit Industries, which offers all sorts of excellent do-it-yourself electronics kits, sponsored a contest to produce open-source drivers for the Kinect:

First person / group to get RGB out with distance values being used wins, you’re smart – you know what would be useful for the community out there. All the code needs to be open source and/or public domain.

Sure enough, within a few days, the contest was won by Héctor Martín Cantero, who is actually rolling his reward into tools and devices for fellow white-hat hackers and reverse engineers that he works with, which is a great gesture. Okay, so where are we now? If I were still in grad school, I’d be incredibly excited–there’s now a $150 off-the-shelf device that provides depth + stereo and a lot more.

It’s time for a new contest

I want to kickstart neat projects, so I’m starting my own contest with $2000 in prizes. There are two $1000 prizes. The first $1000 prize goes to the person or team that writes the coolest open-source app, demo, or program using the Kinect. The second prize goes to the person or team that does the most to make it easy to write programs that use the Kinect on Linux.

Enter the contests by leaving a comment on this blog post with a link to your project, along with a very-short description of what your project does or your contribution to Kinect hacking. The contest runs until the end of the year: that’s Dec. 31st, 2010 at midnight Pacific time. I may ask for outside input on who should be the winner, but I’ll make the final call on who wins.

To get your ideas flowing, I’ll offer a few suggestions. Let’s start with the second contest: making the Kinect more accessible. In my ideal world, would-be hackers would type a single command-line, e.g. “sudo apt-get install openkinect” and after that command finishes, several tools for the Kinect would be installed. Maybe a “Kinect snapshot” program that dumps a picture, a depth map, and the accelerometer values to a few files. Probably some sort of openkinect library plus header files so that people can write their own Kinect programs. I would *love* some bindings to a high-level language like Python so that would-be hobbyists can write 3-4 lines of python (“import openkinect”) and start trying ideas with minimal fuss. To win the second contest, you could write any of these libraries, utilities, bindings or simplify installing them on recent versions of Linux/Ubuntu (let’s say 10.04 or greater).

Okay, how about some ideas for cool things to do with a Kinect? I’ll throw out a few to get you thinking.

Idea 1: A Minority Report-style user interface where you can open, move, and close windows with your movements.

Idea 2: What if you move the Kinect around or mount it to something that moves? The Kinect has an accelerometer plus depth sensing plus video. That might be enough to reconstruct the position and pose of the Kinect as you move it around. As a side benefit, you might end up reconstructing a 3D model of your surroundings as a byproduct. The folks at UNC-Chapel Hill where I went to grad school built a wide-area self-tracker that relied on a Kalman filter to estimate a person’s position and pose. See this PDF paper for example.

Idea 3: Augmented reality. Given a video stream plus depth, look for discontinuities in depth to get a sort of 2.5 dimensional representation of a scene with layers. Then add new features into the video stream, e.g. a bouncing ball that goes between you and the couch, or behind the couch. The pictures at the end of this PDF paper should get you thinking.

Idea 4: Space carver. Like the previous idea, but instead of learning the 2.5D layers of a scene from a singe depth map, use the depth map over time. For example, think about a person walking behind a couch. When you can see the whole person, you can estimate how big they are. When they walk behind the couch, they’re still just as big, so you can guess that the couch is occluding that person and therefore the couch is in front of the person. Over time, you could build up much more accurate discontinuities and layers for a scene by watching who walks behind or in front of what.

Idea 5: A 3D Hough transform. A vanilla Hough transform takes a 2D image, looks for edges in the image, and then runs some computation to determine lines in the image. A 3D Hough transform finds planes in range data. I’ve done this with laser rangefinder data and it works. So you could take a depth data from a Kinect and reconstruct planes for the ground or walls in a room.

Idea 6: What if you had two or more Kinects? You’d have depth or range data from the viewpoint of each Kinect and you could combine or intersect that data. If you put two Kinects at right angles (or three or four Kinects around a room, all pointing into the room), could you reconstruct a true 3D scene or 3D object from intersecting the range data from each Kinect?

I hope a few of these ideas get you thinking about all the fun things you could do with a Kinect. I’m looking forward to seeing what cool ideas, applications, and projects people come up with!

Switching between dev and beta Chrome channels on Linux

If you’re on Linux (say Ubuntu 10.04, also known as Lucid Lynx), you can switch between the developer (dev) and beta channels of Chrome like this:

Switch from Beta to Dev:
sudo apt-get install google-chrome-unstable

Switch from Dev to Beta:
sudo apt-get install google-chrome-beta

That’s easier for me than going back for the .deb file and doing something with it.

Also, if you want to check whether a new dev version of Chrome is out, you can just repeat the same command:
sudo apt-get install google-chrome-unstable

and if there’s no new version, you’ll get something like this back:

Reading package lists… Done
Building dependency tree
Reading state information… Done
google-chrome-unstable is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

I figured I’d document this in case I needed to remember how to do it in the future. 🙂