Reverse engineering a Windows USB driver

For a while, I was really into reverse-engineering USB drivers. Don’t ask why. The heart wants what the heart wants. I didn’t finish this “hairball” post, but it has some info in it that still might be good.

I recently stumbled across this post and it inspired me. I decided to try to reverse engineer the USB protocol for my Omron pedometer, which can upload your step data, but only to a Windows computer.

There are two parts to writing a Linux driver for a new USB device: reverse-engineering the USB protocol, and writing the Linux program.

Reverse-engineering the USB protocol

Typically your problem is that a device only runs under Windows. Like it or not, that means that you’re going to need something that runs Windows. It can be a Windows computer, or you can get fancy and run Windows as a “guest” operating system using something like VMWare to do virtualization. That is, you’d install Linux, then install VMWare, then install Windows to run under VMWare. But let’s start simple.

Step 0. Find the Vendor ID and Product ID of your device

Every USB device should have a Vendor ID plus a Product ID (sometimes called a device ID) that identifies it. You’ll need to discover this information before you can talk to the device. I plugged my Omron pedometer into a linux machine and typed “lsusb”. You’ll get a lot of information back. I saw a line like

Bus 002 Device 018: ID 0590:0028 Omron Corp.

That tells me that the vendorid is hexadecimal value 0x0590 (which is 1424 in decimal) and the productid is hex value 0x0028 (which is 40 in decimal). For other operating systems, this post tells you how to find your vendor id and product id under Mac and Windows. For Windows XP, it looks like you can run “msinfo32.exe” and then look under “Components” and then “USB” and look for “VID_” (vendor id) and “PID_” (product id).

1) The simple approach: a dedicated Windows computer

In the beginning, it’s easiest to just use a Windows computer and run some software to sniff on the USB packets as they go back and forth. The wild part is that the best open-source/free program I found is five years old (SnoopyPro). It still worked fine on Windows XP though. SnoopyPro is the program you want. There’s a whole long history of how it forked from USBSnoopy (evidently also called “sniff-bin“), and there’s another program called sniffusb which is related but different (I think both sniffusb and SnoopyPro are forks off of the original USBSnoopy/sniff-bin program). It’s all very confusing. I went with SnoopyPro and it worked fine for me.

Further reading on SnoopyPro and related USB sniffer programs:
Some documentation on how to use SnoopyPro
If you’re willing to shell out for a book, it looks like USB Complete, now in its third edition, is one of the best. – mentions all the different sniffers – talks about how to convert SnoopyPro (and SniffUsb) logs/traces into hexadecimal data.

Are there other options? Sure. USB Monitor from HHD Software is $85 and runs on Windows. Or you could spend $850-950 to buy a hardware USB protocol analyzer. Since I have only a casual interest, that’s a bit steep for me.

One last option is to run Windows as a virtual “guest” on a Linux system running something like VMWare. VMWare can let programs interact with USB devices. As the virtual version Windows interacts with the USB device, the Linux computer gets to see everything that happens, because it sits between Windows and the USB device. In fact, Eric Preston presented a method that could log all the the output of the Linux usbmon program as binary data. Eric changed usbmon to use relayfs so that large amounts of data could be quickly relayed from kernel space to user space, then wrote a user space program to dump that binary data to disk. Eric also wrote a dissector for ethereal so that he could view the USB data in real-time. Unfortunately the PDF of his slide presentation have disappeared from where they used to be. In fact, all of appears to be gone now. :(

By the way, Ethereal is now known as Wireshark, and it is a protocol analyzer that runs on many platforms and apparently supports USB traces. It looks like Wireshark has supported USB since version 0.99.4:

Wireshark now supports USB as a media type. If you’re running a Linux distribution with version 2.6.11 of the kernel or greater and you have the usbmon module enabled and you have a recent CVS version of libpcap (post-0.9.5) installed you can also do live captures. More details can be found at the USB capture setup page on the wiki.

Follow the link in the quote to find Wireshark’s USB wiki page.

On Ubuntu 7.10 (Gutsy Gibbon), I was able to do these commands:

sudo mount -t debugfs none_debugs /sys/kernel/debug
sudo modprobe usbmon
ls /sys/kernel/debug/usbmon
0s 0t 0u 1s 1t 1u 2s 2t 2u

General USB Reading:
USB in a NutShell is a pretty good overview of how USB communication goes.
This Java and USB tutorial starts with a good overview of USB.
This USB and Linux tutorial starts to get into the nitty gritty of USB on Linux.

Halloween Pumpkin: Portal Turret!

I made a Portal turret for my Halloween pumpkin! I was trying to think of things to carve: vampire Android? R2-D2? Zoidberg? Then I thought: I could do a character from Portal 2! I was going to carve something like GLaDOS or Wheatley, but then I realized that a portal turret would be perfect:

To make the glowing red eye, I punched a small hole in the pumpkin, taped an LED to a battery, and pushed the LED from the center of the pumpkin through the small hole. I thought about doing an actual red laser, but 1) I don’t want to blind any kids walking by my pumpkin, and 2) laser beams aren’t visible unless there’s fog or smoke or something for the beam to hit.

To make the side “gun” sections of the pumpkin, I had a small dowel rod lying around, so I just cut two small stakes from the dowel rod and jammed them into the pumpkin sides. It’s a bit rickety, but it works. For the video, I played some turret sound effects and moved my red laser over the pumpkin as if another turret was locking onto it.

I didn’t want to take long to carve my pumpkin. I finished it in maybe an hour, but check out this really elaborate pumpkin that someone else did. And it turns out that they work at Google too!

I like how my pumpkin came out overall:

Portal 2 turret pumpkin for Halloween!

Happy Halloween, and I hope you’re getting creative with your pumpkin carving!

How to strip JPEG metadata in Ubuntu

If you want to post some JPEG pictures but you’re worried that they might have metadata like location embedded in them, here’s how to strip that data out.

First, install exiftool using this command:

sudo apt-get install libimage-exiftool-perl

Then, go into the directory with the JPEG files. If you want to remove metadata from every file in the directory, use

exiftool -all= *.jpg

The exiftool will make copies, so if you had a file called image.jpg, when you’re done you’ll have image.jpg with all the metadata stripped plus a file called image.jpg_original which will still have the metadata.

How to find start-up ideas

Chris Dixon had an interesting post a while ago about how to find start-up ideas. The advice boiled down to keeping a spreadsheet of ideas and talking to lots of smart people (entrepreneurs, potential customers, VCs, people at big companies). It’s good advice. Paul Graham also wrote in 2008 about startup ideas he’d like to fund.

Here’s another way to come up with startup ideas: walk around your house or apartment, and look for “hot spots.” A hotspot can be an area of high information density, clutter, stress, disorganization, or any place that has a suboptimal solution. Then think about a web or cloud solution to that hot spot. Let’s take a look at a few examples:

Music CDs -> iTunes, Amazon MP3 store, doubleTwist, MP3tunes, etc.
Bookshelf -> Amazon, Kindle, iBooks
Stereo system -> Sonos, Squeezebox, Rhapsody, Pandora,, Spotify, Grooveshark, MOG, Rdio, etc.
External hard drives -> Amazon Simple Storage Service (S3), Pogoplug

Okay, those all seem simple or obvious, right? Let’s go a little deeper. What would you do with this pile of business cards?

pile of business cards

Pile of business cards -> CloudContacts

Here are a few more that come to mind:
Bank statements -> Mint
Photo Albums -> ScanCafe
Bathroom scale -> Withings
Pedometer -> Fitbit
Phone -> Google Voice, Twilio, Ribbit, Rebtel
Camera -> EyeFi
Stack of video games -> Steam, OnLive
DVD player -> Roku, Netflix Instant movies
Treadmill or Elliptical machine -> Nike+ shoe sensor, LoseIt! iPhone app, CardioTrainer app for Android, Fitbit
Pen -> Livescribe

All of these take a hotspot in your home and inject a cloud or web element to make life easier, more efficient or better. So what happens when you look at a pile of manuals, or receipts? Your alarm clock? Those “Learning Japanese” CDs? A stack of take-out menus? A stack of cookbooks? A hard drive full of MP3s that are disorganized? A hard drive that doesn’t have a back-up copy? An out-of-date programming book? A box full of videotapes? All those back issues of magazines? A blank wall, with no posters or other decoration? Stuff in your garage that you’ve been meaning to sell or give away? Your wallet?

Ideas are sitting all around where you live. If you have a small snag, irritation, or hotspot in your life, probably a lot of other people do too. You can make it easier to organize something (can you convert something physical to digital and store it in the cloud?). You can sell niche versions of a product (e.g. Threadless for T-shirts), you can let people make something that they couldn’t make before (CafePress for T-shirts, LuLu for books), you can pool people with similar interests (a blog like Craftzine, or a forum for book lovers or body builders), you can review products in a particular space, you can teach someone to do something. You can become a well-known expert in something and then sell your time or expertise as a consultant. You can make a free version of something useful or fun, then sell more features or consult on more involved cases. You can do meta versions of lots of these, e.g. Etsy is a marketplace for people who like to buy and sell custom crafted objects.

I’ll stop with a story. I have a friend at Google who is really good at noticing things that annoy him. While walking from his car to his desk in the morning, he can easily find six things that irritate him because they should be improved. I’m not recommending that you make yourself more irritable, but I am saying that if you notice all the times you run across something that can be improved, those are opportunities. And I think one of the easy methods of spotting start-up ideas is looking around where you live and how you spend your time. Find the hotspots in your own life and you might identify some great products or services to build.

How to delete “nobody” files from a directory I own in FreeBSD?

Lazyweb, here’s a quick question for you. On a FreeBSD system, suppose I own a directory called “foo”. Inside that “foo” directory are some files (file1, file2, file3) owned by the “nobody” user. How do I delete those files? When I type the command “rm file1″ I get the response “rm: file1: Permission denied” even though I own the parent foo directory.

Any suggestions? It’s on a machine where I don’t have root access.

Update: Problem solved, but in an annoying way. The files were created by a PHP script, so I wrote a new PHP script to remove the files in question. It looked something like



This forum thread from 2001 describes the idea (thanks to Mark and the people that commented).

In case you’re wondering, pair Networks’ FreeBSD systems really don’t work well with the WordPress “WP Cache” or “WP Super Cache” plug-ins, because both plugins make files owned by the “nobody” user which appear to be impossible to delete from the SSH command-line.