Announcing the winners of the Kinect contest

When the Kinect launched, Adafruit Industries ran a contest for the first person who released open-source code to extract video and depth from the Kinect. Adafruit also ended up donating to the EFF after the contest was over.

When I was in grad school, I would have loved to have a device like the Kinect. So I decided to run my own contest:

The first $1000 prize goes to the person or team that writes the coolest open-source app, demo, or program using the Kinect. The second prize goes to the person or team that does the most to make it easy to write programs that use the Kinect on Linux.

It’s time to announce the prize winners. There’s been so many cool things going on with the Kinect that instead of two winners, I ended up declaring seven $1000 winners.

Open-source Application or Demo

I picked two winners in this category.

People that have made it easier to write programs for the Kinect

A ton of people have made the Kinect more accessible on Linux or helped the Kinect community. I ended up picking five winners.

All of these individuals pushed things forward so others can develop great programs on the Kinect more easily. Congratulations to all the winners, and to everyone doing neat things with their Kinect!

My predictions for 2009

Everybody is throwing out predictions for 2011, so I thought I’d do something different. Here are some of my predictions for 2009.

Wha wha wha?? That’s right: 2009. As 2008 drew to a close, I jotted down some predictions, but I never hit the “publish” button. Some of them weren’t fully-formed (or even complete sentences!), so I won’t print all of them. But some are interesting. Here are a few:

– “Market share for IE falls below 50% in some countries in Europe.” TRUE according to StatCounter. Here’s a graph:

IE Market share in Europe

– “Cuil will be acquired by Baidu.” Definitely a MISS. This was an educated guess on my part. I didn’t think Cuil would do well, and someone named Greg Penner was on the board of Cuil and also a board member of Baidu.

– “More people will realize the inevitable truth that Bill Gates saw years ago and that Apple has chased since the introduction of the ROKR: of all the devices in your pocket, the only one you’re not willing to give up is your phone. Therefore, all personal gadgets will eventually be subsumed by your phone. Camera? Already part of your phone. Pen and notebook? Quite close. Video camera? Almost there, give it a couple more years. Car keys, wallet? It will come. In five years, your phone will have fingerprint authentication and be able to start your car or pay for groceries with contactless/RFID chips. It’s all coming. In 10 years you’ll use your phone to authenticate yourself at the doctor, authenticate prescriptions, and store your personal health history, not to mention all your desktop preferences, bookmarks, browser add-ons, and keys to which music you have permission to stream or download from the cloud.” I call this TRUE. Most people now agree that your phone is a personal computer in your pocket. Back in 2008, not everyone realized this.

– “By the way, if anyone figures out how to lick the problem of satisfactory output/input, e.g. heads-up displays or retinal lasers and a virtual keyboard or something with as high a bandwidth as normal typing (and they will), your computer will migrate into your phone. Solving the input/output problem is one of the most important problems for the next decade. Witness the efforts that companies have put into pen-based computing and voice recognition, for example. Re-examine the success of some major products in the new light of better input/output mechanisms:
– Google Maps: direct manipulation
– iPhone: touch and multitouch
– Wii: accelerometer and optical sensors
Smart computer scientists and engineers should be paying as much attention to sonar, inertial, and optical sensing technology as they do to the change from hard-drive platters to solid-state storage.” Um. This one was more of a rant then a prediction.

– Here’s what I wrote back in 2008: “Canny self-promoters plus a few genuine believers will jump on the Facebook backlash early, either for privacy/personal information or for keeping Facebook’s data to itself. But unless the site makes a Beacon-level mistake, Facebook will experience massive growth in 2009, and the Facebook backlash won’t begin in earnest until 2010 or 2011.” Right now it feels true. Maybe we’ll check back after 2011 to see whether that’s accurate.

– “Semantic web technology won’t take off, at least not in the generally-accepted ways.” I’m going to call that TRUE. Remember how Web 3.0 was going to be the semantic web? You don’t hear that meme as much anymore.

– “The most interesting “savviness” test for me will be which of Yahoo, Facebook, or Microsoft let you freely retrieve your email with them into Gmail. Odds are that none will allow it, but if I had to pick one, it would be Yahoo.” A double MISS, because Hotmail did allow POP3 access in 2009. Meanwhile, I believe Yahoo still wants $20/year for POP access. I don’t think you can fetch Facebook mail using POP or IMAP.

– “Vanity iPhone apps.” This was a prediction that individuals would commission their own personal iPhone apps that collected their blog posts, tweets, photos, etc. The other part of the idea was that conferences would commission conference-specific apps. While a few savvy conferences have done this, I wouldn’t call either trend widespread. So it’s a MISS.

– “In the same way that millions of people dropped their land-line for a cell phone connection, thousands of people will attempt to go digital by scanning their photos, ripping their CDS, digitizing their old VHS tapes, and scanning their papers.” MISS, I did a lot of those things in 2009, but average people didn’t. The brunt of the analog to digital migration will happen as data is “born digital” from the beginning. Millions of people will have physical photos stuck in boxes that they don’t look at much, plus digital photos on Facebook, Flickr, or Picasa.

– “Google Chrome becomes one of Google’s most successful products.” I think this is TRUE. (Remember, this prediction was less than four months after Chrome was introduced in September, 2008. Back then I was going out more on a limb.)

– “Breakthroughs in green technology push renewable energy closer to the mainstream, but not into the mainstream. As many people focus their attention in this area, unexpected surprises abound, such as the do-it-yourself solar roof installation or practical solar shingles. Expect to see more green technology scams as well though.” Meh. MISS.

– “Apple will weather the 2009 recession much better than most people expect as people continue to buy with their heart, not always with their head. More people will realize that ‘Once you go Mac, you never go back.’ ” I’m calling it TRUE. Apple radically outperformed the Dow in 2009. Apple has done very well competing against MSFT in operating systems.

– “Obama’s administration mandates that all federal buildings much use Compact Fluorescent Lightbulb (CFL) technology before his presidential term ends.” I’ve still got at least two more years on this prediction. If the mandate is for LED lightbulbs, I’m still counting it.

– I thought some public officials would start lifestreaming as the ultimate endorsement of transparency. That’s a MISS. Chalk it up to believing that politics would be different after the 2008 election. Oh well.

– “Microsoft will demo a snazzy new phone operating system, but it won’t get much traction, either because a) it’s not as snazzy as the iPhone or b) people don’t trust it to be as open as other mobile operating systems.” Given that this was a prediction for 2009, I’ll call it TRUE. Microsoft didn’t get a lot of phone traction in 2009.

– “Someone not officially associated with Android will work on an “iPhone skin” for Android to make Android look like an iPhone.” TRUE. Looks like that did happen in 2009.

– “Twitter’s 2009 business model consists of premium features that either individuals can pay for (at a nominal rate such as $1/month or $10/year) or that businesses can set up for self-contained twittering. The premium features include richer API access as well as some new features, such as the ability to save tweets to be posted in the future or periodically.” That, my friends, is a MISS. Although Twitter is now charging for richer access to Twitter data, so maybe not a horrible miss.

– “Hacking of PC clients will decrease, but hackers will shift their sites to web server software. Massive databases of cross-site scripting attacks will be traded in the underground. Script kiddies will launch dictionary attacks. Anyone that writes their own web server software will probably be at a high risk of being hacked.” Overall I’m going to call this TRUE. As individual PCs become more secure, there’s a multi-year trend toward hacking webservers instead. I fear we’re just at the beginning of this.

– “When Android opens premium apps, one of the big controversies will be developers who take great premium-app ideas for the iPhone and rewrite the ideas behind the app for Android, but without the permission of the iPhone developer. Expect flashlights, lighters, beer, farting applications, goldfish, etc.” I’m not sure if this is TRUE or a MISS. I haven’t read a lot of complaints from iPhone app developers about Android app developers stealing their idea and writing their own apps.

– “Multiple people write a bit of JavaScript that site owners can add to their page to guilt IE6 users to upgrade to a new browser. Any browser, just something that was released after 2001 (IE6 was released in 2001?!?). The JavaScript code becomes a viral sensation and sweeps across the net. Whoever authors or hosts this JavaScript gains status around the web. Through the collective work of a bunch of savvy webmasters, SEOs, and site owners, IE6’s market share drops to 5% in some markets by the end of the year.” IE6 is slowly riding into the sunset, but call it a MISS. A few businesses will probably be running IE6 a decade from now for their internal apps.

– “eBay introduces spell correction for search queries which delivers a small boost for their profits.” I’m going to call this TRUE because the search [ipod touc] returns “Did you mean: ipod touch?” at the top of the eBay’s results now.

Okay, it’s your turn. Which of your predictions for 2009 or 2010 were spot on or off-base?

Open Kinect Contest: $2000 in prizes

I’m starting a contest for people that do cool things with a Kinect. See the details below.

Open Kinect Logo

Before I joined Google, I was a grad student interested in topics like computer vision, motion self-tracking, laser scanners–basically any neat or unusual sensing device. That’s why I was so excited to hear about the Kinect, which is a low-cost ($150) peripheral for the Xbox. The output from a Kinect includes:
– a 640×480 color video stream.
– a 320×240 depth stream. Depth is recovered by projecting invisible infrared (IR) dots into a room. You should watch this cool video to see how the Kinect projects IR dots across a room. Here’s a single frame from the video:

IR Projection

but you should really watch the whole video to get a feel for what the Kinect is doing.
– the Kinect has a 3-axis accelerometer.
– the Kinect also has a controllable motor to tilt up and down plus four microphones.

What’s even better is that people have figured out how to access data from the Kinect without requiring an Xbox to go with it. In fact, open drivers for the Kinect have now been released. The always-cool Adafruit Industries, which offers all sorts of excellent do-it-yourself electronics kits, sponsored a contest to produce open-source drivers for the Kinect:

First person / group to get RGB out with distance values being used wins, you’re smart – you know what would be useful for the community out there. All the code needs to be open source and/or public domain.

Sure enough, within a few days, the contest was won by Héctor Martín Cantero, who is actually rolling his reward into tools and devices for fellow white-hat hackers and reverse engineers that he works with, which is a great gesture. Okay, so where are we now? If I were still in grad school, I’d be incredibly excited–there’s now a $150 off-the-shelf device that provides depth + stereo and a lot more.

It’s time for a new contest

I want to kickstart neat projects, so I’m starting my own contest with $2000 in prizes. There are two $1000 prizes. The first $1000 prize goes to the person or team that writes the coolest open-source app, demo, or program using the Kinect. The second prize goes to the person or team that does the most to make it easy to write programs that use the Kinect on Linux.

Enter the contests by leaving a comment on this blog post with a link to your project, along with a very-short description of what your project does or your contribution to Kinect hacking. The contest runs until the end of the year: that’s Dec. 31st, 2010 at midnight Pacific time. I may ask for outside input on who should be the winner, but I’ll make the final call on who wins.

To get your ideas flowing, I’ll offer a few suggestions. Let’s start with the second contest: making the Kinect more accessible. In my ideal world, would-be hackers would type a single command-line, e.g. “sudo apt-get install openkinect” and after that command finishes, several tools for the Kinect would be installed. Maybe a “Kinect snapshot” program that dumps a picture, a depth map, and the accelerometer values to a few files. Probably some sort of openkinect library plus header files so that people can write their own Kinect programs. I would *love* some bindings to a high-level language like Python so that would-be hobbyists can write 3-4 lines of python (“import openkinect”) and start trying ideas with minimal fuss. To win the second contest, you could write any of these libraries, utilities, bindings or simplify installing them on recent versions of Linux/Ubuntu (let’s say 10.04 or greater).

Okay, how about some ideas for cool things to do with a Kinect? I’ll throw out a few to get you thinking.

Idea 1: A Minority Report-style user interface where you can open, move, and close windows with your movements.

Idea 2: What if you move the Kinect around or mount it to something that moves? The Kinect has an accelerometer plus depth sensing plus video. That might be enough to reconstruct the position and pose of the Kinect as you move it around. As a side benefit, you might end up reconstructing a 3D model of your surroundings as a byproduct. The folks at UNC-Chapel Hill where I went to grad school built a wide-area self-tracker that relied on a Kalman filter to estimate a person’s position and pose. See this PDF paper for example.

Idea 3: Augmented reality. Given a video stream plus depth, look for discontinuities in depth to get a sort of 2.5 dimensional representation of a scene with layers. Then add new features into the video stream, e.g. a bouncing ball that goes between you and the couch, or behind the couch. The pictures at the end of this PDF paper should get you thinking.

Idea 4: Space carver. Like the previous idea, but instead of learning the 2.5D layers of a scene from a singe depth map, use the depth map over time. For example, think about a person walking behind a couch. When you can see the whole person, you can estimate how big they are. When they walk behind the couch, they’re still just as big, so you can guess that the couch is occluding that person and therefore the couch is in front of the person. Over time, you could build up much more accurate discontinuities and layers for a scene by watching who walks behind or in front of what.

Idea 5: A 3D Hough transform. A vanilla Hough transform takes a 2D image, looks for edges in the image, and then runs some computation to determine lines in the image. A 3D Hough transform finds planes in range data. I’ve done this with laser rangefinder data and it works. So you could take a depth data from a Kinect and reconstruct planes for the ground or walls in a room.

Idea 6: What if you had two or more Kinects? You’d have depth or range data from the viewpoint of each Kinect and you could combine or intersect that data. If you put two Kinects at right angles (or three or four Kinects around a room, all pointing into the room), could you reconstruct a true 3D scene or 3D object from intersecting the range data from each Kinect?

I hope a few of these ideas get you thinking about all the fun things you could do with a Kinect. I’m looking forward to seeing what cool ideas, applications, and projects people come up with!

Report on 30 day challenge: A picture a day

In August 2010, my 30 day challenge was to take at least a picture a day. It was a good challenge, because that month I traveled with friends to Tanzania and climbed Mt. Kilimanjaro. I finally got around to making a picture-a-day public photo gallery with the pictures I selected. Here’s one of my favorite photos of the rain forest near the base of Kilimanjaro, for example:

Rain forest at the base of Kilimanjaro

It took a while to pick the photos, upload them, and add captions, so I hope you enjoy the photos!

My 2010 Halloween Costume: Ninja!!

I got the inspiration for this year’s Halloween costume from a parade in Modesto, California. I’ve dressed up as some silly stuff in previous years, so I wanted something cool this year. And there’s not much cooler than a ninja! Here I am showing off my ninja costume:

Me in my ninja costume

(Click to see a full-size image.)

I tried to be authentic. Check out the details of my costume and shuriken:

Me in my ninja costume

(Click to see a full-size image.)

The best part about this costume is that when kids ring our doorbell, I’ll go into stealth mode. They’ll never even notice as I silently drop to the ground behind them, pad up on split-toe tabi boots to drop candy in their bags, then fade into the shadows. I’m looking forward to trick-or-treaters this year. And I can even use the costume after Halloween to slip up behind spammers.

Thanks to the ninja parade for inspiring my ninja costume this year!