Open Kinect Contest: $2000 in prizes

I’m starting a contest for people that do cool things with a Kinect. See the details below.

Open Kinect Logo

Before I joined Google, I was a grad student interested in topics like computer vision, motion self-tracking, laser scanners–basically any neat or unusual sensing device. That’s why I was so excited to hear about the Kinect, which is a low-cost ($150) peripheral for the Xbox. The output from a Kinect includes:
– a 640×480 color video stream.
– a 320×240 depth stream. Depth is recovered by projecting invisible infrared (IR) dots into a room. You should watch this cool video to see how the Kinect projects IR dots across a room. Here’s a single frame from the video:

IR Projection

but you should really watch the whole video to get a feel for what the Kinect is doing.
– the Kinect has a 3-axis accelerometer.
– the Kinect also has a controllable motor to tilt up and down plus four microphones.

What’s even better is that people have figured out how to access data from the Kinect without requiring an Xbox to go with it. In fact, open drivers for the Kinect have now been released. The always-cool Adafruit Industries, which offers all sorts of excellent do-it-yourself electronics kits, sponsored a contest to produce open-source drivers for the Kinect:

First person / group to get RGB out with distance values being used wins, you’re smart – you know what would be useful for the community out there. All the code needs to be open source and/or public domain.

Sure enough, within a few days, the contest was won by Héctor Martín Cantero, who is actually rolling his reward into tools and devices for fellow white-hat hackers and reverse engineers that he works with, which is a great gesture. Okay, so where are we now? If I were still in grad school, I’d be incredibly excited–there’s now a $150 off-the-shelf device that provides depth + stereo and a lot more.

It’s time for a new contest

I want to kickstart neat projects, so I’m starting my own contest with $2000 in prizes. There are two $1000 prizes. The first $1000 prize goes to the person or team that writes the coolest open-source app, demo, or program using the Kinect. The second prize goes to the person or team that does the most to make it easy to write programs that use the Kinect on Linux.

Enter the contests by leaving a comment on this blog post with a link to your project, along with a very-short description of what your project does or your contribution to Kinect hacking. The contest runs until the end of the year: that’s Dec. 31st, 2010 at midnight Pacific time. I may ask for outside input on who should be the winner, but I’ll make the final call on who wins.

To get your ideas flowing, I’ll offer a few suggestions. Let’s start with the second contest: making the Kinect more accessible. In my ideal world, would-be hackers would type a single command-line, e.g. “sudo apt-get install openkinect” and after that command finishes, several tools for the Kinect would be installed. Maybe a “Kinect snapshot” program that dumps a picture, a depth map, and the accelerometer values to a few files. Probably some sort of openkinect library plus header files so that people can write their own Kinect programs. I would *love* some bindings to a high-level language like Python so that would-be hobbyists can write 3-4 lines of python (“import openkinect”) and start trying ideas with minimal fuss. To win the second contest, you could write any of these libraries, utilities, bindings or simplify installing them on recent versions of Linux/Ubuntu (let’s say 10.04 or greater).

Okay, how about some ideas for cool things to do with a Kinect? I’ll throw out a few to get you thinking.

Idea 1: A Minority Report-style user interface where you can open, move, and close windows with your movements.

Idea 2: What if you move the Kinect around or mount it to something that moves? The Kinect has an accelerometer plus depth sensing plus video. That might be enough to reconstruct the position and pose of the Kinect as you move it around. As a side benefit, you might end up reconstructing a 3D model of your surroundings as a byproduct. The folks at UNC-Chapel Hill where I went to grad school built a wide-area self-tracker that relied on a Kalman filter to estimate a person’s position and pose. See this PDF paper for example.

Idea 3: Augmented reality. Given a video stream plus depth, look for discontinuities in depth to get a sort of 2.5 dimensional representation of a scene with layers. Then add new features into the video stream, e.g. a bouncing ball that goes between you and the couch, or behind the couch. The pictures at the end of this PDF paper should get you thinking.

Idea 4: Space carver. Like the previous idea, but instead of learning the 2.5D layers of a scene from a singe depth map, use the depth map over time. For example, think about a person walking behind a couch. When you can see the whole person, you can estimate how big they are. When they walk behind the couch, they’re still just as big, so you can guess that the couch is occluding that person and therefore the couch is in front of the person. Over time, you could build up much more accurate discontinuities and layers for a scene by watching who walks behind or in front of what.

Idea 5: A 3D Hough transform. A vanilla Hough transform takes a 2D image, looks for edges in the image, and then runs some computation to determine lines in the image. A 3D Hough transform finds planes in range data. I’ve done this with laser rangefinder data and it works. So you could take a depth data from a Kinect and reconstruct planes for the ground or walls in a room.

Idea 6: What if you had two or more Kinects? You’d have depth or range data from the viewpoint of each Kinect and you could combine or intersect that data. If you put two Kinects at right angles (or three or four Kinects around a room, all pointing into the room), could you reconstruct a true 3D scene or 3D object from intersecting the range data from each Kinect?

I hope a few of these ideas get you thinking about all the fun things you could do with a Kinect. I’m looking forward to seeing what cool ideas, applications, and projects people come up with!

88 Responses to Open Kinect Contest: $2000 in prizes (Leave a comment)

  1. Matt, this is your finest idea yet. I can’t wait to see what transpires and I hope that Microsoft/Google doesn’t get in the way. Have you mentioned this contest to either company?

  2. I agree this is awesome idea… i got the wiimote hacks running pretty fast so it would be cool to see what type of neato displays and controls that could be hacked together with this technology

  3. Matt,

    Is anyone doing a contest of the best use of Node.js in an Android Webkit Application yet?

    It would be the prefect contest for T Bray to start..

    My Bias: I might be developing one..

  4. could this be used in place of a laser and ps3 cam in a touch table ?

  5. Idea: couple Kinect with a projector. This could make augmented reality much more real! You could texture real-life objects.

  6. How do you connect kinect to the PC? It uses USB + special power line.

  7. Somehow I keep thinking nerf gun sentry turret.

  8. Am I the only one thinking about pairing a kinect together with the wiimote?

  9. Hi guys, you can find the openkinect group here:

    Well have over 350 members in the last 3 days and should have over 400 by tonight. We welcome everyone to come join the initiative.

    In terms of the ‘new’ prize, I personally would have waited. Prizes generally hinder performance, not increase it. They’re good when progress is stuck, but right now things are incredibly active. My hope is the offer of a new prize doesn’t end up setting things back.

  10. Thanks for offering the reward, as you say this can lead to very exciting things.

    However, limiting the second prize to Linux seems unnecessary – why not open it up to windows and OS X as well?

    Fact is there are many more windows than linux end users, and it’d be most impactful if whatever solution is developed could be used by them as well.

  11. It sounds like the current driver doesn’t yet support sound localization using the microphone array. That’d also be a cool project if someone is interested.

  12. “A low-cost ($150) peripheral for the Xbox”?!

    A low-cost sensing device, maybe (and only if you’re the hacker/tinkerer type), but this peripheral is 75% of the cost of the console! Calling the most expensive first-party peripheral for Xbox “low-cost” just doesn’t sound right.

  13. @Parand: for one thing, the Linux development community tends to produce easily-portable software more than Windows or OS X projects; the POSIX (or damn close with Linux) base is a lot more portable than Win32/64 API or Obj-C NSfoo. Don’t feel left out; in a bit of time you’re likely to be able to run these cool hacks written originally for linux natively on OS X or Windows with a little bit of cross-compiling magic.

  14. Is this contest legal to do?

    You may want to check the Terms of Use. Let us know.

  15. @mike
    There’s an AC/USB adapter that comes with the Kinect in case you have an older Xbox that doesn’t have the Powered USB port that the new Black Xbox’s have.

  16. Thumb Wars – The video-enabled, GPS integrated, accelerometer-driven, gyroscope-fueled, mobile depth-stream utilizing social challenge.

    How does it work? Using your next generation mobile device, you challenge your Facebook friends to 3-d thumb wrestling matches from afar, and if you win enough matches you become the thumb-wrestling champion of your locale.

    (suggestion made in jest)

  17. Cool project @josh its not that expensive less than a high quality power supply which is sitting in my big PC realy must buy a replacement mb and the gubbins to go in it

    probably ought to have a bonus for proper full documentation and no a list of methods is not documentation

    wonder if you could rig up a system and have the kinect track and control an ar parrot drone.

    re idea 6 they had something that did this on an episode of sanctuary last season i think.

  18. porneL, I love the idea of using a projector. Interactive tables or active walls would be lots of fun to do.

    mike, I think you hook the USB up to the USB port of your PC, and a splitter lets you plug in power separately. I haven’t actually bought a Kinect yet, but I will when I get back from the PubCon search conference.

    Ryan Jones, a nerf gun sentry would be awesome. Here’s one for $25: that you could (maybe) start with. And here’s a similar project:

    Seth, I definitely don’t want to impede things. My hope is that it will generate some fun, but not enough to cause people to be cutthroat. That’s the idea of the second prize–to look for someone that does a lot to help the community.

    Parand, mainly I wanted to make sure that Linux didn’t get left behind in the development.

    JoshL, maybe a better way to look at it is that if you tried to develop a similar platform yourself, it would cost a lot more either in terms of money or time.

  19. @gue5t realy in some cases others not so much

    ime getting into some of the LSA stuff and am toying with the idea of trying to get an internal project going to run some stuff am possibly considering FORTRAN (non of your namby pamby hippie languages for me) to get into some Algos

    Would probably need a fortran with extentions for CUDA – will be interesting getting that past IS and Finance though “you want how many water cooled dual mother boards and how may high end graphic cards” 🙂

  20. This is a great idea. There have been some interesting projects using a wiimote on linux, such as using compiz and headtracking.

    I heard the open drivers enable the depth sensor at 640×640 too.

  21. 640×480 ^^

  22. Hi Matt,

    I know your intention isn’t to impede and it’s quite an honor to be writing on your blog I must say hehe.

    The ‘first’ prize wasn’t a necessity, but it helped kick-start something and therefore it was a huge benefit. Now that we have a thriving community (and getting bigger by the hour), we’re currently not hurting for another prize because the community is joining together for the love and ‘fun’ aspects of just working together (not for an outside incentive). Cash prizes are great when progress has hit a standstill, but since that’s not the case – it could potentially have the opposite affect of bringing the community closer to a stable device – especially when the prize is geared towards one platform (linux) and not the community as a whole. Hopefully that won’t be the case here, and I understand your not trying to hinder progress.


  23. I just wanted to add that currently, Linux is probably on the thriving end of things right now (in terms of the driver development). We’re actually most behind on the windows aspects, although things are quickly developing on all platforms right now concurrently. We have a huge amount of people from the linux community in the google group and I don’t suspect linux will be left behind in any stage of the development. 🙂

  24. With regards to the possibilities of idea 6 – use of multiple viewpoints to construct more complete 3d data. Additional devices should not be required, most of the suggested uses should be possible to achieve at a little lower resolution using a mirror.

  25. Great contest, cool idea. Wish I had time to play with this…

  26. This guy has multitouch done already:

  27. Perhaps more useful than a single dump of the accelerometer and depth values would be a continuous time series from the device.

    Given that some of these filtering algorithms need derivatives and integrals of the signals, any data structure would need to store the current values plus N sets of historical values and have it continuously refreshed (at an interval specified by the developer).

    However, I am very much looking forward to the day I can open up IPython and type “import kinect” 🙂

  28. They’ve done it with the Wiimote, now Kinect, any chances for PS Move?

  29. Started to build Adobe Air Apps for the Kinect. Planing a product to allow me to control powerpivot and presentationpivot using the Kinect as the interface.

  30. not sure if two kinects would work too well, would be interested to see. Expecting fail due to interference of dots.

  31. I’d like to see a two KINECT system. The first KINECT would do the typical full body scanning in order to accurately locate the position of your hands. The position would be supplied to the second KINECT which would be “zoomed in” on your hands to perform very fine position tracking. This would allow for “fine grain” hand commands. A possible application is a virtual keyboard. The second KINETC would use simple geometry to position a larger than normal virtual keyboard as if it were floating in air in front of you. This requires only simple geometry. As your hand enters the plane of the virtual keyboard, a letter would be “selected” and highlighted on-screen, re-positioning of the hand while still within the plane would adjust selection of the key (similar to what smart phone keyboards do). Removing your hand from the plane of the keyboard would select a letter. This would be more pecking than typing, but should still be effective. A more sophisticated implementation would determine the position of your fingers as your hand enters the plane in order to “double check” the intended key. For example if your left hand enters the plane with the left pinky “extended” an “A” would be “verified”. If your hand enters the plane slightly elevated from the initial or “home” position, a “Q” would be verified, etc. So come on, let’s see it, so we can find out if there’s of any real world benefit!

  32. A random thought: it should be quite easy to automatically generate layered GIMP (or whatever else) images from a Kinect snapshot… could be implemented as a Gimp plugin. Would be quite useful, I think, even if the Kinect resolution is not huge…

  33. You should ask them if they could make compiz based kinect software
    that would be awesome

  34. The Xbox already has some cool programs that use the Kinect to show a digitized version of the user dancing but what the world )or my world) really needs is a program that could teach uncoordinated lanky guys like myself how to dance better. If is had a algorithm that could tell a waltz from a foxtrot, it would be the perfect solution for teaching users how to become a smooth dancer.

  35. Here’s the billion dollar idea: Having the ability to map yourself, then being able to shop online and try on clothes in a virtual store. Online clothing sales would skyrocket.

  36. Simply brilliant matt.

  37. Matt – why not raise additional money through a service like in order to get the prize bigger? I’d me more than happy to chip in if I could with money.

  38. I want a 3D scanner for $150!!! Make it a 3D scanner!!!

  39. Im extreamly excited for the possibilities from two Kinects. Being a 3D animator, I see an amazing opportunity for home-brew motion capture here. if you had the two Kinects from a front and a side view (90 degrees from each other) the resulting depth information would create a perfect 3D representation of the person. If we could then track a skeleton to the data, like the Kinect already does for its games, then we could export that data as an animation file for 3D character rigs. This would be amazing for home-brew marker-less motion capture.

  40. Wow, this is really cool!!! I had no idea how this all worked, so cool you can download drivers also… can’t wait to real more about it! Thanks for the post!

  41. Love the idea, it is great indeed! Cant wait to see how it all works out!

  42. TOTALLY agree with Graham Wiebe! In fact I have asked this officially on EPIC’s UDK forums (since they seem to have a good working relationship with Microsoft) UDK represents a huge step forward to home-brew developers and Kinect and the idea of a 150 (or 300) dollar mo-cap solution is a tremendous proposition. Having a plugin for max,maya,xsi, lightwave (or whatever, perhaps in mo-cap .bms format which is useable by many apps/tools) would be a dream come true.

  43. Matt,
    Great idea!
    Yes, this is the first time to get a device like this at such a low cost, I totally agree.

    I work in the medical field, and for anything to be of medical grade we need a much higher resolution data, but we usually get contacted by other industries who have lower accuracy requirements and want lower cost so I think Kinnect fills this gap.
    Check out one of our projects to see what you can do with depth data.

    The possibilities are endless!

  44. – Suggest better arrangement of furniture and other items in the room
    – Scan a person during his/her sleep and provide report in the morning about their sleeping pattern; send it to the doctor.
    – When someone has been watching tv for a long time without moving, sense that and invite that person to play a kinect game.
    – Let it know about your contacts (fb, whatever), and allow playing games, or have a video call with them.
    – Scan the books lying around and show reviews, user comments..
    – Scan the rental DVDs lying around and remind about due date for return
    – Turn on automatically based on User’s usual play time
    – Draw dot-patterns in the wall, for decoration.
    – Alarm if the front door is left open
    – Sound discomfort if unknown person is present in the room
    – Advice user to wash when window curtains become too dirty
    – If someone is shouting too much, play nice music to calm them down
    – Record the dots in film and project it back to the device to recreate a scene?
    – Take the device with you on a Cruise and let it show you how you are being thrown around due to waves. If it crosses a limit, then it could suggest some precautions.
    – Notify you when newspaper is thrown near your door outside the house (it may have to point out through a window at night).
    – Report about your Pets’ unusual movements; scan their input/output trays and report.
    – Warn you if you have grown fatter compared to a few months ago (assuming you did not want to get fat).
    – Congratulate if you have become slimmer compared to a few weeks ago (assuming you wanted to get slimmer).
    – Scan and recognize the food you are eating and show calories.

  45. Great idea, Matt.
    Go Tar Heels.

  46. idea: a 3D scanner for the reprap, or the cupcake.

  47. Is it possible with the kinect to copy and transmit all images surrounding what it is projecting on, then rather than a human teleport to a place, an actual place teleports to the person?

  48. I like where your going with this idea haha. Someone should really try some of these kinect ideas and put them on youtube…

  49. This is a great idea..
    I love this contest idea
    woop woop…. kinect
    @Seth I agreee with you.. linux does rocks and was or will never be behind when it comes to development.. just got to love it 🙂

  50. 1—run multiple Kinect devices in parallel to improve speed, reliability.

    2—Chinese characters are written in specific stroke order, so they would
    work well in Asia as a gesture language


  51. Hi Matt,
    my name is Salvatore and I have an italian blog (link in my name) about drawing and painting. My home page has more than 4000 backlink (Webmaster tool info) but I have PR 2. I think I have a penalty and I did the following things to remove the penalty:

    1. I use affiliate link to amazon and tradedoubler and some artists and I put nofollow to all of them
    2. Using copyscape I removed all duplicate content or used blockquote for citation.
    3. I removed all book some book review with affiliate links and poor content.

    I tried to do anything I can know on penalty removal and submitted the reconsideration request, but nothing changed. I am working on these problem from several month and I am becoming crazy. Hoping you or someone of your team can help me. Sorry for this OT on the post. I will do everithing is necessary to remove the penalty and I have interest to no avoid in future action that can penalize my site.

    Please feel free to contact me for any suggestion or simply reply to this comment. I know you are very busy person and I hope yu can find few seconds for me.
    Thanks in advace for everithing you can do for me.

  52. Hi Matt,
    my name is Salvatore and I have an italian blog (link in my name) about drawing and painting. My home page has more than 4000 backlink (Webmaster tool info) but I have PR 2. I think I have a penalty and I did the following things to remove it:

    1. I use affiliate links to amazon/tradedoubler/some artists and I put nofollow to all of them
    2. Using copyscape I removed all duplicate content and used blockquote for citation.
    3. I removed all books review with affiliate links and poor content.

    I tried to do anything I know on penalty removal and submitted the reconsideration request, but nothing changed. I am working on these problems from several month and I am becoming crazy. Hoping you or someone of your team can help me. Sorry for this OT on the post. I will do everithing is necessary to remove the penalty and I have interest to avoid, in future, actions that can penalize my site.

    Please feel free to contact me for any suggestion or simply reply to this comment. I know you are very busy person and I hope yu can find few seconds for me.
    Thanks in advace for everithing you can do for me.

  53. Someone needs to write a driver for the open source Guitar Hero “Frets on Fire” – it would be a perfect match.

  54. This is AWESOME, the bounty is getting great PR, hopefully a trend has been started by you!

  55. If anyone has time to programm it, in my opinion a 3D painting programm would be great.

  56. I would love to see anything virtual. 3D scan of your own head would be cool. Or a much bigger size scanner to scan your entire home in 3D (and turn it into a sketchup 3D model)…

  57. Hi Matt,

    I think you’re looking something similar to Pranav Mistry’s SixthSense technology.

  58. @porneL

    ohh if you mounted the projector and kinect verticaly you could project a game board and track the movement of models ala the MS surface tables DnD – pity the perry twins (senior GW types) arn’t more into hacking hardware.

    Wonder if you could use ir paint that responds to diferent frequency’s so you could identify the models

  59. Call me paranoid, but robot equipped with Kinects for sight etc.. tied to Google Goggles, plus a few other datacentres would be pretty terrifyingly ‘smart’, when things get smaller I am just waiting for the genius criminal to program it to go into homes and steal valuable stuff…

  60. Go Matt, go! Kinect groups are raising so fast becouse of you! Thanks!

  61. You should ask them if they could make compiz based kinect software!!

  62. Wow, that was really cool. Love it! Now I might get a Kinect after all 🙂

  63. Kinect is perfect device for relatively cheap and easy to use motion capture and virtual control of a 3D world with out use of a body suit like in The Lawnmower Man. But as game testing has verified its not quite there yet in terms of system response, but may be that’s the problem of the game developer’s and not the Kinect. Open source development kit for Kinect create a 3D World, interact within an existing 3D World such World of Warcrafts or control a Android- Powered robot in the real world. OK may be all of that is a bit to big for now something a little smaller recently an Android-Powered Lego Consumer Robot Solves Rubik’s Cube in 12.5 Seconds. So how about this a Virtual 3D Rubik’s Cube Game. This will demonstrate to things first the creation of a 3D world and second control over the 3D world by interacting with the Virtual 3D Rubiks cube to solve it.

    Virtual 3D Rubiks cube
    0 the 3D Rubiks cube
    1 A random jumble button to mix up all colors on all sides
    2 start button
    3 quit button
    4 timer

    is there any body out there that thinks this is do able before Microsoft gets out there lawyers and start suing any and everyone involve with developing an Open Kinect Developers Kit.

  64. i have one more “3d body avatar”

  65. Hi,

    I was wondering if my robotics project would qualify after open-sourcing?


  66. We really need open drivers for the Emotiv next.

    Imagine the combination of multiple Kinects for 3D interaction, and an Emotiv for the non-physical side.

    No Linux drivers, and all these companies like to lock everything up in proprietary drivers which don’t work that well anyway.

    MMORPGs of the near future will include Rangers swinging their foam swords, Monks played by martial arts students using real punches and kicks, and Psionicists moving things with their mind.

  67. Revive and implement the Videoplace installation of Myron W. Krueger using Kinect.

    The ‘father’ of Artificial Reality, or immersive environments, is Myron Krueger More than any other of the early pioneers in this space, Dr. Krueger’s vision drove the initial development. For several decades from the 70’s through the 80’s, he implemented an AR system called VideoPlace which embodied the concepts now commercialized by Kinect. I worked in the lab where grad students were hacking away at VideoPlace projects… they were fully functioning prototypes involving all kinds of games. The difference was that the projector, cameras, etc were part of the room, not packaged as a product.

    In his book Aritificial Reality Dr. Krueger describes many applications e.g. a ‘karate keyboard’, including a system for therapists which lets children project themselves as equally sized to talk to adults. There was a cool maze which would shift shape behind someone walking through it, an environment where you could define the laws of physics as well. The thing was, people funding Engineering saw his work as more about art, while those funding the Arts saw it more as technology.

    And Krueger’s not getting any credit, from what I can see.

    So my project would be to track down Dr. Krueger and re-implement some of his original VideoPlace projects, like the interative maze or Karate Keyboard, on the Kinect.

  68. PS here’s more links on MWK
    “you are a child in the world the technology has created”

  69. Arthritis movement therapy for the elderly

    Create a game which would make it fun for elderly stroke patients to do gentle exercises
    and regain movement. This currently happens in group exercise / yoga sessions, but
    the activities are fairly simple, it can be boring, and each participant only gets partial attention from the therapist/leader.

    The game would involve fairly simple movements but the projection of these movements would cause amusement and motivate further work. For example, the elder user would see a more youthful/strong version of themselves projected into a world where there is a sense of exploration (e.g. raise arms to fly though the virtual environment) or goal (e.g. move blocks out of the way). Perhaps include other participants in some kind of race or competition. Allow them to temporarily forget that they are, say, confined to a wheelchair or have limited range of movement, and run free once again, as in their youth.

  70. Matt,
    I tested out the Kinect last night and I have to tell you I am pretty impressed. I wonder if we will be able to use it to do some 3D mapping. The one thing I thought was lacking is that it is not compatible with social media. I played the dance game last night and it takes pictures and video of you dancing while your playing. I think XBOX missed the mark here because I wanted to upload my videos and pictures to facebook because they are hilarious. Right now there is no easy way to do this. It takes a long time to get them off of the xbox. They need social media integration. Talk about something going viral.

  71. You should give the $1K linux prize to this guy

  72. Offer a prize for the first Kinect game to be implemented on an Android device!
    Hey… how about angry birds?

    I don’t know if the droids are currently fast enough but — it’s just a matter of time.

  73. Eric Kinect Now :)

    Hi Emo Philips, You mean like the

    Can’t wait for my Kinect.

  74. Here are a few of my ideas:

    If I can integrate Kinect with street view. So I take steps and kinect is able to detect it and is able to move me on the street view, I turn around and kinect turns me around in street view. I stare at a building and kinect is able to detect it and provide me more info/pictures and other relevant info about that building on street view.

    Now imagine this with 3D glasses, I literally do a tour of any city. So say I am planning vacations, I can actually do a virtual tour of the city, go check out the interiors of hotels/restaurants and make my reservations accordingly. Actually this does not really need kinect, but will be super awesome with kinect 🙂

    Now if I can make a game out of it, wherein I can have NFS using kinect but integrated with street view. So basically I am driving on real roads. I can have a Grand theft auto on in real city.

    Also picture social networking with this. I can use my avatar and be present at any location in city. I can strike conversation with people, can take them to virtual restaurants,discs, adventure parks. Picture ability to play games with kinect in these virtual amusement parks. We have already seen games like Dance Central and Kinect adventures. Imagine I was able to make friend in my neighborhood, and then I can take that friend to an amusement
    park and play kinect adventures with that friend. Or can go to a disc and play Dance Central.

  75. Great Idea ! by the way how i ‘ll join this contest?

  76. That YouTube video someone above me posted is crazy. Imagine what others can do in this contest. I’m looking forward to the results.

  77. Speech recognition is still about 90% ….. a laser lip reader

    will boost that …. a Kinect type device is mounted on the phone.

    Speak into laser beam and reflections add extra input for

    recognizing speech. Believe now in Japanese labs.


  78. Hey cool info matt.. i’m thinking of developing and app for this on my final year graduation project in few months. sad to notice noone hav posted any links yet… may b i can get smething ready by next month. Thank u so much for the info on KINECT…

  79. Fantastic idea! My personal favourite so far, which I think a couple have now done is creating a flying drone. It uses the kinect’s cameras and depth sensors to assess how close it is to objects and stops the drone until the obstacle is moved. Very clever, looking forward to seeing what efforts your competition produces.

  80. Hi Matt,
    By thinking outside the box a little, I was able to achieve a minority report style UI, without having to really cut any code.

    There is already a program that allows users to control a cursor with Kinect, and there is an auto-click program available. I simply combined the two. This results in Kinect control over any program installed on the machine, and of course, the OS itself.

    I think I might create a linux distro that packages everything necessary to make this automatically work for any user, without any configuration required. I’ll let you guys know if I decide to pursue this.

  81. Hi Matt

    How about our project for a touch free interface of a DICOM viewer to manipulate
    CT and MRI images in the OR and during autopsy?
    Everything is done using open source software.

    Have a look:



  82. Hi Matt,

    please check

    We’ve basicallly created an opensource tool to send the realtime captured human skeleton trough the OSC protocol so anyone can easily build new tools/games.

    For instance, adding motion capture export capabilities like exporting to .bvh would be extremely simple with this 😉

  83. Hi Matt

    My last posting just disapeared.
    Once again, how about our project using the Kinect as touch free interface
    to review CT and MRI pictures?



  84. Dionne Saunders

    By the way, a video of Kinect controlling a PC is available upon request.

  85. Hey Matt,

    I’d like to recommend you take a look at the FAAST toolkit, see the link for evidence of what it can do. It brings full motion video gaming to old games on the PC. I’m sure the guys at USC would love your support!

  86. I saw a blind gentleman walking and tapping his cane the other day and I imagined a Kinect giving 3D audio clues as to the immediate surroundings perhaps eliminating the need for the cane or augmenting it and providing quicker mobility – this could be a very inexpensive and handy augmentation device – I’m over my head in projects and could not get to it soon so hopefully someone looking for a good project will build it open source:

    SoundDroid: Blind person assist => Kinect on glasses -> 3d Image -> Android -> [One of a number of processing algorithms] -> 3d audio streams -> Earphones on glass stems

    The processing could include whether the sounds represent the environment in the direction of the view or only certain deductions about the environment, such as a change in the floor level in your immediate view or an object you are headed into, different sounds for different colored lights (walk/don’t walk), volumes representing any of a number of items and so on.


  87. Fantastic idea! My personal favourite so far, which I think a couple have now done is creating a flying drone. It uses the kinect’s cameras and depth sensors to assess how close it is to objects and stops the drone until the obstacle is moved. Very clever, looking forward to seeing what efforts your competition produces.