Getting the Kinect to work with OpenCV and libfreenect/My experience with the Kinect

[yash101]

The Kinect is a nice sensor, which has an IR projector, an IR camera and an BGR camera. The IR projector and camera work together to for depth perception because as you go further away, the IR projector's IR dots will be much more scattered. The Kinect also has some great hardware, and is quite competitively priced for what it's got.

Today, when I was hacking around with the Kinect, in Robotics, I learned quite a bit of stuff, and I also got some areas where I need some help and assistance.

So for the software I used, I installed Ubuntu using Wubi (though you can do so any way). After that finished installing, I updated and upgraded the system, so it was running like a hungry lion! I installed some software with the following command:
sudo apt-get install qtcreator pkg-config libopencv-dev python-opencv libfreenect-dev libfreenect-demos chromium-browser flashplugin-installer
That installed a lot of stuff and took about half an hour on our school's wifi (very fast). By the way, skip the last two because that installs a browser and the adobe flash plugin for Firefox/chromium.

So I then blacklisted the Kinect, so it would be accessible to the system.

I was able to successfully run some sample programs, downloaded using apt-get:
freenect-glview

However, I am unable to compile any libfreenect source code! I am using QT Creator as the C++ IDE.

So, I ran pkg-config libfreenect --libs, and got the output I was looking for, so I was able to use it in the project file for QT Creator. However, even though I included the libfreenect.hpp, the which showed up as a valid file, I kept getting random errors every time I tried to compile code.

What did I do wrong, and how could I fix it? Does anyone have some sample code to interface the Kinect, libfreenect with OpenCV for the image processing? I just want to exploit the infrared camera that is in the Kinect! OpenCV can do the rest.