Getting the Kinect to work with OpenCV and libfreenect/My experience with the Kinect

[yash101]

The Kinect is a nice sensor, which has an IR projector, an IR camera and an BGR camera. The IR projector and camera work together to for depth perception because as you go further away, the IR projector's IR dots will be much more scattered. The Kinect also has some great hardware, and is quite competitively priced for what it's got.

Today, when I was hacking around with the Kinect, in Robotics, I learned quite a bit of stuff, and I also got some areas where I need some help and assistance.

So for the software I used, I installed Ubuntu using Wubi (though you can do so any way). After that finished installing, I updated and upgraded the system, so it was running like a hungry lion! I installed some software with the following command:
sudo apt-get install qtcreator pkg-config libopencv-dev python-opencv libfreenect-dev libfreenect-demos chromium-browser flashplugin-installer
That installed a lot of stuff and took about half an hour on our school's wifi (very fast). By the way, skip the last two because that installs a browser and the adobe flash plugin for Firefox/chromium.

So I then blacklisted the Kinect, so it would be accessible to the system.

I was able to successfully run some sample programs, downloaded using apt-get:
freenect-glview

However, I am unable to compile any libfreenect source code! I am using QT Creator as the C++ IDE.

So, I ran pkg-config libfreenect --libs, and got the output I was looking for, so I was able to use it in the project file for QT Creator. However, even though I included the libfreenect.hpp, the which showed up as a valid file, I kept getting random errors every time I tried to compile code.

What did I do wrong, and how could I fix it? Does anyone have some sample code to interface the Kinect, libfreenect with OpenCV for the image processing? I just want to exploit the infrared camera that is in the Kinect! OpenCV can do the rest.

[Greg McKaskle]

Compilers shouldn't really give random errors. They are almost guaranteed to return errors which are hard for people to understand, and small changes in a #include or a build setting can lead to very different errors.

Perhaps this would be easier if you were to include some of those error messages.

Greg McKaskle

[yash101]

I'll try to take screenshots/videos/etc. to show what's going on

[faust1706]

Shot in the dark: you didn't set up up the gcc as the c++ compiler.

http://ratchetrockers1706.org/vision-setup/

[yash101]

Originally Posted by faust1706

Shot in the dark: you didn't set up up the gcc as the c++ compiler.

http://ratchetrockers1706.org/vision-setup/

LOL. That might be it. However, why did OpenCV work, then? That's kinda wierd! Let me tweak those settings and report back! I need to write a whitepaper about this once I'm done, so rookies don't fall into the same hole as I!

[yash101]

OK. So I did change some compiler settings, and create a new project. However, I'm still unable to compile with libfreenect. I did get rid of all the errors that libfreenect gave me, until my compiler gave me 3000+ errors !

I think it was because I was trying out some C code, and the setup was really a mix of both C and C++! Bad me

I need to now work on porting this IplImage and CvArr stuff to CvCapture and Mat!

[faust1706]

Do you have the namespaces cv and or std?

You can just put mat img = cvcapture() I think. I'm on my phone so I apologize if my capitalization isn't proper for functions.

Look into cvqueryframe. That is one I have used in the past and you can save the output as a mat.

[MoosingIn3space]

I have attached my sample code for my school's KinectDevice class, written in C++, which provides a simple interface to the Kinect. You simply write

// assuming 'capture' is a KinectDevice
cv::Mat frame;
capture >> frame;

in order to read a frame from the Kinect. Hope it helps.

[yash101]

This is the problem I am facing (below)

[faust1706]

Will you (attempt) to compile through terminal? It _should_ give a more detail error. More to work with.


make clean
make
./(executable name) will execute the program

In case you don't know of forgot

[yash101]

Here's the output:
rm -f main.o
rm -f *~ core *.core
g++ -c -pipe -O2 -Wall -W -D_REENTRANT -DQT_WEBKIT -DQT_NO_DEBUG -DQT_CORE_LIB -DQT_SHARED -I/usr/share/qt4/mkspecs/linux-g++ -I../TestOpenCVLibFreenect -I/usr/include/qt4/QtCore -I/usr/include/qt4 -I. -I../TestOpenCVLibFreenect -I. -o main.o ../TestOpenCVLibFreenect/main.cpp
g++ -Wl,-O1 -o TestOpenCVLibFreenect main.o -L/usr/lib/x86_64-linux-gnu `pkg-config opencv --libs` libfreenect --libs` -lQtCore -lpthread

Sadly, I don't speak robot (or do I?) so I don't really understand what the problem is!