I am working on an unrelated-to-FIRST project at college in which I am doing image processing on tiny robots with cameras. The only thing is, the only image processing I have ever dealt with is the CMUCam, which does all it’s processing onboard so it can give you back nice “x1 y1 x2 y2 x_center y_center” values.
Unfortunately, I don’t get such pretty data. I get an array of bytes that represents an image in the YUV color space. I want to do some analysis of this image to find a blob of a certain color. I am not entirely sure how to do this and am going to poke around to try and find things smart people who have done this before have written.
Has anyone else worked with this sort of do-it-yourself image processing? Can you point me at any resources? I’d be most grateful!
Well, to do the task that you are describing should be pretty simple:
Run the entire image through a color thresholding algorithm–replace pixels that are “close” to your desired color with a 1, and everything else with a 0.
If you want to do this the way the CMUCam does, scan through the resulting binary array, and every time a 1 is encountered add its X and Y coordinates (your scan variables) to separate large registers. Also increment another register that will hold the total number of 1s encountered. Divide X and Y by that register, and you will have the center and size of the blob in a very similar manner to the CMUCam.
If you need to isolate different blobs of the same color, this becomes more involved. You can take a look at my FALCON system’s algorithms for one complicated but quite effective way to do it.
There are a lot of vision algorithms, and picking the right one isn’t straightforward. It really depends on what your source data is and what you are trying to get out of it, as well as your performance limitations. If you are trying to find colored blobs without definite shapes, some things that I can think of off the top of my hand are expanding shapes (such as rectangles or circles), fitting shapes, random sampling algorithms, and weighted averages / other statistical calculations. If shape does matter, it gets more complicated. Basically, the best way to start learning about this sort of thing is just to mess around and to see what works. Reading papers is also useful for researching algorithms / strategies if you can get through the math (at least the important bits).
Here are a few pdfs from a robot competition. The code samples inside are in java, so you can ignore the topics on java-specific features (i.e. how it handles binary manipulations or performance of various ways to access image data). But the ideas are mostly pretty general (they don’t go into that much depth) so it could be a good jumping off point.
To repeat, the pdfs won’t help you with implementation, but they may spark some ideas / give a basic overview of a few concepts.
National Instruments has “Vision” software. It was used during the Aim High season to count the balls in the lower goals. The cameras used were unibrain fire-I firewire cameras.
Unfortuantly, all my good sample code is at home and I’m going to be leaving for the Portland regional after work. When I get home on Sunday I will try to post a few samples of what you can do with this library.
I don’t have much experience with this but as far as centroid tracking goes, I would guess that it can probably be done pretty easily since there is a lot of coherency from image to image, so you can with a fair amount of confidence start at the centroid of the previous frame then expand out in all directions.
You may be able to use this algorithm: Phase Correlation
it involves taking the FFT of the image then doing some fancy stuff with it. basically if you run it from frame to frame it will exploit the coherency to very quickly tell you how the centroid has shifted
OpenCV is the standard choice for most hobby robotics projects. OpenCV requires knowledge of C or C++. If you want more links on OpenCV, I can give a few of the CV functions I use commonly to find blobs. If you don’t know C, and you have time to post process the image, I would suggest using Matlab’s image processing toolbox (it works well with webcams). If you have a IEEE 1394 compliant camera(not a webcam), then the Labview vision toolbox would also be useful for realtime and post-processing.
I am, unfortunately, programming for an embedded controller (on a Surveyor SRV1) in an unheard of programming language (occam pi). This library seems to be for PCs, so won’t work.
I am investigating some other stuff, thanks for all the suggestions so far! If anyone else knows any good resources please pass them on!
You might want to play with the PPM format, it is really easy to work with all the data is stored in ASCII text or similar. Then you can look into using a compression algorithm.
When I was working on an embedded system image processing project, I used the CIMG library, it is really easy to use, and allows you easily display the images in a gui with a lot of feed back with about 10 lines of code. If you want some sample code from my project, ask.
I would also recommend OpenCV. However, if you looked at it and it does not fit your needs (they are for PC’s as you said), you can write your own functions for doing it.
Please correct me if I’m wrong (I have never dealt with color tracking), but I would take this approach:
Find the YUV values (or range of values) that correspond to your target color.
Locate the pixels that are in that range (assuming your array indices correspond to image pixels), cancel out all the pixels and small blobs that are alone (representing false positives) and you are left with your blob.
Often rectangle-fitting is adequate; start the rectangle on a pixel that you know to be part of the object (due to color), and expand the rectangle in all directions until the colors are outside the threshold.
