stereo vision system with OV7620

[steveg]

Now that build season is over, I've been thinking about a fun little (or huge!) side project that would be a fun challenge. Since we've got two cmucams now, I was thinking that it would be fun to try to set up a stereo vision system for path finding not un-like that which is used on the Mars rovers.

A few days ago I hit the library and picked up a bunch of books on computer vision algorithms and have been pouring through them, and it definitely looks feasible except the CMUcam, as it is now, doesn't have nearly enough image processing capability to handle what I’m going to need to do to correlate points on the images, generate the 3d terrain, etc. I noticed that it's possible to reprogram the controller on the CMUcam with a parallax sx-key. Has anyone tried that?


Unless anyone has had great success reprogramming the CMUcam, I think I want to switch over to a different embedded controller to do my processing. Right now, I'm leaning toward using a Blackfin DSP running uCLinux (something similar to the stamp board Eldarion used for his sweet ball tracking system) or a Gumstix. Ideally, I'll do all the processing on the DSP, then pass instructions to the RC that will be basically acting as a PID controller for the drive train. Also, I want to integrate ultrasonic sensors to do proximity warning and collision prevention. A scanning LADAR system would be nice too. I'll have to ask the guys I work with here at Lincoln Laboratory about that, but it's probably out of my price range :)

So, I guess any insight or recommendations people could offer would be very much appreciated. Of course, I'll keep you all updated on my progress.

[KenWittlief]

wow! awesome project.

Im thinking if you use stereovision you should not need the rest of the stuff? (distance sensors?)

could you use an auto focus camera instead to find the distance to specific objects in view? I believe most auto focus cameras now on the market focus optically (sharpest edges in the image).

[Kevin Watson]

Now that build season is over, I've been thinking about a fun little (or huge!) side project that would be a fun challenge. Since we've got two cmucams now, I was thinking that it would be fun to try to set up a stereo vision system for path finding not un-like that which is used on the Mars rovers.

A few days ago I hit the library and picked up a bunch of books on computer vision algorithms and have been pouring through them, and it definitely looks feasible except the CMUcam, as it is now, doesn't have nearly enough image processing capability to handle what I’m going to need to do to correlate points on the images, generate the 3d terrain, etc. I noticed that it's possible to reprogram the controller on the CMUcam with a parallax sx-key. Has anyone tried that?


Unless anyone has had great success reprogramming the CMUcam, I think I want to switch over to a different embedded controller to do my processing. Right now, I'm leaning toward using a Blackfin DSP running uCLinux (something similar to the stamp board Eldarion used for his sweet ball tracking system) or a Gumstix. Ideally, I'll do all the processing on the DSP, then pass instructions to the RC that will be basically acting as a PID controller for the drive train. Also, I want to integrate ultrasonic sensors to do proximity warning and collision prevention. A scanning LADAR system would be nice too. I'll have to ask the guys I work with here at Lincoln Laboratory about that, but it's probably out of my price range :)

So, I guess any insight or recommendations people could offer would be very much appreciated. Of course, I'll keep you all updated on my progress.I think it will be a fun project, but I strongly suspect that your correlation algorithm will have problems with the CMUcam2's output. In an earlier discussion, I posted a few links that may be of interest: http://www.chiefdelphi.com/forums/showpost.php?p=434122&postcount=7

-Kevin

[steveg]

Im thinking if you use stereovision you should not need the rest of the stuff? (distance sensors?)

could you use an auto focus camera instead to find the distance to specific objects in view?
Well, I was thinking about that, and in theory, yes, but if someone decides to walk in front of my little robot while it's making its way from point A to point B, it might not 'see' the person with the vision system until it's too late. So I wanted to add a little bit of redundancy for those sorts of instances.

Also, with regard to the use of autofocus lenses, I would think that just like in regular photography, you would want to just focus the lens at the hyperfocal distance. that way, everything from half of that (hopefully relatively small) distance to infinity would be in focus.

I strongly suspect that your correlation algorithm will have problems with the CMUcam2's output. In an earlier discussion, I posted a few links that may be of interest: http://www.chiefdelphi.com/forums/s...122&postcount=7

-Kevin
I subscribed to the OpenCV mailing list a while back when I saw that thread, and proceded to get my inbox flooded :], and I was planning on maybe using some of their algorithms where possible. I was just planning to use the OV cameras just because they're what I have access to. Kevin, can you think of different cameras that are low cost and may be better suited to my application?