2013 Lessons Learned for a Vision Co-Processor

[Jerry Ballard]

On the eve of Kick-Off, we'd thought it would be a good time to post our lessons learned from last year's competition.

Team 456 (Siege Robotics)

Hardware: Pandaboard with Linux Ubuntu, Logitech C110 Webcam
Software: OpenCV, yavta, iniparser, mongoose
Code: C

1) Control your exposure: For consistent results, avoid over saturation of bright targets, set and control the exposure (integration time) setting on the webcam. We found that a 8ms integration time worked best in most settings. Camera exposure settings were set using yavta: (https://github.com/fastr/yavta). Remember, many webcams have auto-exposure set as a default. The auto-exposure setting won't allow you to consistently identify targets in changing lighting conditions (see #2).

2) Do calibrate at competition: Each competition arena will have different lighting. Take advantage of the time provided (before competition begins) and calibrate camera to those settings. We setup a simple executable script that recorded movies at different camera exposures that we then ran against our target tracking systems off the field. I've attached frame shots of the same field at different camera exposures. Corresponding movies are available, let me know if you would like a copy of them.

3) Use a configuration file: We found that using a configuration file allowed us to modify target tracking settings during competition without having to recompile code. Recompiling code during competition is rarely a good thing.

4) OpenCV isn't always optimized for FRC Competition: In the pursuit of faster frame processing, we found that going to the traditional OpenCV RGB->HSV->Threshold(Value)&Threshold(Hue) pipeline was a major time-eater. We found a significant gain in performance from 15 fps to 22 fps by doing the thresholding on Value and Hue while converting from RGB. This allowed a reduction in computation load by eliminating the Saturation calculation and skipping obvious non-target pixels. If requested, we can post the algorithm here or it is also available on our github site. (GitHub is awesome!)

5) Avoid double precision: If our input source is a 640x480 image, most if not all calculations can be done with integer and single precision arithmetic. Using doubles adds more processing time = reduced fps processing.

6) Data transfer via HTTP was questionable: We used the Mongoose webserver code to transfer target angular coordinates to the control system (https://github.com/cesanta/mongoose). Coding-wise, the Mongoose code was easy to compile, simple to integrate, and multi-threaded (that is another discussion). During competition, sometimes it appeared that we would not get target lock as quick as was tested and we suspect that it was related to the HTTP communication between the PandaBoard and the CRIO. We are still trying to decide the best way to communicate between the vision co-processor and the CRIO (looking at UDP next).