On board micro-controller with camera

[adciv]

Running processing on the RoboRIO at higher than 640x480 is possible, but I would not recommend it. The frame rates drop too low. I'll have to dig out the benchmarking I did last year, but I think it was single digits.

Latency will depend on what you use for processing,what you need, and how big your images are. The variables get a little too complex to estimate, but you can code around them somewhat. In 2012, we did offboard processing and use a sample and move for our turret where we'd pause the turret, get the angle to change to, move to that positiion, stop, and recheck.

We haven't tried onboard in a bit, but next up will be trying to use the Tegra TK1 offered to First. It's a quad core 2+ghz Arm A15 running linux and 192 cuda cores. With OpenCV, this should be sufficient for anything we need. I'm not sure of anything which gives you more bang for your buck.

[RoboNerd01]

Don't forget about the very limited bandwidth FIRST allows you. Most of the time streaming high quality video causes problems. I know this has little to do with video processing, but you should always take the bandwidth into consideration.

[Greg McKaskle]

The image sizes in WPILib were selected to match the initial Axis IP cameras. When later Axis camera models introduced other sizes, the camera VIs were not updated. This wasn't a technical limit, but a practical one.

Other questions you may want to try and answer -- what do you gain by using a higher resolution? The related question is ... What is the minimum resolution you can process in order to make the measurements you are interested in?

To answer your first question directly, you are not technically limited to 640x480, but the processing time is proportional to the number of pixels, and there may be no need to use a higher resolution. It depends on the size of the objects you are trying to identify, the lens, and the distance.

To answer the second question, you can break the processing down into its components and look at the overall time spent for the different approaches. To process an image, it first has to be requested, the camera digitizes, encodes, transmits, there may be more steps here depending on the connection, etc. Then the processing end needs to decode the image and store it. It perform its processing steps, and transmit the results back to the code that cares.

Think about what affects each step and the tradeoffs of higher resolution. Your initial post already captures some of this. A PC has a faster processor than the one on the robot. Do I transmit the data to the remote PC, or do I mount the PC on the robot and simplify the transmission? Or do I simplify the processing? Can I better optimize the request and capture of the image?

It is useful to hear advice from other teams, but keep the steps above in mind as they describe why they made various tradeoffs. Then think about what you really need to use the sensor for and what resolution and framerate is sufficient.

Greg McKaskle