Camera tracking at 110ms

[Bomberofdoom]

Hi,

Following some of the calculations I made, our robot needs to be able to process an image from the camera at least every 110ms.

From what I've heard, the default settings in the Two Color Servo Camera Example, the robot processes the images once every ~200ms.

Can someone tell me how to set the properties of the camera in order to lower the process time to 110ms or less?

I'm looking for a specific (and if possible, exampled) process of doing so, but any kind of help will do.

Thanks,

[Greg McKaskle]

I've done the LV timings hundreds of times. 640x480 is around 5fps, 320x240 is around 10fps, and 160x120 is around 25fps. If you decimate some, you can improve it further.

You can definitely get 110ms or better. What are your other constraints?

Greg McKaskle

[TheOtherGuy]

Originally Posted by Greg McKaskle

I've done the LV timings hundreds of times. 640x480 is around 5fps, 320x240 is around 10fps, and 160x120 is around 25fps. If you decimate some, you can improve it further.

You can definitely get 110ms or better. What are your other constraints?

Greg McKaskle

I was wondering about this; my team is running the LabView tracking code and the best we get with 160x120 without decimation is 12 or 13 fps. Is there some trick to increasing this speed? We have yet to try tracking without live debugging (not sure how much time is lost in the communications with the laptop).

[Bomberofdoom]

Were those measued with or without decimating? How much does decimating help? (numbers please )

And I'm not quite sure what other kinds of constraints I might have.

[Alan Anderson]

Originally Posted by TheOtherGuy

I was wondering about this; my team is running the LabView tracking code and the best we get with 160x120 without decimation is 12 or 13 fps. Is there some trick to increasing this speed? We have yet to try tracking without live debugging (not sure how much time is lost in the communications with the laptop).

Most of the time is "lost" in keeping the LabVIEW front panels updated. Close them and you'll probably be a lot happier with the timing (assuming you have a way to see what the timing actually is).

[Greg McKaskle]

Here is the breakdown.

Open the demo panel, but not subVI panels. The example defaults to 320x240 with 2x2 decimation and both the image mask displays on.

With those settings and a target in view I get a little above 6 fps. If I turn off the display (by turning on gimbal tracking), I get 10.5. If I turn off the mask (by hitting the button below the mask, next to the fps) I get 11.5.

If you do not get these numbers there are possibly some things on your camera that are not set correctly. The ones I know of are -- you don't have an FRC/FRC account because the camera setup utility wasn't run. This causes twice the comms setup overhead before getting an image. Connect to the camera using a web browser and log in with FRC/FRC. If it doesn't work, run the camera setup or create the account by hand.

Another camera issue is if you set the compression near 0 or near 100. The default is 30. Near 0 or 100, the camera pipeline doesn't seem to be able to keep up and you will get lower frame rate.

Another issue is if you are in a dark room. To get a decent image when there isn't much light, the camera has two choices, expose the sensor longer, or increase the gain on the sensor to amplify stuff. The default setting for Exposure Priority is None which lets the camera decide. If you think this is your issue, you can set it to Frame Rate. The images may be noisier since the sensor noise will be amplified. The other option is to move to somewhere with more light.

If none of these get you similar issues, please post so that we have a reasonable baseline.

Here are the effects of image size and decimation. Both image display and mask are turned off.

Small (160x120)
1x1 decimation, No tgt in view : 20
2x2 decimation, No tgt in view : 24

1x1 decimation, One tgt in view : 16
2x2 decimation, One tgt in view : 19

Medium (320x240)
1x1 decimation, No tgt in view : 12
2x2 decimation, No tgt in view : 15
3x3 decimation, No tgt in view : 15

1x1 decimation, One tgt in view : 9
2x2 decimation, One tgt in view : 12
3x3 decimation, One tgt in view : 13

Large (640x480)
1x1 decimation, No tgt in view : 3.5
2x2 decimation, No tgt in view : 3.7
3x3 decimation, No tgt in view : 4

1x1 decimation, One tgt in view : 3.3
2x2 decimation, One tgt in view : 3.7
3x3 decimation, One tgt in view : 4

These are all taken with the panel open and with the servos running. Clearly you may have more I/O running than this, and if there is no delay or too small a delay, you could clearly have little CPU left to do vision.

Greg McKaskle

Greg McKaskle

[Bomberofdoom]

Porbably last question,
These FPS represent the amount of pictures taken and proccessed per second, right?

[Greg McKaskle]

Yes. They are the value in the fps indicator and represent completed image processing loops per second. 1000/fps will give ms per image.

By the way, I'd like to know what your issue was.

Greg McKaskle

[Bomberofdoom]

Wasn't an issue, it was more of a requirment from the cRio's and camera's capabilities in order to get a "panromaic scan" of the field , in whic the camera moves from right to left (0-170) at full speed (servo's max speed is 0.4s). From calculations of angles that the camera sees, where the camera is positioned, pixel area of a target from specific distance and more, the maximum process speed I need from my system is ~110ms. It's about 80-90ms if I include max-acceleration of robots and trailers on the field and when they're the closest to us

[Loubear]

Why are you scanning the entire panorama of the field? If you know where the robot's starting position, you could always just turn the camera before going onto the field to orient it in the right direction...

[Bomberofdoom]

and once you're done following one target (say, it's too full of empty cell's to be a potentioal target), won't you want to find the other targets?

[Loubear]

True, but you can only focus on one target at a time anyways, so why not save yourself some processing time by locking on, following that target, and switching targets only once you score (and still have time).

[Bomberofdoom]

First target in view while doing Panoramic scan gets locked. We'll put a "Search first Left" and "Search first Right" for more driver control on where the camera should search.