Target acquisition with camera- color issues

[Quarkster]

We are at the DC competition and having significant trouble getting the camera to recognize targets.

Our code is based on the example camera code for labview, although teams using C++ could still be helpful to us.

To those teams who have cameras working at the competition, what settings are you using?

One thing that might be an issue is that the camera is supposed to be returning HSL values, but when I probe it in the block diagram, it says we have an RGB image. But it could just be converting it for display purposes

[billbo911]

Originally Posted by Quarkster

We are at the DC competition and having significant trouble getting the camera to recognize targets.

Our code is based on the example camera code for labview, although teams using C++ could still be helpful to us.

To those teams who have cameras working at the competition, what settings are you using?

One thing that might be an issue is that the camera is supposed to be returning HSL values, but when I probe it in the block diagram, it says we have an RGB image. But it could just be converting it for display purposes

Have you set the White Balance and Brightness adjustments for the environment you are playing in??

[keehun]

Originally Posted by Quarkster

One thing that might be an issue is that the camera is supposed to be returning HSL values, but when I probe it in the block diagram, it says we have an RGB image. But it could just be converting it for display purposes

We noticed this phenomenon as well! We fortunately noticed this mid-season and was able to solve this.


(Sorry the icons are "?"; I'm on my mac and it does not have the WPI library drivers)

I don't know if this will work for you, but this worked for us so I decided to post it. Yes, for tracking, we also use HSL images.

I believe we saw "HSL" in the probe as well. I may be mistaken, but that is what I remember. We observed that the first image going in to the while loop was of HSL kind, but all the subsequent images were in RGB... We thought the "setting" was not retained. We somehow got it to stay in HSL after a long battle.

If you want more help, I can post more code.

[Greg McKaskle]

The easiest way to see HSL is to open the Find Two Colors VI. Unless you have the very first release of that example, then panel of that VI has an HSL display on it and a Boolean beneath to turn on the HSL conversion explicitly. This will display HSL so that you can hover with the mouse and make good measurements. If you you don't have this incorporated, you can add it pretty easily. Note that the implicit HSL threshold is faster, which is why this isn't on all the time.

Greg McKaskle

[Greg McKaskle]

As for values, what I can tell you is that the fields are quite bright and the black curtain also complicates things. When the camera faces the black curtain, it receives less light and increases the exposure time to compensate. Doing this only makes the glare on the targets worse.

Things that will help are listed in the document containing the recommended camera settings. The primary thing is to try lowering the Brightness setting on the camera. This will lower the exposure and keep the targets from being overexposed. Don't be concerned about setting it too low when things are as brightly lit as the event, but you may want to restore it when you return to the classroom or shop setting.

To set brightness in LV, the example was written with this as a parameter which you can change on they fly. Don't forget to make the current value default once you finish experimenting.

Another thing to consider is the effect that bright light has on color saturation. As luminance increases in an image saturation will naturally drop. So in an event where there are bright lights, you will likely see the saturation numbers of everything drop somewhat. When you threshold the image looking for the pretty saturated hues of the target fabric, you are supplying both upper and lower saturation thresholds. If the lower saturation thresholds are too high, you will get streaks and parts of the target image with glare not making it into the mask.

Greg McKaskle

[Rich Kressly]

Greg! You were a huge help to our team in DC. Thanks! Our cam values from DC are posted in a screenshot here: http://www.chiefdelphi.com/forums/sh...0&postcount=36

[Chexposito]

i know our programmers messed with the settings to the two game colors became pretty much the only visible colors on the display. making the targets more obvious. ours doesn't track to well because of the problem with one of the axis of rotation being controlled by a motor, instead of servo, which likes to move sporadically. talking to the ni programmer at the kc regional helped a lot.

[Andrew Bates]

Has anyone been able to track a trailer and run traction control at the same time. We have been having trouble with our camera tracking taking to much time to execute causing our traction control to break. We changed it so that when we are tracking our traction control is turned off but this is unsatisfactory for us. Has anyone else had a similar problem?

[gwytheyrn]

I'm not sure if this is just our own problem, but when we run the camera example and the target is near the camera and near the top of the image, the masks begin to split apart, making the script believe that the target leaves the frame before it really should. The higher up the target is, the farther apart the two masks become until the top one flies off the screen (even though a large portion of the color is still on the actual input image). Anyone know a fix for this?

[Manoel]

Originally Posted by gwytheyrn

I'm not sure if this is just our own problem, but when we run the camera example and the target is near the camera and near the top of the image, the masks begin to split apart, making the script believe that the target leaves the frame before it really should. The higher up the target is, the farther apart the two masks become until the top one flies off the screen (even though a large portion of the color is still on the actual input image). Anyone know a fix for this?

We have noticed that, so you're not alone, but it does not affect our use of the camera, so I'm afraid I can't offer a solution. Hopefully someone from NI will check this and be able to help you.

[byteit101]

one way to get the hsl or rgb values is to use the dashboard (create a PCVideoServer in C++), and take a pic of the target from the camera. then go into gimp, and crop the green or pink.
for RGB, go to Colors, Info, Histogram, and slect the color field, and you can find out the range of colors for rgb, then do the same for green.
for hsl, go to colors>components>decompose and select hsl. then note the layer name, and then go to colors info histogram, and that is the h s or l max and min. then you can plug them in

[Greg McKaskle]

To learn about the performance, you may want to isolate the tasks and determine the performance characteristics of each independently. If using LV, you can try using the profiler, but as with any advanced tool, it takes a bit of practice to learn how to interpret the feedback.

If you want to use less CPU to process the images, the easiest way with the default code is to shrink the image size or to decimate the image a bit more. I have no idea what your code for traction control does, but if you post it or describe it you may get some input.

The issue with the camera looking up to the target and catching glare is caused primarily by the placement of the lights. I want to try something tomorrow to improve the image. I have the equipment, but I'd rather not post before giving it a try.

If you can't improve the image captured by the camera, you may be able to recognize the situation by noticing that the image, once whole, is splitting, and build the rect to encompass the halves. Tricky, and it'd be much better to have a good image, but as you've noticed, the lighting is really harsh.

Greg McKaskle

[Manoel]

Greg,

The issue gwytheyrn described above has nothing to do with lightning. If you put your hand in front of the target and bring the camera close enough, the mask will separate, as he described, and your hand's shadow will show on BOTH colors, even if they're very separated from each other in the mask.

[The Lucas]

Originally Posted by gwytheyrn

I'm not sure if this is just our own problem, but when we run the camera example and the target is near the camera and near the top of the image, the masks begin to split apart, making the script believe that the target leaves the frame before it really should. The higher up the target is, the farther apart the two masks become until the top one flies off the screen (even though a large portion of the color is still on the actual input image). Anyone know a fix for this?

I've noticed that as well when calibrating the camera values in LV. We are actually using C++ to program the robot so I haven't investigated the LV specific issue thoroughly. I think it has with the masks overlapping when the target is near. LV seems to artificially separate the masks with dark gap in between that only get larger as the target gets nearer. I would guess this has to do with LV's approach of drawing boxes around the expected area for the target and allowing little overlap. In C++ I compensated for the lost target (initially LV could see but C++ couldn't) at these ranges by allowing more overlap. Keep on mind, I program in C++ and have only glanced at the LV code (I understand LV), so my understanding and guesses could be poorly concieved (but I thought they were worth stating)

[Greg McKaskle]

Oh, the vertical separation. I thought you meant a vertical stripe of glare and a horizontal separation.

Yes, the vertical separation is an artifact of the LV algorithm that doesn't do the mask to the entire image on the second color. It copies out small rectangular images and applies the algorithm to those. As you get close to the image and get more of an angle, the curvature of the color boundary no longer looks much like a line. I hadn't realized this would lead to issues.

To fix/improve it, I'd change the subVI that computes the top and bottom search rectangles so that they overlap with the initial rectangle by some percentage. More explicitly, the bottom of the top rect should increase by say 25% of the rect height, and the top of the bottom rect should decrease by the same amount. The subVI has a misleading name of Green Rects.vi.

Let me know if that doesn't fix it.

I'm still going to look at the glare thing, and I'll post back if the idea shows any promise.