How to use vision targets?

[The Doctor]

Our team is going to attempt vision targeting this year. We are wondering how other teams have gotten the targets to show up reliably on a camera? Things I have seen include shining a specifically-colored light at the targets (like green), in order to make them easier to distinguish for the program. Another option, though more complicated, involves using an infrared camera and infrared lights to make the reflective tape shine in IR.

Have any teams used either of these in the past? Or has anyone had good luck without any sort of assistance to target the tape?

[nardavin]

I would recommend that you start here: https://wpilib.screenstepslive.com/s/4485/m/24194

[Xanawatt]

What we did last year was got a camera that is not specifically designed for IR, but rather put an IR filter lens on the camera. Then just get IR leds and surround the camera with them pointed at the target.

[SamCarlberg]

Originally Posted by nardavin

I would recommend that you start here: https://wpilib.screenstepslive.com/s/4485/m/24194

Make sure you pay attention to GRIP and the official vision examples. The examples come with a GRIP pipeline that works to find the vision targets on the upper ring with a green ring light (you'll need to change the image source to point to the folder on your computer to get it working)

[jee7s]

Originally Posted by The Doctor

Our team is going to attempt vision targeting this year. We are wondering how other teams have gotten the targets to show up reliably on a camera? Things I have seen include shining a specifically-colored light at the targets (like green), in order to make them easier to distinguish for the program. Another option, though more complicated, involves using an infrared camera and infrared lights to make the reflective tape shine in IR.

Have any teams used either of these in the past? Or has anyone had good luck without any sort of assistance to target the tape?

Full disclosure: I work for Thorlabs, where my job is dealing with cameras, optics, and photonics. I point to some thorlabs.com webpages below purely for reference. You don't want to buy high quality optics for FRC. You don't need them, and they are too expensive to use on your robot under the rules.

A few points...

First, specifically regarding the light, more important than the wavelength is the source of the light. You want the light source to be as close to the detector as possible. Why? Because retroreflectors bounce light back to the source. There's an image in the WPI documentation that reflects that. The tape used in the competition is a sphere type retroreflector, so it doesn't impact polarization like a corner cube does (https://www.thorlabs.com/newgrouppag...ctgroup_id=145 see the 'Lab Facts' tab). There's a change to polarization as the wave bounces around the sphere retroreflector, but it's not as definite as the corner cube.

So, on to green vs IR. IR is nice because the light is invisible to humans. You can put a pretty bright IR source out there and not see it and not have it be damaging to the human eye. This works on many sensors because the silicon used to build them is sensitive to IR. One aspect of that sensitivity is the Quantum Efficiency, or the likelihood that the sensor converts the photon to an electron. A typical QE curve is here: https://www.thorlabs.com/images/TabI...ciency_780.gif

Note from that QE curve that there's a lower response in IR (on the right side >750nm), but keep in mind that you can blast the target with IR photons.

There's a catch, though. Most color cameras have some amount of IR blocking in them. The high quality ones have a pretty hard cut at about 750nm. That's because the filters used to create the RGB bayer pattern are all pretty transmissive in the IR range. They are good at separating red, green, and blue, but they all pass IR. So, the camera manufacturer puts an IR blocking filter in front of the sensor to get good color reproduction. To test this on your camera, point a TV remote (anyone remember the 2008 season?) at the sensor and see if it lights up white. If it does, your camera can see IR, if it doesn't, your camera blocks IR.

Also note from that QE curve that the curve peaks around 500nm, which is "green" to the human eye. That curve is an example of a typical silicon detector, and if your camera is under $400, it almost certainly has a silicon detector. So, silicon peaks in the green, which means that if you use a green light, you need the least amount of light to get a signal in the camera.

The basic method is this:
Get a ring light of LEDs and put your camera aperture in the center of the ring. That establishes the part above regarding putting the source as close to the detector as possible. Ideally you would use a beamsplitter to illuminate through the camera lens, but the ring light is a cheaper solution that accomplishes the job for FRC purposes.

Next, capture a short exposure image of the target. You want short enough an exposure that the background remains dark and the tape is green (or whatever color your illumination is). If the tape/target is white in your image and your illumination is green, then your exposure is too long and you are flooding the sensor with excess photons, saturating the green pixels. Those photoelectrons then spill over into the red and blue pixels resulting in a white part of the image. If you hit that condition, you are over exposing and as a result you are probably getting some background returns in your data.

Background image isn't worth anything to you, generally speaking. You want a fairly dark image, like the ones provided as examples by WPI. If you have a look at those examples for the peg, you can see a pretty clear specular reflection of the ring light in ../Vision Images/LED Peg/1ftH2ftD1Angle0Brightness.jpg.

One way to eliminate specular reflection is to polarize the light going in and filter the light coming back. As I mentioned, a retroreflector changes polarization. Well, a specular reflection does not change polarization. So, you could polarize the illumination using a linear polarizer, then filter your return light with another polarizer, and provide an adjustment of angle. Change the polarization angle between the camera and the light and you should see reduced intensity in specular reflections while not losing the retroreflected intensity. If you want to try that, there are cheap linear polarizer sheets available out there for less than $10. It takes some experimentation because the tape isn't as predictable as the retroreflector, but you can find an angle that maximizes retroreflector return while minimizing specular reflection return.

Specular reflections aside, the important thing is that your targets are rectangles. Please make sure your illuminator doesn't look like a rectangle in specular reflections. That's another good reason for a ring light: it's a circle, not a rectangle.

After you get the image, you want to extract the correct color plane that corresponds to your illumination. Then, you threshold that image and "find the rectangles". I put that last part in quotes because it involves some math (like a thresholding operation, edge detection, hough transform on the edge detection, search of the HT space, etc) that is really the "fun" part of the vision challenge. I don't want to spoil that for you.

Hope that helps you get started on how to illuminate a retroreflector for vision processing.

[orangeandblack5]

Originally Posted by jee7s

-snip-

As somebody with a similar set of questions, thank you very much. This will be super helpful, as our team has no clue where to start.