Infrared Camera Results

[sparkytwd]

I believe this video adequately makes the case for why my team uses infrared for the retro illuminator.

This video is shot in our workshop under normal lighting conditions. The thin grey lines in the video are the overhead fluorescent lights. The camera is a PS3 eye cam, capturing 320x240 @ 125fps with gain set to 0. It has been modified by replacing the OEM lens assembly with one including an IR-pass, visible block filter and 4.3mm, 70deg FOV lens. The illuminator in is a single 2W 850nm IR Led on a standard star heatsink/pcb.

http://www.youtube.com/watch?v=HzzN5bDGuBc

Raw Video:
https://drive.google.com/file/d/0B4u5dB6M1G2_bjBGZHpZYTk4UlU/edit?usp=sharing

[billbo911]

Nicely done. That makes target detection very easy!

I am toying with the concept of using an IP camera with built in IR for night vision, and Pan/Tilt.
This will be run through an offboard processor. These cameras are available for under $100 from multiple sources. Hacking them for control has already been done, so acquiring and processing images should be a snap.

[sparkytwd]

I am toying with the concept of using an IP camera with built in IR for night vision, and Pan/Tilt.


You'll want to make sure you can block visible light as well as control the gain of the camera.

[billbo911]

You'll want to make sure you can block visible light as well as control the gain of the camera.

Yep, that's why this will be an off season project.

Some camera's have a moveable IR filter to help correct color in daylight. That filter will need to be swapped out for a IR pass/visible block filter. Additionally, that filter will need to be in instead of out for IR use.
Controlling exposure might be a bit more tricky. Some camera's are locked into "Auto Exposure". So, this will be a "hack", but a real learning process for all involved.

[Greg McKaskle]

That looks fine. Though I don't know that it looks any better than what teams achieve with visible light and a quick exposure setting.

Also, I've taken infrared video of the field using a Kinect. The trusses with about 100 high intensity CANs pointing down at the field make it a bit more interesting. The tape they use to mark lines of the field, any lexan or shiny element of the field that has the right orientation will start reflecting that light back into your camera. This doesn't make it impossible, but it makes both IR and visible light a bit more challenging.

Since you don't necessarily know what will be at the arena -- you may even have a Mountain Dew vending machine within view of the field -- it isn't a horrible idea to have additional ring lights in your parts alliance. And you can use the calibration time and the dashboard's logged images to look for problems -- no matter the wavelength of the light you are processing.

Greg McKaskle

[faust1706]

We are also using ir light (for the third year in a row). We removed the ir filter from our 3 cameras and have added ir leds (40 per camera I believe). A method to implement would be to copy your raw image and threshold both of them different ways: grayscale and hsv, then only keep the overlap. The "equation" is img = gray - hsv, then subtract img from gray again so the lights are removed. Just a simple fix to the light issue.

[kmusa]

...The illuminator in is a single 2W 850nm IR Led on a standard star heatsink/pcb...

Don't know if 2W is supply power or emitted power, nor the angle of the emitted beam, but IR light in that wavelength range is potentially hazardous, since the eye generally won't react to it by blinking or closing.


Also, I've taken infrared video of the field using a Kinect. The trusses with about 100 high intensity CANs pointing down at the field make it a bit more interesting. The tape they use to mark lines of the field, any lexan or shiny element of the field that has the right orientation will start reflecting that light back into your camera. This doesn't make it impossible, but it makes both IR and visible light a bit more challenging.


This is so true. We used a Kinect as a IR camera for Rebound Rumble. We were amazed how bright the nets appeared in our image.

-Karlis

[sparkytwd]

We are also using ir light (for the third year in a row). We removed the ir filter from our 3 cameras and have added ir leds (40 per camera I believe). A method to implement would be to copy your raw image and threshold both of them different ways: grayscale and hsv, then only keep the overlap. The "equation" is img = gray - hsv, then subtract img from gray again so the lights are removed. Just a simple fix to the light issue.


Not sure what you mean. In this case the HSV image is basically 0,0 in H and S, we only need to threshold the value.

[sparkytwd]

Don't know if 2W is supply power or emitted power, nor the angle of the emitted beam, but IR light in that wavelength range is potentially hazardous, since the eye generally won't react to it by blinking or closing.
Supplied power.

It's a risk, but even small (32 LED count) ring lights emit more. It also has a 135degree beam, so it's a bit less concentrated.

[faust1706]

Not sure what you mean. In this case the HSV image is basically 0,0 in H and S, we only need to threshold the value.

you might as well threshold the other two channels as well since it is already split, it doesn't hurt.

I only use this as a fail safe. Regular thresholding works, and differentiating based off the characteristics off the remaining pixel clusters.

[sparkytwd]

Also, I've taken infrared video of the field using a Kinect. The trusses with about 100 high intensity CANs pointing down at the field make it a bit more interesting. The tape they use to mark lines of the field, any lexan or shiny element of the field that has the right orientation will start reflecting that light back into your camera. This doesn't make it impossible, but it makes both IR and visible light a bit more challenging.

A large part of those problems will depend on the chemistry of the lights involved, halogen, mercury arc, and LED sources emit practically no IR, while incandescent, fluorescent, and xenon can.

To help with that this year we're using specific 850nm filters, which should help keep down the extraneous light.


Since you don't necessarily know what will be at the arena -- you may even have a Mountain Dew vending machine within view of the field -- it isn't a horrible idea to have additional ring lights in your parts alliance. And you can use the calibration time and the dashboard's logged images to look for problems -- no matter the wavelength of the light you are processing.
Greg McKaskle

We've got enough headroom from the power driver (and spare LEDs) for up to 3, as well as polarizing film to further "select" just our own light source.

[sparkytwd]

you might as well threshold the other two channels as well since it is already split, it doesn't hurt.

I only use this as a fail safe. Regular thresholding works, and differentiating based off the characteristics off the remaining pixel clusters.

Ideally, from a memory and processing time perspective, in this case you would get the camera to send you a pure grey image.

If you get the camera image in RGB format, you can discard 2 of the channels and just treat that as your grey image. If you get it as a Y??? image, you can just keep the Y channel.

Converting to an HSV image for thresholding is a bit of a waste in this case.

[Greg McKaskle]

It sounds like you are well prepared. Bringing equipment to adjust to the field was exactly what I was suggesting. The lighting at the events can be any combination of the bulbs you listed, and can also include diffused or direct sunlight. I haven't attended a district event, but have seen huge variability from regional to regional.

The major benefit of the retro-reflective material is that it lets teams provide their own source light. They can use color/frequency, pulses, etc. I haven't tried polarized light with the model of tape that is being used. Does it depolarize or not?

I'd be curious to see images or video from the actual event.

Greg McKaskle

[kmusa]

Supplied power.

It's a risk, but even small (32 LED count) ring lights emit more. It also has a 135degree beam, so it's a bit less concentrated.

With the {visible) light rings, even if there is a significant IR component, your eyes will react by blinking, squinting, or turning away. The issue with IR only sources is this defense mechanism doesn't apply.

In addition, it's really easy to increase "brightness" by adding LEDs. Unfortunately, there isn't a obvious indication that the resulting power output is approaching levels of concern.

-Karlis

[sparkytwd]

The major benefit of the retro-reflective material is that it lets teams provide their own source light. They can use color/frequency, pulses, etc. I haven't tried polarized light with the model of tape that is being used. Does it depolarize or not?

I'd be curious to see images or video from the actual event.

Greg McKaskle

We used polarizers 2 years ago, and testing with them showed polarization was preserved upon reflection (done by rotating the illuminators polarizer 90deg out of phase with the cameras). In practice though you would want to use a single polarizer covering both camera and light.

Our first competition is March 8th, hopefully we'll have some videos from that event. One challenge at this time is the infrared camera itself is a bit useless for the drivers, so we may end up dynamically adjusting the game as the match progresses.

[sparkytwd]

With the {visible) light rings, even if there is a significant IR component, your eyes will react by blinking, squinting, or turning away. The issue with IR only sources is this defense mechanism doesn't apply.

In addition, it's really easy to increase "brightness" by adding LEDs. Unfortunately, there isn't a obvious indication that the resulting power output is approaching levels of concern.

-Karlis

In this case I meant IR ring lights, like those commonly used for security cameras.

The real risk, as I understand it, is not that you don't blink or squint, it's that you're pupils don't dilate. This is a problem for sunglasses that don't filter UV as your wider pupils take in more UV than is healthy. I suspect a 2W led with 135 degree emission pattern is going to be roughly in line with the lighting of a workshop or a competition field.

[yash101]

Just filter the saturation! Your goals are the only things lit up so a relatively high sat value will mean a perfect threshold!

[sparkytwd]

Just filter the saturation! Your goals are the only things lit up so a relatively high sat value will mean a perfect threshold!

I believe you mean value? Saturation is how colorful a pixel is.

[sparkytwd]

Results from the Week 5 Auburn PNW District Match:

http://vimeo.com/90479090#t=2m15s

We do get some reflection in the diamond plate material, but as you can see only the occasional glimpse of another overhead lightsource, and nothing during the autonomous period.

ocupus now provides in match camera control (https://github.com/hackcasual/ocupus/commit/95c18c963fef05405b3bc21362ae6321cb10a5d1) which allows the driver to switch camera settings from 0 gain autonomous mode back to automatic gain, allowing them to line up shots with the goals and human players.