Appreciate's bird's eye (Human vision assist in autonomous)

[VacioArconte]

Howdy,
We are Team 2468 Appreciate in Austin, Texas. As other teams have been doing, we have also utilized vision triggering in our autonomous program in order to successfully shoot into the hot goal. Our approach is different from CheesyVision because our robot always immediately drives forward, but it shoots only when we tell it. We use LabView along with our laptop's webcam(bird's eye) to achieve this.

Our LabView program uses NI Vision Acquisition Software to get images from our driver station's webcam to process in the code. Using Vision Assistant, we generated LabView geometric pattern matching code to recognize our sign. Now, we track the recognized image in our webcam and surround it with a red box and fire when the sign is held less than ten degrees to the horizontal, as indicated by a shift in the color of the box to green. The VI outputs true or false; true if the box is green, false if the box is red. This allows us to communicate during autonomous to tell the robot when to fire.

Tracking:


Fire!:



Here is how to implement it~

Download the NI Vision Acquisition Software. It can be found here:
http://www.ni.com/download/ni-vision...13.09/4409/en/

Download our software, visit:
https://github.com/Casolt/2468-Sign-Autonomous

A sign will need to be created prior to using this code. The sign will need to have high contrast between the symbol and background and be matt to avoid reflection. We use a black poster board with white tape on top to write the word "Fire".
We have included our image that we use to detect the sign. This image should be a clear, non-distorted picture of the sign which you are using. The picture will be referenced by a file path given to the Wait Until Sees Fire.vi.

You will need to use the IMAQdx Open Camera and IMAQdx Configure Grab VI to begin grabbing images from the webcam. Place the Wait Until Sees Fire VI inside of the Default Case in the Dashboard loop. Connect the Image out of the Configure Grab to the Image in of the Wait Until Sees Fire VI.

Use smart dashboard to write a Boolean to the robot and in the robot's Autonomous Independent, wait until that Boolean is true before launching your ball.

How the VI is implemented in our Dashboard VI:


How the robot responds to the smart dashboard:


We appreciate help from Greg McKaskle during the development of this system. He dedicated his own time to teach several students about NI vision tools.

[nick1023]

Thanks for posting your work on this! We did make the Kinect work for MSC but are unhappy with the reliability of the Kinect server (crashes sometimes when we plug in the kinect) and the bulk of carrying around the connect.

We tried running this last night using your Match Geometric Pattern Alg combined with a print out of your sign as well as trying to generate our own. We've tried this while connected to our actual robot as well as while running the simulation robot.

We are not getting any matches. I am able to see the cam image and the "fire" png image that you provided after drilling all the way into the Find Fire VI.

When we drill down into the subvis we can successfully probe wires to confirm that data is flowing (ROI Descriptor, Image Dst, Settings, Error Out)...until we open the IVA Match Geometric Pattern Algorithm vi block inside of Find Fire Symbol VI. All probes from inside of this IVA Match Geometric VI show "not executed". Probing the terminals exiting the IVA Match VI (Matches, Calibrated Matches and Number of Matches) shows 0 as the values for all of the fields (as opposed to not executed or another numbers).

I've taken the PNG file and printed it to 11X17 copy paper. It does have a little bit of gloss on it but I suspect that I should be seeing something happening inside of that IVA Match Geometric VI regardless. Also my print of the fire sign png file does show up well in the probed black and white value I'm pulling from the webcam inside of Find Fire Symbol.

Attached is my code zipped.

Any thoughts are appreciated. Thanks.

[Greg McKaskle]

The most likely cause for not matching is that the scale is wrong.

Team 2468 built the sign they wanted to use, held the sign standing behind the line. We used Vision Assistant to capture the image, opened it into the Geometric Match, trained it by selecting the content and saving as a png.

The code is generic, and it is the file that you would change if you wanted to adapt to a new sign. The matching parameters are actually rather strict. The more variation you tell the function to expect, the longer it takes to complete.

If you experiment with the distance or scale of the sign, you can probably get it to work, but perhaps it would be easier, and more informative to just train it again on your own sign. May I suggest "WE APR3C8", or something similar.

The other thing to glance at in Vision Assistant is to zoom in and verify that the green contours trace most of the high contrast area. If not, experiment with the way you hold the sign, the sliders for contrast, etc. I believe that 2468 uses pretty standard settings.

Greg McKaskle

[VacioArconte]

Hello Nick,
After testing the code you attached, everything appeared to work fine.

I only needed to make 2 minor changes, but neither seem to be related to the problem you experienced.

I made the file path wired to the Wait Until Sees Fire.vi. I also put the code in the "0.0.0.0" case in the dashboard main so that I could test without using a robot.

Try the attached code. It is the modified version of your code and it works on my laptop.

Make sure you have the NI Vision Acquisition Software installed.

Your labview should also have vision libraries installed.

When you do have it working, you may need to hold the sign very far away from the webcam.