I’m the lead programmer on our third year team, and this year we’ve decided to see if we can tackle vision targeting.
Our problem? We have no clue where to even begin.
Do we need a special camera? A special program? How do we get it to line up where we want it relative to the vision tape?
We’ll be doing our own research in the coming weeks, but any advice from others that have done this before would be greatly appreciated.
Thank you!
A great place to start is the Screensteps Live tutorials by WPI. They have lots of information and a whole section about vision tracking this year. You can find it here.
Start with GRIP:
Just as an FYI that video is 8 months old and is missing a lot of stuff that’s been added since. Screensteps and the wiki are the most up-to-date.
(Shameless plug – I’m a lead developer for GRIP)
So we’ve decided to let the RoboRio do our image processing, because getting something else up and running is pretty unrealistic for our team. Is there any way to use GRIP to generate LabView code? Because all we see is Python/Java/C++. Thanks!
The first thing that I’d do when starting on vision tracking is getting used to GRIP and opencv, you will be using both of these a lot. Download the opencv library from here and GRIP from here.
Play around with the GRIP settings, if you want examples on a GRIP pipline, check out our code here.
The steps that we did to vision track the lift were:
You need to find the distance, for this we found a distance constant that we can manipulate in the code to give us our actual distance, again check out our code here.
Once you the distance, you need to find out your distance from goal. Again, what we did was find a constant from the number of pixels to inches, and manipulated that in the code, you can find an example here
Once you have those two side, you can do a little bit of trig (tangent) and find the angle measure
Magic! PM me if you need anything else from me
GRIP can’t generate LabView code. You can download GRIP here and run it on your driver station laptop. There are some tutorials on screensteps on how to use GRIP. You can publish information about contours, blobs, and lines that GRIP finds to NetworkTables that you can read from your robot program. You can also generate Java, Python, or C++ code to run in a standalone program on a co-processor if you want. A lot of teams are using Raspberry Pis specifically for this.
TheGuyWhoCodes: You don’t need to have OpenCV installed to use GRIP. It’s completely standalone. Also, OpenCV only has downloads for Windows builds; for anything else (macOS, Linux, etc) you have to build from source. It’s a bit misleading to imply that you can just download and use it right away.
If we do run it on a Pi, is it even possible to let it communicate with the RoboRio which is running LabView? We assumed that was not possible.
Almost anything is possible. Remember the magic behind the scenes is just 1’s an 0’s, digital ons and offs. The languages you are using dont matter, so long as you have a ‘sender’ and a ‘reciever’ for those signals. Since this happens at the lowest level you will probably be using standardized protocols to do all the talking. There will be libraries for multiple languages to send/receive on these protocols.
They’re using LabVIEW and don’t want to use a coprocessor, and I’m pretty sure they don’t want vision lag through DS communication.
From the LabVIEW start screen, click Tutorials on the left, then click on the center column item that talks about vision. From there you can figure out how to implement the vision example, which is written for the high goal, and get it into your project.
If you’d be able to point us in the right direction to find the co-processor protocols, that’d be great (just in case) 
On another note, we’re looking to order hardware. I have heard that our Microsoft HD 3000 camera should be fine, but is there any specific place we should look for LEDs?
Something you also might consider is using IR instead of visible light. You just have to make sure you get a camera that does not have IR filters built in.
You could use a simple filter on the camera and attach an LED illuminator (850 or 950 nm) that matches the filter to light the target… this really helps with changing field lighting conditions.
We have found this to be quite effective.
THESE ring lights are pretty bog-standard for FRC now. You do not need a lot of light, just a very distinguishable hue from the rest of the field.
As far communication goes here is an excerpt from our internal vision document:
With a stream of position data being produced by the code it is necessary to pass that information on to the roboRio. This task can be accomplished a variety of ways, through GPio, USB, and I2c. We are going to use the networking capabilities of both systems. This can be accomplished with Network Tables.
It is important to understand the roboRio is the true hub of the system. All actuator commands need to be made via its protocols.
Communication via network tables can get as involved as you want. The general method is to pass a numeric array.
Installing network table packages to the Pi: https://www.chiefdelphi.com/forums/showthread.php?t=154302
Passing values out of the Raspberry Pi via network tables: https://www.chiefdelphi.com/forums/showthread.php?t=153838
Example Python code for passing values: GRIP-code-generation/python/samples/frc_find_red_areas/frc_find_red_areas.py at master · WPIRoboticsProjects/GRIP-code-generation · GitHub
Reading network tables data in LabView (and getting default vision code working): https://docs.google.com/document/d/1oPrW1DOTnh5DE7wL9wgypC0BRb3fzcpXRgqk1IGgzmc/edit
Also take a look at THIS thread about cameras. I ordered a Playstation EyeCamera to play with in Linux.
Are we able to install LabView on a Pi? Because I have one at my house I can donate if we can. That’s the easiest way for us to try a co-processor by far.
Vision is a major idea this year, however it may be wise to wait until next year. I’m not saying that it’s impossible, especially if you have a large and powerful programming team, but it may be smarter to spend your time on getting encoders perfected than trying to create a sort-of-working vision program, however, if you are interested in attempting, the best way to do it is to acquire a powerful processing board. The one thing you should never do is use the RoboRio for processing. FIRST added support for OpenCV, the vision processing library, on the RIO, but the RIO is far too slow for it to work, in fact the Raspberry Pi has proven almost too slow for vision tracking. However, as you gain power you lose ease of use. A NVidia graphics chip will carry vision processing to an unnecessary level, but it is a pain to program and communicate with the RIO, and even worse to power on the robot.
The first thing you are going to want to do is choose your board. Consider the languages that OpenCV is available for. The most support is clearly for C++, it’s native language, followed by Python. Java is new, and lacks support, however the library is similar across languages, so it may be possible, but copy-paste solutions become impossible. Once you choose a board, a programming language will be apparent. If it is viable to use C++, but if not you can make Python or Java work. Finally you need to install OpenCV onto your board. Installation depends on the board you choose.
Note: You need to use an LED mounted very close or around the camera for the use of the retro-reflective tape.
Now you have to start programming. Research HSV and thresholding. Once you understand that create a simple vision program that uses sliders to change HSV Values to get the optimal color reading. Now create your program based on the Sample Code in the OpenCV library. I cannot give you all of the code, as that’s pretty anti-FIRST, but I will tell you the program flow you need.
Pull Image-> Filter to HSV -> Filter out to Threshold -> FindContours -> Convert to Rectangles using lots of fun stuff (Polygons are important), -> Assign polygons to an array for pulling of corners -> Do math using those corners to get the information you need -> Send that information back to the RIO over PyNetworkTables.
All of this took me 3 months to figure out and get working, so I hope this gives you a head start, and I hope to see you at comp
-Max DeVos
KING TeC 2169
Also I recently foundthis 254 presentation about Vision, and I wish I would have found it earlier
Hi,
So my team started using the Pixy Cam for visioning during the summer, their are a couple of other teams that have used it. It is super simple for starting visioning teams to use, especially when one axis is only needed and the pixy does all its own processing. You just use the companion app, for pc, to teach the Pixy what you want it to track. Their are some teams that use SPI or I2C to communicate with it. During the offseason my team used its analog feature which gives you a voltage reading, you just need to convert it to pixels, which isn’t that difficult, and then you use a PID loop to tell it where to go. My team just used a P loop and we would get it to the exact location we wanted. The only bad thing about the analog feature is that you can only get x or y axis not both. You also need Led Ring Lights so it can pick up the retro-reflective tape.
http://charmedlabs.com/default/pixy-cmucam5/
http://cmucam.org/projects/cmucam5/wiki
You can buy it at either amazon or andymark
Hope this helps.
We only need one axis! We’ll look into this, if it’s that easy we may go with it.
Just want to confirm, it does work with LabView though, right?
If you’re using LabVIEW, you might want to look at the provided labview examples, since they’ve already written vision code for you that will detect the boiler quite well.
I thought the one thing you should never do was to release the magic smoke. Doing vision on a roboRIO does not release the smoke, BTW.
But seriously, vision processing is possible on the roboRIO and on far slower processors. It may be easier if you bring more horsepower, but this really depends on your image size, rate, the amount of processing you are doing, and what you are using the results for. Finally, the efficiency of the processing matters as well.
There are many good slide presentations around, and many opinions for how to do vision. Enjoy the challenge.
Greg McKaskle