Quote:
Originally Posted by MrRoboSteve
Our team is wanting to get serious about vision this year, and I'm curious what people think is the state of the art in vision systems for FRC.
Questions:
1. Is it better to do vision processing onboard or with a coprocessor? What are the tradeoffs? How does the RoboRIO change the answer to this question?
2. Which vision libraries? NI Vision? OpenCV? RoboRealm? Any libraries that run on top of any of these that are useful?
3. Which teams have well developed vision codebases? I'm assuming teams are following R13 and sharing out the code.
4. Are there alternatives to the Axis cameras that should be considered? What USB camera options are viable for 2015 control system use? Is the Kinect a viable vision sensor with the RoboRIO?
|
There are a number of threads that already talk about computer vision in FRC, however, it never hurts to talk about it every once in awhile as the participants of the competition are always changing.
I'll go through your questions one at a time.
1. I do not know if this question has an objective answer. Teams have had success with with the cRIO, off board, or using a second computer like the PI or an o-droid. The trade off for using an on board computer is obviously weight, but PI weighs something like 200g so that shouldn't concern you. You have to somehow communicate what the vision system outputs to your cRIO, for the past 3 years we've been doing a simple udp.
As for off board, I have heard it can be slow, and with the cap on data per team, I personally think it is no long a valid option.
Using the cRIO, and programming it in labview or something, is doable, and there are plenty of resources around to do so, but the truth is it won't be as fast a something running on an SBC.
I haven't looked at the specs of the roboRIO, so my opinion on that shouldn't be taken seriously. I heard it is "linuxy," what have yet to get a clear definition of what that actually means. It would be cool to make your vision program an application and run it on the roboRIO, but I don't know if that is doable.
2. As for vision libraries, any will do, but some have more support than others. I don't know the state of openni, but last time I checked it was no longer funded. OpenCV has been around since....I want to say 1999. It is used in industry and academia and that is the one I suggest using, but of course there exists bias. I played around with roborealm a tad, but it seems too simple, I feel someone using it wouldn't get the fundamental understanding of what is happening, but if you are just looking to figure out how far away you are from something and don't really care about understanding, then I suggest that.
There is always the option of writing your own computer vision library, of course, but I'd be willing to bet money it wouldn't be as encompassing or efficient as existing open sourced cv libraries, like ni and opencv.
3. I like to think computer vision is a little unique. Yeah, you can develop code bases, but the vision task changes so dramatically over the years, except for game piece detection, that I don't think it'd be worth it. Take opencv for example. It is an extremely developed library that now focuses on readability and ease of developing. If you really know what you're doing, you can solve the vision program in 100 lines of code, if you don't know what you're doing, it can take upwards to 2k. You could maybe group some routine operations, such as binary image morphology, into one step, but even then you'd only be saving one line of code.
Once you get a game piece detection code, however, that can just be "pulled from the shelf" and used year after year.
4. As long as your library can get the image from the camera, it really doesn't matter which one you use. The common option is the axis camera. We've used the kinect for 2 years, then this past year we used 3 120 degree cameras (it was more a test of concept for future years). If you want to do some research on hardware, look up: asus xtion, pixy, playstation eye, kinect, and keep and eye out for the kinect one to be "hacked" and thus being usable.
I feel that computer vision is a developing aspect of frc. My freshman year it was logo motion. I didn't ask around in the pits, but I didn't see many cameras to my memory. When 1114 and 254 were doing multiple game piece autonomous (in the einstein finals, mind you), I think it really inspired people I think, as did 254's autonomous routine this year. Just look at all path trajectory stuff that has been posted over the path several months.
This is my 2 cents on the topic. Take it how you will. If you have any questions, or want to see some code from my previous vision solutions (from 2012-2014), I'd be more than happy to go over it with you.
I emailed a first rep requesting they consider an aerial camera for future games, and if they do that, or allow teams to put up their own aerial camera, then on board vision systems become obsolete because essentially you have an "objective" view, instead of a "subjective" one.