Go to Post Help me Chief Delhpi forums, you're my only hope. - Sev [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rating: Thread Rating: 29 votes, 5.00 average. Display Modes
  #1   Spotlight this post!  
Unread 10-10-2014, 12:37
MrRoboSteve MrRoboSteve is offline
Mentor
AKA: Steve Peterson
FRC #3081 (Kennedy RoboEagles)
Team Role: Mentor
 
Join Date: Mar 2012
Rookie Year: 2011
Location: Bloomington, MN
Posts: 582
MrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond reputeMrRoboSteve has a reputation beyond repute
Vision: what's state of the art in FRC?

Our team is wanting to get serious about vision this year, and I'm curious what people think is the state of the art in vision systems for FRC.

Questions:

1. Is it better to do vision processing onboard or with a coprocessor? What are the tradeoffs? How does the RoboRIO change the answer to this question?

2. Which vision libraries? NI Vision? OpenCV? RoboRealm? Any libraries that run on top of any of these that are useful?

3. Which teams have well developed vision codebases? I'm assuming teams are following R13 and sharing out the code.

4. Are there alternatives to the Axis cameras that should be considered? What USB camera options are viable for 2015 control system use? Is the Kinect a viable vision sensor with the RoboRIO?
__________________
2016-17 events: 10000 Lakes Regional, Northern Lights Regional, FTC Burnsville Qualifying Tournament

2011 - present · FRC 3081 Kennedy RoboEagles mentor
2013 - present · event volunteer at 10000 Lakes Regional, Northern Lights Regional, North Star Regional, Lake Superior Regional, Minnesota State Tournament, PNW District 4 Glacier Peak, MN FTC, CMP
http://twitter.com/MrRoboSteve · www.linkedin.com/in/speterson
  #2   Spotlight this post!  
Unread 10-10-2014, 13:26
billbo911's Avatar
billbo911 billbo911 is offline
I prefer you give a perfect effort.
AKA: That's "Mr. Bill"
FRC #2073 (EagleForce)
Team Role: Mentor
 
Join Date: Mar 2005
Rookie Year: 2005
Location: Elk Grove, Ca.
Posts: 2,384
billbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond repute
Re: Vision: what's state of the art in FRC?

Quote:
Originally Posted by MrRoboSteve View Post
Our team is wanting to get serious about vision this year, and I'm curious what people think is the state of the art in vision systems for FRC.

Questions:

1. Is it better to do vision processing onboard or with a coprocessor? What are the tradeoffs? How does the RoboRIO change the answer to this question?

2. Which vision libraries? NI Vision? OpenCV? RoboRealm? Any libraries that run on top of any of these that are useful?

3. Which teams have well developed vision codebases? I'm assuming teams are following R13 and sharing out the code.

4. Are there alternatives to the Axis cameras that should be considered? What USB camera options are viable for 2015 control system use? Is the Kinect a viable vision sensor with the RoboRIO?
Solid questions Steve!

I can not speak for all of FRC. I can only speak from the perspective of a 3 time award winner for the vision/object tracking system we developed and used in 2014.

I'm going to try to answer your questions, from our perspective, in order.

1) Up through 2014, it was "better" to use a co-processor. Vision tracking on the cRio will work, but unless it is done VERY CAREFULLY and with limited requirements, it could easily max out the processor. We did use it successfully a few years ago, but we really limited the requirements. Since 2013, we started using a PCDuino (This link is to a retired version, but it is the version of the board we used.).
How does the RoboRIO change the answer to this question? Sorry, I can't say either way. We are not a Beta team, so we have no direct experience with the RoboRio. My guess is, it will have "much better" performance, but how much better remains to be seen. (I too am curious!)

2) We used OpenCV running under Ubuntu. Our scripts were written in Python. We will likely stick with this for 2015, but that determination is yet to be made. The PCDuino offers excellent access to the GPIO pins, thus allowing us to do some really neat software tricks.

3) Our code was shared in this post. That post is the main reason we won the "Gracious Professionalism Award" on Curie this year. This code is board specific, but can easily be modified to run on many different boards.

4) Sorry, without being a Beta team, we can not address this question.
__________________
CalGames 2009 Autonomous Champion Award winner
Sacramento 2010 Creativity in Design winner, Sacramento 2010 Quarter finalist
2011 Sacramento Finalist, 2011 Madtown Engineering Inspiration Award.
2012 Sacramento Semi-Finals, 2012 Sacramento Innovation in Control Award, 2012 SVR Judges Award.
2012 CalGames Autonomous Challenge Award winner ($$$).
2014 2X Rockwell Automation: Innovation in Control Award (CVR and SAC). Curie Division Gracious Professionalism Award.
2014 Capital City Classic Winner AND Runner Up. Madtown Throwdown: Runner up.
2015 Innovation in Control Award, Sacramento.
2016 Chezy Champs Finalist, 2016 MTTD Finalist
  #3   Spotlight this post!  
Unread 10-10-2014, 13:31
faust1706's Avatar
faust1706 faust1706 is offline
Registered User
FRC #1706 (Ratchet Rockers)
Team Role: College Student
 
Join Date: Apr 2012
Rookie Year: 2011
Location: St Louis
Posts: 498
faust1706 is infamous around these partsfaust1706 is infamous around these parts
Re: Vision: what's state of the art in FRC?

Quote:
Originally Posted by MrRoboSteve View Post
Our team is wanting to get serious about vision this year, and I'm curious what people think is the state of the art in vision systems for FRC.

Questions:

1. Is it better to do vision processing onboard or with a coprocessor? What are the tradeoffs? How does the RoboRIO change the answer to this question?

2. Which vision libraries? NI Vision? OpenCV? RoboRealm? Any libraries that run on top of any of these that are useful?

3. Which teams have well developed vision codebases? I'm assuming teams are following R13 and sharing out the code.

4. Are there alternatives to the Axis cameras that should be considered? What USB camera options are viable for 2015 control system use? Is the Kinect a viable vision sensor with the RoboRIO?
There are a number of threads that already talk about computer vision in FRC, however, it never hurts to talk about it every once in awhile as the participants of the competition are always changing.

I'll go through your questions one at a time.

1. I do not know if this question has an objective answer. Teams have had success with with the cRIO, off board, or using a second computer like the PI or an o-droid. The trade off for using an on board computer is obviously weight, but PI weighs something like 200g so that shouldn't concern you. You have to somehow communicate what the vision system outputs to your cRIO, for the past 3 years we've been doing a simple udp.

As for off board, I have heard it can be slow, and with the cap on data per team, I personally think it is no long a valid option.

Using the cRIO, and programming it in labview or something, is doable, and there are plenty of resources around to do so, but the truth is it won't be as fast a something running on an SBC.

I haven't looked at the specs of the roboRIO, so my opinion on that shouldn't be taken seriously. I heard it is "linuxy," what have yet to get a clear definition of what that actually means. It would be cool to make your vision program an application and run it on the roboRIO, but I don't know if that is doable.

2. As for vision libraries, any will do, but some have more support than others. I don't know the state of openni, but last time I checked it was no longer funded. OpenCV has been around since....I want to say 1999. It is used in industry and academia and that is the one I suggest using, but of course there exists bias. I played around with roborealm a tad, but it seems too simple, I feel someone using it wouldn't get the fundamental understanding of what is happening, but if you are just looking to figure out how far away you are from something and don't really care about understanding, then I suggest that.

There is always the option of writing your own computer vision library, of course, but I'd be willing to bet money it wouldn't be as encompassing or efficient as existing open sourced cv libraries, like ni and opencv.

3. I like to think computer vision is a little unique. Yeah, you can develop code bases, but the vision task changes so dramatically over the years, except for game piece detection, that I don't think it'd be worth it. Take opencv for example. It is an extremely developed library that now focuses on readability and ease of developing. If you really know what you're doing, you can solve the vision program in 100 lines of code, if you don't know what you're doing, it can take upwards to 2k. You could maybe group some routine operations, such as binary image morphology, into one step, but even then you'd only be saving one line of code.

Once you get a game piece detection code, however, that can just be "pulled from the shelf" and used year after year.

4. As long as your library can get the image from the camera, it really doesn't matter which one you use. The common option is the axis camera. We've used the kinect for 2 years, then this past year we used 3 120 degree cameras (it was more a test of concept for future years). If you want to do some research on hardware, look up: asus xtion, pixy, playstation eye, kinect, and keep and eye out for the kinect one to be "hacked" and thus being usable.

I feel that computer vision is a developing aspect of frc. My freshman year it was logo motion. I didn't ask around in the pits, but I didn't see many cameras to my memory. When 1114 and 254 were doing multiple game piece autonomous (in the einstein finals, mind you), I think it really inspired people I think, as did 254's autonomous routine this year. Just look at all path trajectory stuff that has been posted over the path several months.

This is my 2 cents on the topic. Take it how you will. If you have any questions, or want to see some code from my previous vision solutions (from 2012-2014), I'd be more than happy to go over it with you.

I emailed a first rep requesting they consider an aerial camera for future games, and if they do that, or allow teams to put up their own aerial camera, then on board vision systems become obsolete because essentially you have an "objective" view, instead of a "subjective" one.
__________________
"You're a gentleman," they used to say to him. "You shouldn't have gone murdering people with a hatchet; that's no occupation for a gentleman."
  #4   Spotlight this post!  
Unread 10-10-2014, 16:10
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Vision: what's state of the art in FRC?

A few clarifications and a few of my own answers.

NI Vision isn't the same as openni.

The OS on the roboRIO is described in quite a bit of detail here http://www.ni.com/white-paper/14627/en/

The code is on github.

1. There are tradeoffs here that depend on the robot, the strategy, and the team's resources. All have been made to work. The roboRIO is somewhere between 4x and 10x faster. It has two cores and includes NEON instructions for vector processing.

But vision is a hungry beast and will bring any computer to its knees if you just brute force it. The "right" solution depends on where the camera is mounted, where it is pointed, lighting, lenses, and of course processing. Makes it challenging, huh.

2. All of them are capable of solving the FRC imagining problems quite well, IMO.

3. I've seen plenty who make it work.

4. The WPI libraries will continue to support Axis and will support many UVC cameras via NI-IMAQdx drivers. This means that many USB webcams will work. Additionally, we have also tested the Basler USB3 industrial cameras. I have not tested Kinect directly on the roboRIO, though if you google myRIO and Kinect you will see others who have.

Greg McKaskle
  #5   Spotlight this post!  
Unread 10-10-2014, 18:13
ToddF's Avatar
ToddF ToddF is offline
mechanical engineer
AKA: Todd Ferrante
FRC #2363 (Triple Helix)
Team Role: Mentor
 
Join Date: Apr 2011
Rookie Year: 2011
Location: Newport News, VA
Posts: 602
ToddF has a reputation beyond reputeToddF has a reputation beyond reputeToddF has a reputation beyond reputeToddF has a reputation beyond reputeToddF has a reputation beyond reputeToddF has a reputation beyond reputeToddF has a reputation beyond reputeToddF has a reputation beyond reputeToddF has a reputation beyond reputeToddF has a reputation beyond reputeToddF has a reputation beyond repute
Re: Vision: what's state of the art in FRC?

One word: CheesyVision

(From the perspective of a Mech E)
__________________
Todd F.
mentor, FIRST team 2363, Triple Helix
Photo gallery
video channel
Triple Helix mobile
  #6   Spotlight this post!  
Unread 10-10-2014, 19:46
faust1706's Avatar
faust1706 faust1706 is offline
Registered User
FRC #1706 (Ratchet Rockers)
Team Role: College Student
 
Join Date: Apr 2012
Rookie Year: 2011
Location: St Louis
Posts: 498
faust1706 is infamous around these partsfaust1706 is infamous around these parts
Re: Vision: what's state of the art in FRC?

Quote:
Originally Posted by Greg McKaskle View Post

But vision is a hungry beast and will bring any computer to its knees if you just brute force it. The "right" solution depends on where the camera is mounted, where it is pointed, lighting, lenses, and of course processing. Makes it challenging, huh.
I don't think people realize this. After the image is acquired, it is literally pure mathematics. If you aren't careful, as in don't put in a wait function, your computer will try to execute the program as fast as possible, and the program is basically in an infinite loop until you somehow close the program. (I believe the majority of teams don't gracefully exit their program after a match, but instead simply power off).

To give an example, when you have a binary image you desire, as in your targets are white (1) and everything else is black (0), you preform the operation described in this paper (http://wenku.baidu.com/view/6cb52ede...aa811dad5.html) This paper was published in 1983. There do exist other methods, but the computational cost is about the same. (See also: http://en.wikipedia.org/wiki/Edge_detection).
Basically what this step does is reduces a 640x480 (or whatever resolution image you have) matrix to a sequence of curves that allow you to do cool things, such as recursively approximate a polygon to guess how many sides it has.
__________________
"You're a gentleman," they used to say to him. "You shouldn't have gone murdering people with a hatchet; that's no occupation for a gentleman."
  #7   Spotlight this post!  
Unread 10-10-2014, 21:01
yash101 yash101 is offline
Curiosity | I have too much of it!
AKA: null
no team
 
Join Date: Oct 2012
Rookie Year: 2012
Location: devnull
Posts: 1,191
yash101 is an unknown quantity at this point
Re: Vision: what's state of the art in FRC?

What is the purpose? Do you want to process a handful of images at the beginning of the match, or do you want to continually process images and perform AI like pathfinding? Those questions are extremely important for answering many of your questions. The first option should not require more power than the RoboRIO offers. The second option would probably benefit from a coprocessor. These tiny boards can offer a lot of power, easily many times the available oomph of the RoboRIO.

Running OpenCV on the RoboRIO is far from realistic. The hard disk space is quite limited, and as it seems to me, there will be missing pieces of linux which will make it extremely difficult to compile all the dependencies of OpenCV.

Also, I think OpenCV can be a tad inefficient in general. I was running a small test and OpenCV's windows maxed out one of my system's cores (i3)!

I really think converting color spaces, thresholding and some of these relatively simple transformations should be much faster, especially because the formula should be constant.
  #8   Spotlight this post!  
Unread 10-10-2014, 22:11
faust1706's Avatar
faust1706 faust1706 is offline
Registered User
FRC #1706 (Ratchet Rockers)
Team Role: College Student
 
Join Date: Apr 2012
Rookie Year: 2011
Location: St Louis
Posts: 498
faust1706 is infamous around these partsfaust1706 is infamous around these parts
Re: Vision: what's state of the art in FRC?

Quote:
Originally Posted by yash101 View Post
Also, I think OpenCV can be a tad inefficient in general. I was running a small test and OpenCV's windows maxed out one of my system's cores (i3)!

I really think converting color spaces, thresholding and some of these relatively simple transformations should be much faster, especially because the formula should be constant.
It can be inefficient, but it great at what it does. It is an open source library that people volunteer their time to contribute to. I think I'm going to be able to get away with taking 6 credit hours next semester for doing research in the field of computer science. That means I'll have dedicated time to work on stuff, like optimizing the opencv library (or just the functions I use regularly, such as erode, dilate, find contours, appoxpoly, solvepnp, and optical flow)

I've still not looked at the source code for opencv in depth. I really hope some things in it aren't parallel-ly computed because I love doing parallel computing and it'd give me great practice.

Like you mentioned...2 months ago to me, it spawns threads when it'd be quicker not to do it. Maybe I could simple put a condition that if the image resolution < (x,y), then don't spawn another thread.
__________________
"You're a gentleman," they used to say to him. "You shouldn't have gone murdering people with a hatchet; that's no occupation for a gentleman."
  #9   Spotlight this post!  
Unread 10-10-2014, 23:43
Caboose's Avatar
Caboose Caboose is offline
Programmer - LabVIEW, C++, C#, Java
AKA: James Parks
FRC #0900 (Zebracorns)
Team Role: Alumni
 
Join Date: Jan 2009
Rookie Year: 2008
Location: Seattle, WA
Posts: 72
Caboose has a spectacular aura aboutCaboose has a spectacular aura about
Re: Vision: what's state of the art in FRC?

Team 900 this year is currently working with a nVidia Tegra TK1(Quad ARM Cortex-A15 with Kepler GPU with 192 CUDA cores, <$200) to tackle vision processing. So far we are seeing good results with high FPS(>40) with large image resolutions(720p) doing just filtering at the moment all on the GPU with OpenCV. We are working on tracking next.
__________________
navX Labview Library

"Robots are aluminum shavings, held together by zip-ties."

myManga
  #10   Spotlight this post!  
Unread 11-10-2014, 00:02
Abhishek R Abhishek R is offline
Registered User
FRC #0624
Team Role: Alumni
 
Join Date: Mar 2013
Rookie Year: 2012
Location: Texas
Posts: 892
Abhishek R has a reputation beyond reputeAbhishek R has a reputation beyond reputeAbhishek R has a reputation beyond reputeAbhishek R has a reputation beyond reputeAbhishek R has a reputation beyond reputeAbhishek R has a reputation beyond reputeAbhishek R has a reputation beyond reputeAbhishek R has a reputation beyond reputeAbhishek R has a reputation beyond reputeAbhishek R has a reputation beyond reputeAbhishek R has a reputation beyond repute
Re: Vision: what's state of the art in FRC?

Answering question 4, the Kinect is a viable means of vision sensing. I'd recommend checking out this paper from Team 987, who used the Kinect very effectively as a camera in 2012's FRC challenge, Rebound Rumble. I believe one of the major advantages of the Kinect is it's depth perception is much better than a standard camera, though I'm not really a vision expert.
__________________
2012 - 2015 : 624 CRyptonite
Team Website
  #11   Spotlight this post!  
Unread 11-10-2014, 00:24
faust1706's Avatar
faust1706 faust1706 is offline
Registered User
FRC #1706 (Ratchet Rockers)
Team Role: College Student
 
Join Date: Apr 2012
Rookie Year: 2011
Location: St Louis
Posts: 498
faust1706 is infamous around these partsfaust1706 is infamous around these parts
Re: Vision: what's state of the art in FRC?

Quote:
Originally Posted by Abhishek R View Post
I believe one of the major advantages of the Kinect is it's depth perception is much better than a standard camera, though I'm not really a vision expert.
I don't really understand what you mean by depth perception. It has a depth map, if that is what you mean. The kinect is a great piece of hardware because it has an rgb, ir, depth camera, and a built in IR light (though I suggest adding some IR leds if you're going the IR route). It is fairly cheap now, and there has been a lot of work done with it, but if you're using only the rgb camera, the only thing different is how much it distorts the image. All the cameras you will ever think about using basically have the same amount of distortion, so that shouldn't be a concern. If it is, then calibrate your camera:

http://docs.opencv.org/doc/tutorials...libration.html

But I'm going to say that it is a waste of time unless you are doing an camera pose calculation.

The kinect does have a subtle advantage over other cameras, like the axis: It draws attention to it. Little kids love seeing things they see everyday used in a way they never thought possible.
__________________
"You're a gentleman," they used to say to him. "You shouldn't have gone murdering people with a hatchet; that's no occupation for a gentleman."
  #12   Spotlight this post!  
Unread 11-10-2014, 08:29
mwtidd's Avatar
mwtidd mwtidd is offline
Registered User
AKA: mike
FRC #0319 (Big Bad Bob)
Team Role: Mentor
 
Join Date: Feb 2005
Rookie Year: 2003
Location: Boston, MA
Posts: 714
mwtidd has a reputation beyond reputemwtidd has a reputation beyond reputemwtidd has a reputation beyond reputemwtidd has a reputation beyond reputemwtidd has a reputation beyond reputemwtidd has a reputation beyond reputemwtidd has a reputation beyond reputemwtidd has a reputation beyond reputemwtidd has a reputation beyond reputemwtidd has a reputation beyond reputemwtidd has a reputation beyond repute
Re: Vision: what's state of the art in FRC?

1. For me, I think this mostly depends on whether or not you want to processes as you move. If you're able to process from a stopped position, then you should be fine processing from the crio. Pray for more reflective tape, because that stuff makes life 100x easier. Also make sure you grab a camera that you can modify the exposures on, and ideally one that will hold those exposure settings after a reboot.

That being said, if you're looking to process while moving, I'd recommend going with a coprocessor. Where it may be feasible to do it on the roborio, its probably safer to do it on a coprocessor, and you'll get much better performance/resolution utilizing a coprocessor. Like 900 I'm very excited about the Jetson TK1, and with the new voltage regulator, getting set up with it should be simplified.

2. I have stayed away from RoboRealm. This is more of a personal preference but any software that requires licenses immediately makes it more challenging to work in a distributed fashion. This year we utilized both NI Vision and Open CV. The NI Vision library was used for hot and cold, and Open CV for Java was used for distance to target calculations. I was fond of this set up, because it allowed us to work in the same language offboard and onboard. That being said, I think the advantages the TK1 presents may outweigh the advantages of working in a single language.

NI Vision was a lot easier to get started with. Once the camera was calibrated correctly, it was relatively simple. We spent 90% of our time trying to make it more efficient (essentially a moot point). Once we landed on only doing hot cold detection onboard, that made things easier.



Open CV took a lot more to get set up with. If you haven't done vision processing in the past, and opt to go with something like open cv, be prepared to spend most of the season on it and even with that, it may never find its way on the robot. In the backpacking world we have an expression "Ounces make pounds", and it's definitely true in first. Expect a coprocessor to be one of the first things to go when you have to shed weight.

3. They're out there, but you'll have to dig, and you'll have to dig through full code bases to find it. From what I've seen teams often release their whole code base as a whole rather than a set of libraries.

4. As many others, I'm not on a beta team, but if you're using NI Vision, the Axis Camera is actually a fairly nice solution. The big things I've found is that WPIlib is already configured for the Axis Camera specs. Also, it holds it's values after a reset, so you don't have to worry about your exposure settings resetting in between matches.

All that being said, I think 900 is probably a good reference point for the state of the art solution/execution. The way they implemented their single frame vision processing was rather clever. Also, it sounds like they're following the FIRST roadmap by getting started on the Jetson TK1 early. I think that is where a lot of the beneficial state of the art stuff is going to go. Teams have been utilizing the kinect and doing object detection for several years now, but it seems to me these are more bullet points for awards as opposed to practical implementations.

For me, the state of the art stuff I'm interested in, is that which eases implementation while addressing the various constraints for the platforms we work on.
__________________
"Never let your schooling interfere with your education" -Mark Twain

Last edited by mwtidd : 11-10-2014 at 10:19.
  #13   Spotlight this post!  
Unread 11-10-2014, 19:29
yash101 yash101 is offline
Curiosity | I have too much of it!
AKA: null
no team
 
Join Date: Oct 2012
Rookie Year: 2012
Location: devnull
Posts: 1,191
yash101 is an unknown quantity at this point
Re: Vision: what's state of the art in FRC?

Quote:
Originally Posted by faust1706 View Post
It can be inefficient, but it great at what it does. It is an open source library that people volunteer their time to contribute to. I think I'm going to be able to get away with taking 6 credit hours next semester for doing research in the field of computer science. That means I'll have dedicated time to work on stuff, like optimizing the opencv library (or just the functions I use regularly, such as erode, dilate, find contours, appoxpoly, solvepnp, and optical flow)

I've still not looked at the source code for opencv in depth. I really hope some things in it aren't parallel-ly computed because I love doing parallel computing and it'd give me great practice.

Like you mentioned...2 months ago to me, it spawns threads when it'd be quicker not to do it. Maybe I could simple put a condition that if the image resolution < (x,y), then don't spawn another thread.
I've been working on a library extension for OpenCV that adds features (there's currently no vanilla way to rotate an image x number of degrees!). I am also going to use OpenMP to attempt to create extremely high-performance functions that utilize all cores on your system!

The nice thing about OpenMP is that if it is not available, everything will work properly. The function won't be multithreaded though!

OpenCV actually has three modes for threading, as I remember. OpenCV can use OpenMP and IntelTBB and disabled.

I guess that you could use C++11 threading to parallelly perform many simple tasks. It is quite cool how the threading can be done with lambdas:
PHP Code:
std::thread x([](){printf("Hello World, from a thread :D");}); 
or using regular function pointers:

PHP Code:
void hello()
{
    
printf("Hello World, from a thread :D");
    return;
}

int main()
{
    
std::thread x(hello);
    
x.join(); //We're not doing anything new afterwards, so the thread would crash if we did not join it!

Instead of just skipping threading, better threading techniques should be used! Let's say cvtColor:
If the res is less than 32 by 32 (let's just say), then skip threading. This resolution should be quite low because threading is actually incredibly fast! Use libraries like pthread because of their speed. News Flash: C++11 threading under POSIX uses pthread .
What I would do is use divide the image up into equal parts, with the denominator as the hardware concurrency. I would then run my own version of cvtColor with no optimizations on that small image. Afterwards, I would stitch those images back together to return. Voilla! You have a cvtColor function that is highly optimized using your hardware concurrency. This really dictates how many threads the computer can run TRULY parallelly (not switching back and forth in threads!).

I believe the DLib has some optimized color conversion code too. Just convert the Mat to cv_image and it should work beautifully:
dlib::cv_image<dlib::bgr_pixel> image(cv::imread("image.png"));

Good luck! Maybe we can work on this stuff together!

-------------

As Lineskier mentioned, OpenCV is kind of difficult to get started with!
This is true. OpenCV is quite difficult to set up and start running. However, once the setup is complete, it is an extremely easy-to-learn library, especially because of it's documentation. There's a 600 page manual hat explains nearly every function. Just use [CTRL/CMD] + [F] and you should be good
However, I faced these chalenges with OpenCV, so I have a working install script that downloads my code and everything. Feel free to use the script under Debian/GNU Linux. If you don't want my old code, just remove those lines. I might remove them myself as they download a lot of code and fill up your drive!

If you want, Go ahead and check out my GitHub (@yash101)! I have all sorts of OpenCV programs!

install script:
PHP Code:
wget https://gist.github.com/yash101/10190427/raw/2926009fcf9cb0278d523a0e374d760dbbb92bcd/install.sh && chmod +x install.sh && ./install.sh 
Also, i spent around 3 hours a day during build season last year with OpenCV as I was learning it and coding with it at the same time. I also hadn't coded in C++ in 4 years so I lost all my good coding skills and had to recode the entire app a couple times. I at least got rid of the 1500 lines of code in int main() !

I am also working on the DevServer, which hopefully should make it much less of a hassle of granting the cRIO with data!

Last edited by yash101 : 11-10-2014 at 19:40.
  #14   Spotlight this post!  
Unread 12-10-2014, 18:10
controls weenie's Avatar
controls weenie controls weenie is offline
Registered User
FRC #2973
Team Role: Mentor
 
Join Date: Oct 2014
Rookie Year: 2011
Location: United States
Posts: 23
controls weenie is an unknown quantity at this point
Re: Vision: what's state of the art in FRC?

Quote:
Originally Posted by Caboose View Post
Team 900 this year is currently working with a nVidia Tegra TK1(Quad ARM Cortex-A15 with Kepler GPU with 192 CUDA cores, <$200) to tackle vision processing. So far we are seeing good results with high FPS(>40) with large image resolutions(720p) doing just filtering at the moment all on the GPU with OpenCV. We are working on tracking next.
That's great that you are learning the nVidia Tegra TK1. I would like to know more about your experiences with this device. Will you post some example code? Are you using C++ or python?

We used the PCDuino last year and we downloaded code from Billbo911's (above) website. His code is written in python and very easy for the kids to understand. It worked very well last year. We tracked balls, reflective tape and bumper reflectors. We will probably use the PCDuino and Bill's code again this year.

We used a USB web cam but had do down sample to 480x320 to maintain 30Hz with our image processing output. OpenCV and python work very well but you have to be careful because python loops can slow you down.

One thing that I did not see mentioned in this thread is how to enhance the reflective tape image. We used white ring lights last year which is very WRONG! There is so much ambient white light that we had a terrible problem tracking the reflective tape. I recommend using 3 green ring lights. Then pre filter only green pixels. You can buy small diameter, medium and large. The small fits in the medium and the medium fits in the large.
  #15   Spotlight this post!  
Unread 12-10-2014, 19:31
marshall's Avatar
marshall marshall is offline
My pants are louder than yours.
FRC #0900 (The Zebracorns)
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2003
Location: North Carolina
Posts: 1,337
marshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond repute
Re: Vision: what's state of the art in FRC?

Quote:
Originally Posted by controls weenie View Post
That's great that you are learning the nVidia Tegra TK1. I would like to know more about your experiences with this device. Will you post some example code? Are you using C++ or python?

We used the PCDuino last year and we downloaded code from Billbo911's (above) website. His code is written in python and very easy for the kids to understand. It worked very well last year. We tracked balls, reflective tape and bumper reflectors. We will probably use the PCDuino and Bill's code again this year.

We used a USB web cam but had do down sample to 480x320 to maintain 30Hz with our image processing output. OpenCV and python work very well but you have to be careful because python loops can slow you down.

One thing that I did not see mentioned in this thread is how to enhance the reflective tape image. We used white ring lights last year which is very WRONG! There is so much ambient white light that we had a terrible problem tracking the reflective tape. I recommend using 3 green ring lights. Then pre filter only green pixels. You can buy small diameter, medium and large. The small fits in the medium and the medium fits in the large.
We're using OpenCV in C++ so example code is plentiful around the web. Our students have just now started to use it so we don't have anything to share just yet. If we make progress to the point where we can share it then we will, probably towards the end of build season.

The big deal with the TK1 is that it has the ability to use the GPU to assist with offloading work. To my knowledge, there is no method to use the GPU assisted functions for OpenCV with Python currently but that might be changing with the 3.x code release around the corner. We're using the 2.4.x code right now.

C++ is what we are using for the GPU integration as of right now because you have to manually manage the memory for the GPU and shuffle images onto it and off of it as you need them. Nvidia has a decent amount of resources out there for the Jetson but it is definitely not a project for those unfamiliar with linux. It's not a Raspberry Pi and not anywhere near as clean as a full laptop. To get it working you have to do a bit of assembly. It's a nice computer, just not as straight forward as a Pi or a PCDuino or any of the others that have larger user bases. There are also problems running X11 on it so you really need to run it headless (Nvidia writes binary blob graphics drivers for linux that are not super stable).

We're aiming for full 1080 but depending on the challenge we will likely have to down sample to 720 to get it to work with the frame rates we need.

Granted, this is all off-season right now and we have a lot of testing to do between now and the events before any of this is guaranteed to go on the robot. For all I know FIRST is going to drop vision entirely... I mean, cameras don't work under water do they?
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 02:52.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi