Getting Started with Vision Processing

[Lireal]

In previous years, my team has always struggled to simply put a robot out on the match floor, but this year we tripled in size and would like to start learning more advanced techniques, such as vision processing.

The previous threads that I have looked at seem pretty dated, and there aren't many resources on the FIRST website about how to start using vision from scratch. So, here are the questions I have:

1. What additional devices or software is needed to process images?
2. Where are good resources to learn how to do vision processing without any knowledge of it?

My team programs in Java, but we are willing to learn other languages if necessary(as long as it doesn't affect how we code the rest of the robot).

Any help that you can give is really appreciated .

[marshall]

Originally Posted by Lireal

In previous years, my team has always struggled to simply put a robot out on the match floor, but this year we tripled in size and would like to start learning more advanced techniques, such as vision processing.

The previous threads that I have looked at seem pretty dated, and there aren't many resources on the FIRST website about how to start using vision from scratch. So, here are the questions I have:

1. What additional devices or software is needed to process images?
2. Where are good resources to learn how to do vision processing without any knowledge of it?

My team programs in Java, but we are willing to learn other languages if necessary(as long as it doesn't affect how we code the rest of the robot).

Any help that you can give is really appreciated .

If you've never done it before then I would start with the Screensteps site that FIRST provides:
http://wpilib.screenstepslive.com/s/...ifecam-hd-3000

I can't speak to Java but the LabVIEW vision examples are great. I suspect the Java examples are just as valuable.

Start with basic image filtering and shape detection. Once you've got that down, move on to more advanced processing with OpenCV. Some teams last year had luck running OpenCV on the RoboRIO. We went for the secondary processor setup.

If you're super cutting edge then the Jetson TX1 and a stereoscopic camera with neural network processing might be the way to go.

[Knufire]

In previous years, my team has always struggled to simply put a robot out on the match floor

If this is the case, I would strongly recommend not pursuing vision processing. Advanced techniques like this are things that already strong teams pursue to get even better; say a jump from the 80th percentile to the 90th percentile. If your team has barely fielded a robot in the past, and you're looking to make your team more competitive, I would recommend some of the following:

• Get some licenses of Solidworks or Inventor (both of which are free to FRC teams) and start learning how to CAD. Look at CAD of other teams' robots, there's tons of examples in CD-Media and http://frcdesigns.com/
• Explore writing basic PID loops and learn how to use them to control your robots more accurately, both in teleop and autonomous
• Go through all the products on AndyMark and VEXPro's websites. Start thinking about how to use more off-the-shelf products on your robtos; they're a great way to maintain your robot quality while getting it done faster.
• Spend the next month trying to find a practice area. Without a doubt, the one thing you can do to make your team perform better on the field is getting as much drive practice in as possible. If you can't build a duplicate robot to practice during the season, make your drivetrain first and throw some weight on it and have your drivers start messing around with it. Aim to finish your robot earlier (Week 5) and spend the last week and a half of build practicing and tuning a basic autonomous mode.

[Greg McKaskle]

If your team is large enough to take care of the basics, then vision can be a good sub team project. The Screensteps pages seemed to only discuss setting up the cameras of various types. Good stuff, but not really vision processing. If you are going to spend time on this, I'd search for FRC vision white papers. There are ones from teams and for various games. This one https://decibel.ni.com/content/docs/DOC-20173 is the one I'd suggest for a basic paper that is useful for many FRC target types.

Greg McKaskle

[SamCarlberg]

There's a program being worked on by a few WPI seniors for their capstone project to make it much easier to work on computer vision problems for FRC. They released the first alpha yesterday here

[SoftwareBug2.0]

I'm also not sure that you will find vision processing useful yet. If someone had given you a sensor last year that told you exactly where every tote on the field was relative to your robot how useful would it have been to you? If you're just emerging from the stage where it was a struggle to get a robot on the floor I suspect the answer would be "not very".

However, if you do have a bunch more people it might be possible to develop both your computer vision skills and the capabilities needed to take advantage of them at the same time. I'd have some people study controls and others study search algorithms. That way when you find out you want to go someplace 5 feet ahead and 3 to the right you can both plan a path to get there and a way to carry it out.

[otherguy]

Our team put together a pretty comprehensive guide, using the beagle bone black as a coprocessor.

Instructions and example code is available here: https://github.com/Team2168/2168_Vis...ster/README.md

[marshall]

Originally Posted by otherguy

Our team put together a pretty comprehensive guide, using the beagle bone black as a coprocessor.

Instructions and example code is available here: https://github.com/Team2168/2168_Vis...ster/README.md

Very cool! Thanks for posting this.