Vision Summer Project

[s5511]

Our team is going to try to develop a vision solution over the summer. We currently have an Nvidia Jetson TK1, Rasberry Pi, and Nexus 5 all available for us to use for this purpose. We also have LabVIEW robot code and a turret with an SRX encoder on our robot that would use the vision programming to aim at the high goal.

Which method of vision processing would you all recommend for us to try out during the summer? What are the disadvantages/advantages to the methods you all have used this past season?

Thanks!

[marshall]

Originally Posted by s5511

Our team is going to try to develop a vision solution over the summer. We currently have an Nvidia Jetson TK1, Rasberry Pi, and Nexus 5 all available for us to use for this purpose. We also have LabVIEW robot code and a turret with an SRX encoder on our robot that would use the vision programming to aim at the high goal.

Which method of vision processing would you all recommend for us to try out during the summer? What are the disadvantages/advantages to the methods you all have used this past season?

Thanks!

I would suggest you use the resources you have available at the Team 900 Home for Displaced Robotics Teams. We're happy to help.

The specific solution doesn't matter until you set some goals first for what you are aiming to do. No, those weren't puns. You need to set more parameters such as "we want to be able to aim/shoot/fire in X seconds with Y accuracy". Then you begin to build a solution around that.

[Hitchhiker 42]

I have met success with doing processing on the dashboard using the NI Vision Assistant Programming. Certainly, this isn't the best, but IMO it's a great starting step before going on to optimize everything more.

[euhlmann]

Originally Posted by marshall

You need to set more parameters such as "we want to be able to aim/shoot/fire in X seconds with Y accuracy". Then you begin to build a solution around that.

I disagree. You don't build a solution around shooting in 5 seconds by learning that you can aim in 2 seconds and then sleeping the other 3. You just shoot as fast as possible and as accurately as possible. It's hard to predict what these levels will be until you try it.

We built vision this year around NI vision and vision assistant. Vision assistant (and grip as well) are great places to start because you can visualize processing steps. The disadvantage of NI vision is its lack of documentation. We had issues with overlays, scaling, and undocumented error codes that we still haven't figured out.

OpenCV may be a bit more complicated, but it's got a large community around it that can help you if you encounter problems.

Good luck developing vision!

[RobotMonkey]

I would use your jetson along with python and opencv. You could get the bulk of it done in a couple days, even if you don't much about it. Make sure you turn off auto white balance on the camera, and set the exposure low to avoid washed out images. Superbrightleds.com sells some appropriate 12v green led ring lights.

The actual details of the implementation can be pretty much taken right from this ball tracking tutorial. It does most all of the same things you'll want to do.
http://www.pyimagesearch.com/2015/09...g-with-opencv/

HSV convert, threshold on bright green, maybe a dilate or two, find contours, pick the largest contour, find bounding box, find offset from the center of the bounding box to some aim point, send offset to roborio for aiming correction. UDP sockets preferred, or network tables if you have to.

Using the above tutorial and OpenCV documentation, I had something which could track the tape in about a day. That is coming from somebody with no python or opencv background too.

[vScourge]

Originally Posted by RobotMonkey

I would use your jetson along with python and opencv.

Is there a build of OpenCV that fully utilizes the Jetson's GPU? If there still isn't, I'm not sure why a Jetson would be preferable over the smaller, easier-to-power RPi2.

I would be glad to be mistaken, however, since we have a Jetson TK1 collecting dust.

[AlexanderTheOK]

Originally Posted by vScourge

Is there a build of OpenCV that fully utilizes the Jetson's GPU? If there still isn't, I'm not sure why a Jetson would be preferable over the smaller, easier-to-power RPi2.

I would be glad to be mistaken, however, since we have a Jetson TK1 collecting dust.

While I believe there's an opencv package called OpenCV4Tegra, the whole point of using a CUDA card on the jetson was that many vision processing libraries (especially opencv) already have cuda implementations.

[tomy]

Originally Posted by s5511

Our team is going to try to develop a vision solution over the summer. We currently have an Nvidia Jetson TK1, Rasberry Pi, and Nexus 5 all available for us to use for this purpose. We also have LabVIEW robot code and a turret with an SRX encoder on our robot that would use the vision programming to aim at the high goal.

Which method of vision processing would you all recommend for us to try out during the summer? What are the disadvantages/advantages to the methods you all have used this past season?

Thanks!

I made this post a while ago there is a lot of useful information there.

I found it easiest to do it in python with OpenCV. A few key links are:

Team 1648 Python Vision Code - That gave me a good guide line. Lines 150 - 193

Installing python and openCV on raspberry

Open Cv Tutorials

If you want the code I am using just PM me.

[Ben Wolsieffer]

Originally Posted by vScourge

Is there a build of OpenCV that fully utilizes the Jetson's GPU? If there still isn't, I'm not sure why a Jetson would be preferable over the smaller, easier-to-power RPi2.

I would be glad to be mistaken, however, since we have a Jetson TK1 collecting dust.

I'm not sure what you mean by "fully utilizes," but if you install the CUDA libraries on the Jetson and then compile OpenCV with CUDA support enabled, you can write a vision program that uses the GPU. AFAIK, there are no wrappers available for the GPU functions in Python, so that part would have to be written in C++.

I used this approach this year, writing most of our vision code in Java with a small JNI library in C++ doing some processing on the GPU. This significantly helped with performance, boosting our framerate from around 20 FPS to 30 FPS (it would have gone higher but was limited by the camera).

There is also the OpenCV4Tegra library, which includes some CPU optimizations for the Jetson (NEON, SIMD, multithreading), and I believe it also includes CUDA support.

Also, powering the Jetson TK1 (I've heard the TX1 is more sensitive) is not very hard. We used this regulator and have had no problems with it all season.

[Greg McKaskle]

If anyone has questions or issues with NI Vision or Vision Assistant, please post about it. There is quite a bit of documentation about the functions for all languages and conceptual as well, but you have to open up the chm files.

Also, one of the things I'll be messing with is the OpenCV utilities that they recently released. It allows for image translation so that you can use both NIVision and OpenCV in the same program -- but I think it only has a few functions wrapped and it'll be fun to see which ones are helpful in FRC situations.

Greg McKaskle