Vision: what's state of the art in FRC?

[marshall]

Originally Posted by controls weenie

That's great that you are learning the nVidia Tegra TK1. I would like to know more about your experiences with this device. Will you post some example code? Are you using C++ or python?

We used the PCDuino last year and we downloaded code from Billbo911's (above) website. His code is written in python and very easy for the kids to understand. It worked very well last year. We tracked balls, reflective tape and bumper reflectors. We will probably use the PCDuino and Bill's code again this year.

We used a USB web cam but had do down sample to 480x320 to maintain 30Hz with our image processing output. OpenCV and python work very well but you have to be careful because python loops can slow you down.

One thing that I did not see mentioned in this thread is how to enhance the reflective tape image. We used white ring lights last year which is very WRONG! There is so much ambient white light that we had a terrible problem tracking the reflective tape. I recommend using 3 green ring lights. Then pre filter only green pixels. You can buy small diameter, medium and large. The small fits in the medium and the medium fits in the large.

We're using OpenCV in C++ so example code is plentiful around the web. Our students have just now started to use it so we don't have anything to share just yet. If we make progress to the point where we can share it then we will, probably towards the end of build season.

The big deal with the TK1 is that it has the ability to use the GPU to assist with offloading work. To my knowledge, there is no method to use the GPU assisted functions for OpenCV with Python currently but that might be changing with the 3.x code release around the corner. We're using the 2.4.x code right now.

C++ is what we are using for the GPU integration as of right now because you have to manually manage the memory for the GPU and shuffle images onto it and off of it as you need them. Nvidia has a decent amount of resources out there for the Jetson but it is definitely not a project for those unfamiliar with linux. It's not a Raspberry Pi and not anywhere near as clean as a full laptop. To get it working you have to do a bit of assembly. It's a nice computer, just not as straight forward as a Pi or a PCDuino or any of the others that have larger user bases. There are also problems running X11 on it so you really need to run it headless (Nvidia writes binary blob graphics drivers for linux that are not super stable).

We're aiming for full 1080 but depending on the challenge we will likely have to down sample to 720 to get it to work with the frame rates we need.

Granted, this is all off-season right now and we have a lot of testing to do between now and the events before any of this is guaranteed to go on the robot. For all I know FIRST is going to drop vision entirely... I mean, cameras don't work under water do they?