Vision Systems results

I have been noticing a couple threads spawning about different teams’ vision systems. I was going to create a new thread about my software, until I realised that there were already threads aplenty about our vision systems
In this thread, please discuss how your vision systems functioned in the competition, how effective they were, how you did it, what you learned from it and what you want to learn to improve the system for next year.


To start off, my program didn’t have enough time to be bug tested and used on the robot, but I did a couple tests to find out that it was quite accurate to what I was seeing. We ended up using just basic hot goal detection using NI vision. My program was modular, and pressing a key would allow you to enable and disable features. Whilst it was meant to run off a USB camera for optimal performance, we didn’t have one of those on hand, so I ended up switching to the MJPEG stream from the AXIS camera. My programs are all written in C++ because I love performance. Also, my program was geared towards running not only in autonomous, but also in teleop period, to allow the robot to home itself, lock onto a target (and reposition if pushed around), and attempt to automate the shooting process
Now that I am done with the description of my vision software, let me point out some important things that I realised. My first version of DevVision2014 had no threading. The grabber had to wait for the processing to be completed. The problem came when the framerate dropped just below the capture rate. Images started piling up and the next thing I knew was that I was recieving 30 seconds of laaaggg! :ahh:. I fixed this by threading my next program versions. Now, the image is grabbed by one thread, but copied over and processed by another!

Through vision processing, I learned quite a bit. It, first of all, pushed me into self-teaching trigonometry so I can add more and more features to my program. I started off by grabbing an image from a camera and displaying it. I did this with VideoCapture because that was in the tutorial. I STRONGLY SUGGEST NOT TO USE VideoCapture! It is deprecated with CvCapture and is a bit harder to use anyways. I then learned about the different color spaces and why RGB (BGR) is horrible for complex color tracking. I settled with HSV because that was the color space I was introduced to be perfect for my needs. I learned how to then find the features in the image. This allowed me to actually locate the target instead of just filtering it out from everything else. However, a bunch of point data is useless naked. Next, I started pulling the relevant data from this point data, a vector containing vectors of cv::Points! I first learned how to calculate a regular bounding rectangle and using that for crazy cool things, like target location, target type and distance and how centered it is on the screen! Now, I have moved onto using minAreaRect instead, to give me even better accuracy, even if the camera is a bit tilted. I also learned that being able to send multiple images and tap into images during any stage of processing using imshow can make debugging and setup easier. It took me much less time to set up the thresholding in my program (5 mins) as compared to my mentor’s program (~1 hour) because I was able to tap into the threshold process and play with the variables accordingly. I also was able to use sliders so I never had to recompile the code for setup.

There are many things I want to learn for next year. I want to be able to take two cameras and use stereo vision to triangulate the position of the object relative to us. I also want to attempt to create some sort of image stabilisation algorithm to make the code automatically move the image ROI up and down to compensate for the terrain roughness. I also want to learn how to use other thresholds, like canny, to make my findContours even more efficient! It would be fun to see a 100% accuracy in my program in the near future. I also want to have a node-based system where there are multiple onboard computers, communicating all through a tunnel protocol I want to create. Basically, there will be a RasPi as a host, and all the ODROIDs are going to:
send the destination
send a request (purpose of the data)
data
requested action

All these requests will be concatenated with a specific magic number sequence.
This way, only one ethernet port would be required, optional because that would be for DS communication

I have a couple repositories that may interest you if you want OpenCV.
https://github.com/yash101/ai
https://bitbucket.org/devyash_lodha/opencv-programs
https://bitbucket.org/devyash_lodha/robotai2014/overview

For those who are having problems in installing OpenCV, I’ve written a script to automate the apt-get install.
Script Here

Congrats on your accomplishments. If you are up for additional challenges and would like to work on it in the offseason, I’d suggest looking at previous games and their vision elements and game objects. I believe the example code and guides for building them are still shipped in the 2014 SW, and if they aren’t, you can probably find them online.

It may also be a good time to read a good or some papers.

Greg McKaskle

Greg,
As far as I can tell, most of the prior years vision code is still included back to 2011. There are some really good examples in there. But, the one I really miss is the one from Breakaway. It integrated the use of a Gyro. If someone can post it, I would really appreciate it!!

However, I am using OpenCV, and it has so many features and capabilities, I am not eay to downgrade. At least I can run my code under Ubuntu on an onboard SBC. I also remembered that I want to try taking pictures at multiple exposures and putting them together to creeate a perfect picture for processing!

We would not have been able to get the code to work if another team had not let us take a look at their code. Which was based on the use of the ds for processing. Once we did get it working and calibrated to the competition field we tested it on the practice field and it worked perfectly every time. Once we got to the actual competition field though our bot swerved left or right randomly. (jaguar was setup incorrectly) so tape was often left out of view of camera.

That is the exact scenario here. I would like to accredit Team 1706 for answering every question I had. That really saved me hours from reading the RefMan.
The thing that really helped me is that they shared all their previous years worth of code with me, so I as able to understand how HSV works, how findContours works, and even how the OpenNI-OpenKinect-OpenCV implementation works. I guess I should have taken some of the advice from some of the CDers, who were telling me that the learning curve is quite large.
Thankfully, that doesn’t scare me. I literally used OpenCV in labs, where the teacher didn’t have access to expensive equipment. Basic color tracking did the job though.

Some advanced programming knowledge, like OpenCV can be very handy in your toolbox because you never know when it will be useful, even for adding features to your favorite video editor!

My latest stable code that works runs in the DS. However, I switched to OpenCV Linux so I can run embedded without shelling hundreds of dollars for an x64 system.

Next year, I want to learn even more hardware-level programming and develop my own communication protocol to rapidly transfer a lot of data between multiple wired computers.

However, I am using OpenCV, and it has so many features and capabilities, I am not

I can’t tell for sure whether this was a response to my suggestion. To clarify, I wasn’t suggesting anything about what tools you use. I was suggesting that you can look at the previous few games to get some challenges to tackle with whatever tools you choose. The field, game objects, and game objectives are likely to evolve and change, but if you can handle what they have done in the past, that is good prep for the future, IMO. So if you have photos or videos of those games, you can write algorithmic code to measure and count stuff in those photos. You can also get your hands on a moon rock, a tube, an orange basketball, a frisbee. You can place them on the carpet, on some aluminum diamond-plate, on a robot, and see if you can still identify them, determine how far away they are, etc.

Lots of challenges that once were and may once again be relevant.
Greg Mckaskle

I know. I wasn’t really trying to answer your question there. Sorry for the confusion. However, I am trying to write vision code for the previous years. It would help me learn even more tricks with OpenCV to increase speed and accuracy!

I want to design my own P2P networking protocol for next year that depends on 1-2 serial ports. My goal is to have multiple computers, each processing a specific portion of data. For this, I need some help! I want to have serial-UART connections from each computer to a hub, a RasPi. When a computer needs data, it sends the RasPi a request, which contains the data wanted, and the destination. The RasPi then forwards that message to the other computer. That computer will then send back the most recent data. This will provide the data transport layer. Log files can be hard to use if they are separate ones on each machine. I am thinking about adding a feather, where the request is a log. It supplies the computer name, and then the data is logged on a RasPi. Then, the data to be logged is sent. The RasPi then records it in a file, line-by-line!. What do you guys think? Is this practical? This is so max data can flow in a structured manner with little lag and with minimal network impact.

Also, logging large files like images and videos can be tough on a 3MBit/s connection. I am thinking about having a second protocol, that mimics FTP, for the programs to send all the data to the central drive. Also, instead of having 6 RSH sessions open, I am thinking about forwarding all that data to the RasPi, to combine into a single session!