paper: ZebraVision 3.0 – Team 900

#1

Thread created automatically to discuss a document in CD-Media.

ZebraVision 3.0 – Team 900
by: ForeverAlon

Zebravision 3.0 is a culmination of libraries and several applications that are designed to use a Logitech C920 camera mounted to a robot to detect recycling bins in the 2015 FRC game: Recycle Rush. It is designed to run in real time using the OpenCV library onboard a NVIDIA Jetson TK1 DevKit mounted to the robot.

Zebravision 3.0 represents the culmination of many hours of work and research by the programming students for FRC Team 900, also known as the Zebracorns. It encompasses a library and several applications that are designed to use a Logitech C920 camera mounted to a robot to detect recycling bins in the 2015 FRC game: Recycle Rush. It is designed to run in real time using the OpenCV library onboard a NVIDIA Jetson TK1 DevKit mounted to the robot. The detection is done using the Cascade Classification method and Local Binary Patterns.

White Paper.pdf (902 KB)

#2

Zebravision 3.0 is Team 900’s 2015 initiative to take robot vision in FRC farther. In the 2015 season we successfully integrated cascade classification using feature detection as well as an automated tracking and navigation system. This paper details what we did and how we did it, as well as offering a tutorial so that other teams can use this application. If you have any questions please post here and someone who worked on the paper will respond.

#3

What cost function did you use in your classifier?

#4

It seems that you didn’t fully utilize the classifier. You classify a bin for instance, then compute on that. Extremely inefficient considering you could simply use a CNN and a SVM to compute anything you want about the object, including distance and rotation to it.

#5

These are the parameters we used for a typical run : https://github.com/FRC900/2015VisionCode/blob/master/cascade_training/classifier_bin_14/params.xml

Not sure the opencv_traincascade code exposes the option you’re asking about, so if it isn’t in there it’ll be hard-coded in the OpenCV source.

#6

I figured you guys did that, just wanted to make sure though.

Will you be releasing an analysis of your data? Not your training sets, but rather a statistical analysis of the classifier’s output.

#7

Wow, great job! Can’t wait to see what you guys do next!

#8

Man… you guys don’t ask for much do you?

Seriously though… we might but if we do then it will take us some time to get it done. We’ll do our best to push students towards publishing whatever we can though. It’s the second paper we’ve published on vision and I think it came out pretty well considering.

#9

Could you publish a white paper about that? I’d like to understand more about it. :smiley:

#10

Thanks! Neither can we. We’ve got some plans we’re working on though. Something about depth perception and neural networks last I heard.

#11

I do have one more request, could you post the raw data that you analyze?

#12

Will you be releasing an analysis of your data? Not your training sets, but rather a statistical analysis of the classifier’s output.

I do have one more request, could you post the raw data that you analyze?

This is something I would be willing to work on in the next few weeks. However I’m not sure exactly what you are asking. My first thought was that you wanted data about how much the classifier improves after each stage or iteration. This data could be useful but it’s also easy to predict: the classifier removes 50% of the negatives each stage so the improvement should be approximated by an exponential decay function. I’m not sure what other analysis type you’re looking for.

#13

For starters, when nothing is moving, how much do your output variables change? How much noise does your output data have? Can said noise be classified as Gaussian? What is the exact relationship between resolution and frame rate? How much precision do you lose / gain with different resolutions?

#14

We didn’t do a rigorous analysis of these questions so we don’t have specific answers here. I’m sure you could easily hack up our code to generate this data - let us know if you want pointers on where to start.

#15

I am struggling to find the time for this inquiry. Here is a question you may be able to answer for me: How often did you get false positives? False negatives?

I’ll eventually find the time to compile all the data of the vision programs in FRC the past few years: 341’s, 1706’s and yours, and do an analysis on each one. But that might be tricky considering I have zero of the materials they were all designed for.

Here is what @bernini (if we all start to do this, eventually chief delphi will add the feature, one can hope) was talking about with CNN (convolutional neural network) and SVM (support vector machine): http://yann.lecun.com/exdb/publis/pdf/huang-lecun-06.pdf

Your implementation of the same algorithm for FRC would yield better results due to the smaller scale of the network and SVM, I would suspect it to be 100 percent accurate in detecting with so few classes to classify something into (ball, robot, goal, etc…).

If you do go the neural network route, I highly suggest you have everyone involved in it watch Andrew Ng’s class on machine learning on coursera. I find it to be the best introduction to the topic. Fortunately, there is an amazing tool at your disposal for deep learning with (convolutional) neural networks: caffe. It is not as user friendly as it could be, but it is an extremely powerful tool. Something to keep you busy in the offseason (I would not leave the task of learning caffe plus getting a data set as well as designing a network during build season).

#16

I know I’m late to the party, but still: that’s awesome.

How did you guys get your project to run when the robot started up? Is that done with code on the RoboRIO or on the Jetson?

#17

Both. It takes code on the RoboRIO to communicate with the Jetson. The Jetson had a startup script that would run our code at boot and we had to make sure it booted up before we ran the robot. Wasn’t a huge problem to press a button and power it on before a match.

#18

Hi. Im from 6436 Pars Robotics.

NVIDIA has donated our team Jetson TX1. It’s our first year at FRC. We will ask you for help in a few ways.

1- How will you use GRIP and Java in TX1? We know little about using GRIP, but we do not know how to get X and Y values on TX1.

  1. How do we send the two values we receive via TX1 to RoboRIO? Can we send the values via GPIO? It is easier for us to send the values via GPIO.

This is our first year. We would be very happy to tell you how to do image processing through jetson. Thank you for your interest, we wish you success.
:slight_smile: :smiley: :rolleyes:

FRC! :stuck_out_tongue:

#19



Zebravision 3.0 is a culmination of libraries and several applications that are designed to use a Logitech C920 camera mounted to a robot to detect recycling bins in the 2015 FRC game: Recycle Rush. It is designed to run in real time using the OpenCV library onboard a NVIDIA Jetson TK1 DevKit mounted to the robot.

Zebravision 3.0 represents the culmination of many hours of work and research by the programming students for FRC Team 900, also known as the Zebracorns. It encompasses a library and several applications that are designed to use a Logitech C920 camera mounted to a robot to detect recycling bins in the 2015 FRC game: Recycle Rush. It is designed to run in real time using the OpenCV library onboard a NVIDIA Jetson TK1 DevKit mounted to the robot. The detection is done using the Cascade Classification method and Local Binary Patterns.

White Paper.pdf (902 KB)

Yo this dope. 10/10 sign m8

#20