[KJaget]

Another 900 mentor here (snow's coming so no one cares about work@work today).

Originally Posted by Jared Russell

Cool! This is AFAIK the first working NN implemented on an FRC robot.

For various definitions of working, but yeah we're really excited about it. Glad other people appreciate it.

Have you tried comparing your NN approach to a more traditional model-based vision approach?

A little bit, but this was one of those things where we thought it might be interesting to play with. The initial experiments worked way better than expected so we ran with it rather than prototyping a bunch of approaches.

With a single camera I'd suggest trying the Hough circle transform on an intensity or edge image. If you assume that balls are sitting on the ground plane and your camera is at a fixed height and angle, you can constrain the range of radii that you need to consider.

With a stereo/depth camera rig you can do even better; estimate and remove points near the floor plane, and then look only at the points that remain and cluster into spheres (using a 3D template matching algorithm or Hough sphere transform, only now scale is entirely fixed).

Marshall - remember "robot maintains fixed height and angle throughout the match" for one of our design goals this time around. I know that takes all the fun out if it

More seriously, we do incorporate some of the scale-limiting ideas you mention to speed up our code. It is still CNNs doing the bulk of the work, though.

(C)NNs are awesome technology and are quickly become ubiquitous in computer vision (and if only for this reason alone, they are a worthwhile learning exercise for your team!)

We kinda knew this was ambitious, potentially to the point of [hopefully spectacular] failure. Which kinda sums up what the team does every year.

I like to think the students who worked on the various parts got a lot out of it. Plus it works, so bonus.

They excel at solving hard detection and classification problems where humans don't have good intuition about how to specify useful visual features and the relationships between them. But for tracking a roughly monochromatic sphere on a level surface, I'd wager that human intuition is pretty good.

We haven't spent a ton of time on comparable approaches. 2015 we had a cascade classifier working tracking recycling bins - but that's a similar amount of work as the CNN approach but with more drawbacks. Those LBP cascades were quicker at run time, though.

We spent a bit of time with meanshift and camshift and didn't have much luck.

Some of the non-intuitive things we found about the problem :

1. the balls are reflective, so they pick up the color of field lighting pretty easily. People can easily determine that they're gray despite this. Computers have a tougher time.
2. they are sorta monochromatic, except for the random white markings on them depending on the ball's rotation plus what I mentioned above. Plus shadows and whatever. Typical computer vision problems.
3. There's lots of gray stuff on the field in addition to the balls.

None of these are insurmountable. But we only had so many people to throw at the problem, which goes back to the point that the CNNs worked way better than expected pretty quickly.