We Want Pictures and Videos of Your Boulders
 

Hi Teams,

We here at Team 900 have a favor to ask. Can you show us your balls?

We've been hard at work training our vision system to recognize our own balls and I can safely report that we're beginning to get a feel for them. That being said, we're always looking to the future and we will need to be able to spot balls other than our own.

We would like pictures of your balls resting on various surfaces. We would like to see your balls on your field elements. Videos of your balls moving around would be helpful too. Some teams have been uploading teaser videos of their balls flying in the air and those are great.

If you have access to a full practice field and can get footage of balls being played with on it from a robot's viewing angle then that would be awesome!

If you have damaged your balls in the course of your prototyping then we would love pictures and video of that too. We would of course like to see all teams treating their balls with care but we understand that incidental damage to balls can occur during rough matches and would like to prepare for that too.

Here is a picture of me holding up our balls so you can understand what we are after:


(Yes, we really would like pictures and videos of the game pieces/boulders/balls.)
(Yes, I will report you to the admins if you take the opportunity of my carefully worded post to send me inappropriate pictures, get your mind out of the gutter!)

Re: We Want Pictures and Videos of Your Balls
 

You do realize that you could have avoided all possible retaliation in terms of inapropriate responses by calling them by their PROPER name, Boulders, right????


(G39 avoided the fate of the identical rule from 2014 by just that method...)

Re: We Want Pictures and Videos of Your Balls
 

uhm

Re: We Want Pictures and Videos of Your Balls
 

Our balls look super weird, should I call someone?

Re: We Want Pictures and Videos of Your Balls
 

I love that the request was from team 900.

Re: We Want Pictures and Videos of Your Balls
 

I ran a search and replace on my original post and it resulted in just as much hilarity among our students...

In all seriousness, we do need pictures and videos. For those willing to take a moment and create them, we'd be grateful.

Re: We Want Pictures and Videos of Your Balls
 

Probably see a doctor about that. I know a guy.

Re: We Want Pictures and Videos of Your Balls
 

I hear that they can look that way when exposed to a carcinogenic environment, even if only for short periods of time. .......something about Regolith......

Re: We Want Pictures and Videos of Your Balls
 

I presume that you guys will be using some learning algorithm to find balls in your image. Congrats if you get it to work, that is no easy task. While I don't have a dataset to give you of my own, you might be interested in taking your learning algorithm a step further and computing the "DeepPose" of your balls from images. Remove the middle man if you will. Your algorithm last year ran at a speed that left some to be desired. The jetson boards can handle CNN models like no other (thanks to their cuda cores), with inference that is, and not necessarily with training. If you have a CNN with, say, 3 convolution layers, it would run at well over 30 fps.

Just a thought you might want to consider.

Re: We Want Pictures and Videos of Your Balls
 

We'll try to get some pictures up tonight! We got some damaged ones actually.
Shall I PM them to you? Or through mail?

In week 6 we have a scrimmage going on with some EU teams and we will be playing (hopefully) some matches. Let me know which angles u want shots from and we could send it your way.
Our field is from wood fyi.

Re: We Want Pictures and Videos of Your Balls
 

It would be nice if they were made to the public. Maybe upload them to some cloud service such as Google Drive and share the link.

Re: We Want Pictures and Videos of Your Balls
 

During a practice match in 2012 our drives team snuck one out into the field and threw it on the field during the endgame. They should have game them a red card to carry for the rest of practice day.

Re: We Want Pictures and Videos of Your Balls
 

Indeed. Drive or Imgur or similar please.

Re: We Want Pictures and Videos of Your Balls
 

Would it be possible for other teams to use the pictures and videos you receive so we can also use deep learning and neural networks? As Ron said we have some damaged balls and more new ones on the way, as soon as we have those we will upload more pictures.

Re: We Want Pictures and Videos of Your Balls
 

We are happy to see them posted publicly and would invite others to use them for their own purposes.

Re: We Want Pictures and Videos of Your Balls
 

why you gotta let the dutch get domestic secrets re: our balls tho

Re: We Want Pictures and Videos of Your Balls
 

Have you ever had a stroopwafel? I'd give my right... arm... for those.

Re: We Want Pictures and Videos of Your Balls
 

In all seriousness, can people post pictures of how the game pieces are dealing with wear and tear for those of us who are thinking about using mechanisms designed to squish the ball when scoring.

Re: We Want Pictures and Videos of Your Balls
 

We painted our balls.

This is actually surprisingly significant and here's why:

For a stencil, I used an old aluminum plate with our number milled into it. I took the first boulder that we had been testing with for the past couple weeks, and pressed the stencil against the foam to compress it so all the edges were in contact with the boulder surface. This is to keep the edges crisp. I was able to do this by myself with my left hand pressing the stencil down and my right spray paining. I took the next boulder, which had never been used, and found that I could not press the stencil hard enough to compress the boulder so that all edges touched the boulder surface. I had to ask a student to press the stencil down while I sprayed, and eventually after getting a few boulders painted, he was showing signs of fatigue and his arms were shaking a bit.

The moral: The new balls are much, much harder to compress than ones that have been used over and over.

Re: We Want Pictures and Videos of Your Balls
 

You have no idea how excited I am that someone posted a picture finally! Thank you!

Re: We Want Pictures and Videos of Your Balls
 

Re: We Want Pictures and Videos of Your Balls
 

We took pictures of our boulders next to a variety of the field defenses. I hope this helps. Look forward to seeing your robot at one of the NC District events this year.

https://docs.google.com/document/d/1...it?usp=sharing

Re: We Want Pictures and Videos of Your Balls
 

THANK YOU!!!!!

Re: We Want Pictures and Videos of Your Balls
 

Get your rocks off (my screen).

Re: We Want Pictures and Videos of Your Balls
 

No need for arms if I can use the all the pictures of boulders you guys receive I will personally make sure you are going to get stroopwafels.
PS: our pictures are coming soon!

Re: We Want Pictures and Videos of Your Boulders
 

Ours are perfect.

https://www.youtube.com/watch?v=xT9pygVQwgk

Re: We Want Pictures and Videos of Your Boulders
 

I can't believe nobody has quoted AC/DC yet...

Re: We Want Pictures and Videos of Your Boulders
 

Yeah... I'm not saying our teaser will have a theme song but our teaser will have a theme song... ;)

Re: We Want Pictures and Videos of Your Boulders
 

At least take me out to dinner first.

Re: We Want Pictures and Videos of Your Boulders
 

We've got some videos of our shooter prototypes:

https://www.youtube.com/watch?v=uTn8bSvJ0vw

https://www.youtube.com/watch?v=BzS2Mmha0HI

Re: We Want Pictures and Videos of Your Boulders
 

Here are our pictures (be warned there are a lot of them). Enjoy!
https://drive.google.com/open?id=0B6...DZmVVlrSWRUeWs

Re: We Want Pictures and Videos of Your Boulders
 

Best pictures yet!!!

Re: We Want Pictures and Videos of Your Boulders
 

We actually don't consider it "a lot" until you get into the 7 digits range ;) - but we do appreciate these and will check them out.

Re: We Want Pictures and Videos of Your Boulders
 

Guess we need to take some more then. Be right back.

Re: We Want Pictures and Videos of Your Boulders
 

I wanted to let you know that AndyMark might have sent you one of the rare invisible boulders. Our vision code seems to be pretty good at finding where these balls are hiding. See the one labelled "7" in the upper right corner of this image. You'll want to do your best not to lose it - I'm sure they'll be valuable some day.

Thanks again for the pics, they have honestly been quite helpful for testing.

Attachment 20019

Re: We Want Pictures and Videos of Your Boulders
 

Thank you we are extremely proud of that boulder, gives us a little more sensation during practice :cool:

Re: We Want Pictures and Videos of Your Boulders
 

Just in case anyone thought we were joking about this. It's finally been merged into our code and we are aiming to put this on a robot before too long:



The white paper should be coming soon too. :)

I'm super proud of my students who have been doing this research. They are some of the smartest people I know.

Re: We Want Pictures and Videos of Your Boulders
 

Can't wait to see it at the Rumble in the Roads.

Re: We Want Pictures and Videos of Your Boulders
 

We only have 2 of them- those things are expensive (though this is FRC we're talking about, so maybe not comparatively). One is pretty new and the other got caught up on a nail in a wooden prototype so it's all ripped. Oh, and also we played soccer outside with the ripped boulder.

Re: We Want Pictures and Videos of Your Boulders
 

Alright... it took us a year to get this working but it finally works:

https://youtu.be/OT5FHyLBjCg
https://youtu.be/eRMo1_hJNa0

That's a video of our robot from 2016 using a neural network based vision system to detect and pick up a game piece by itself (NO DRIVER INTERACTION) from the 2016 game FIRST Stronghold.

And contrary to what some teams are saying about the Nvidia TX1... we love it and we are using it and a Stereolabs ZED camera to detect the boulder and then send data back via ZeroMQ to a a National Instruments RoboRIO running LabVIEW along with a KauaiLabs NavX MXP IMU board to orient the robot, drive towards, and grab the ball.

So to all of you who thought we were joking when we were asking for pictures. We weren't. This is what we've done with them.

Also, check out this awesome custom case for our TX1:


Files for reproduction of the case are available here:
https://workbench.grabcad.com/workbe...vJsPGiurrAmGW7

EDIT: Code is available on our Github... it's been out there for a while though.

Re: We Want Pictures and Videos of Your Boulders
 

Wow! I'm looking forward to reading through the code. Is this something you would consider this season if the opportunity arises? How reliable is it? Can it follow moving game pieces (I noticed you waited for it to stop rolling in the video)? What are the benefits of this over using two infrared sensors, putting them on either side of the robot, and using the difference in distances to align the robot.*


*FRC272 used this method in 2010 to allign with the soccer balls. They wrote about their experience with this method in a powerpoint they used for a seminar last year here. The video isn't working, but I know this solution worked well for them.

Re: We Want Pictures and Videos of Your Boulders
 

Yes.

Ehh... not as much as we would like but it finally works on the robot and that's impressive.

Yes. https://www.youtube.com/watch?v=v9gof9Rafks

In theory, this has the ability to be more reliable and the ability to select different arbitrary targets. Unlike IR, it's not just looking for the presence of something... it's actually looking for the ball (the shape/color/ and whatever else the neural network has decided to focus on).

With all that said. If you're thinking about doing this yourself then understand that there is a ton of work that has gone into it and a ton more will go into it before it is reliable. This isn't something a potential championship winning team should be relying on to establish dominance for a match. It's untested, unreliable, and fragile.... it's still really frickin' cool though.

Re: We Want Pictures and Videos of Your Boulders
 

Cool! This is AFAIK the first working NN implemented on an FRC robot.

Have you tried comparing your NN approach to a more traditional model-based vision approach?

With a single camera I'd suggest trying the Hough circle transform on an intensity or edge image. If you assume that balls are sitting on the ground plane and your camera is at a fixed height and angle, you can constrain the range of radii that you need to consider.

With a stereo/depth camera rig you can do even better; estimate and remove points near the floor plane, and then look only at the points that remain and cluster into spheres (using a 3D template matching algorithm or Hough sphere transform, only now scale is entirely fixed).

(C)NNs are awesome technology and are quickly become ubiquitous in computer vision (and if only for this reason alone, they are a worthwhile learning exercise for your team!) They excel at solving hard detection and classification problems where humans don't have good intuition about how to specify useful visual features and the relationships between them. But for tracking a roughly monochromatic sphere on a level surface, I'd wager that human intuition is pretty good.

Re: We Want Pictures and Videos of Your Boulders
 

Another 900 mentor here (snow's coming so no one cares about work@work today).

For various definitions of working, but yeah we're really excited about it. Glad other people appreciate it.

A little bit, but this was one of those things where we thought it might be interesting to play with. The initial experiments worked way better than expected so we ran with it rather than prototyping a bunch of approaches.

Marshall - remember "robot maintains fixed height and angle throughout the match" for one of our design goals this time around. I know that takes all the fun out if it :)

More seriously, we do incorporate some of the scale-limiting ideas you mention to speed up our code. It is still CNNs doing the bulk of the work, though.

We kinda knew this was ambitious, potentially to the point of [hopefully spectacular] failure. Which kinda sums up what the team does every year.

I like to think the students who worked on the various parts got a lot out of it. Plus it works, so bonus.

We haven't spent a ton of time on comparable approaches. 2015 we had a cascade classifier working tracking recycling bins - but that's a similar amount of work as the CNN approach but with more drawbacks. Those LBP cascades were quicker at run time, though.

We spent a bit of time with meanshift and camshift and didn't have much luck.

Some of the non-intuitive things we found about the problem :

1. the balls are reflective, so they pick up the color of field lighting pretty easily. People can easily determine that they're gray despite this. Computers have a tougher time.
2. they are sorta monochromatic, except for the random white markings on them depending on the ball's rotation plus what I mentioned above. Plus shadows and whatever. Typical computer vision problems.
3. There's lots of gray stuff on the field in addition to the balls.

None of these are insurmountable. But we only had so many people to throw at the problem, which goes back to the point that the CNNs worked way better than expected pretty quickly.