Galileo 2012 - Share your scouting data !

[diviney]

All:

One critical key to success in FIRST is the ability to collect, process, and analyze good scouting data. With the limited number of matches, and the random alliance selections, there are always some great disparities between the official FIRST rankings and the actual performance of the robots. This can go either way (ie: excellent teams / robots with poor rankings or vice-versa).

It takes a lot of effort to do this well, but we take scouting very seriously. I firmly believe that we were able to win a past regional competition in large part because our scouting identified a "diamond in the rough" (an under-rated team) that fit into our alliance perfectly.

How good is your scouting system? How would you really ever know how well your scouting system did? Did you ever wonder how your team looked to the other teams in your division? How close was your team to being chosen to enter the eliminations?

I would like to share our scouting data for Galileo division so you can see how we ranked all the teams with our scouting. My hope is that in sharing this, it will inspire others to do the same. It will be interesting to see how consistently our various scouting systems come up with performance-based rankings for teams in the division !!

If you are interested / willing, please share your results with others too !

If teams from other divisions would like to participate, it may be good to start a separate thread for each division.

A brief explanation of our data -- We use an Excel spreadsheet. There are two tabs (Raw Data Entry, and Tabulated Results). The raw data is match by match as entered from the scouting sheets, and the Tabulated Results show our overall point calculations and shows the teams ranked from best performance to worse. For reference, I have also attached our scouting sheet so you can see how the data was collected.

Hopefully this can be a nice learning experience for all, and help to spread the word about the value added through good scouting.

Tom Diviney
Team 2377
C Company
Pasadena, MD

[R.C.]

We didn't scout at champs because we had very few kids there and we wanted to try something new. Which worked well for us.

But I'd like to thank you for this data. It honestly really nice to see it. As we always wonder how we stack up against other teams in our division!

Thanks!

-RC

[Alex.q]

I believe my team did not scout matches either simply because we knew that we were not going to make it to be an alliance captain or be picked. However, I would like to see how other teams go about collecting and compiling data. One thing that is harde for us is deciding who to pick after we have collected that data. This might not be the thread for that, but do you mainly go directly with who scored the most or do you try to incorporate other things into your scouting decisions?

[diviney]

Originally Posted by Alex.q

I believe my team did not scout matches either simply because we knew that we were not going to make it to be an alliance captain or be picked.

Don't short-change yourselves !!

Good scouting data can help you sell your team to those who are in a position to pick. If you are not an alliance captain but get picked in the first round, your data can be instrumental in helping to choose the third pick.

The numerical data is not the end-all, but it is an important part of the decision process. Other intangible factors include synergy and compatibility of the alliance robots and the personalities of the drivers / coaches...

[akoscielski3]

Yea your right. We were ranked 96th in Galileo. IMO we had bad luck for the qual matches, losing by only a few points usually, and then had only a few co-op points because of unfortunate events. Your ranking system ranked us 48th in the division.

Thanks for the data

[Kristian Calhoun]

Our match scouts had to leave the venue early on Friday, so we have records through match 121 for all the teams on Galileo. I've attached a screenshot from our scouting program that shows the cumulative number of points scored by each team and the average number of points scored by each team per match (the scores are adjusted such that a balance, whether on the coop or alliance bridge, is worth 10 points if it was successful).

Our scouting program can also sort all teams by the number of balls scored high/mid/low for both hybrid and teleop, as well as successful balances on the coop/alliance bridges and other criteria. Individual team reports can also be generated. If anyone is interested in any of these particulars, feel free to contact me via PM.

[JABot67]

Call me crazy, but I think the picking order on Galileo made little sense. Why did 25 slide to fourth selection? Why was 2054 a second round pick? 103 not picked?? They were awesome at shooting from the back of the key - a perfect supplement to those front-key shooters out there.

I am surprised to see that 3322 was ranked so high (11th place) on lists that only took number of points scored into account, as we had died after hybrid in two of our matches. Looking at 2377's data, we had the most accurate hybrid in the entire division. I was confident we'd get picked early, but I guess the top teams had other strategies in mind. Instead we got the thrill of getting picked last!

Can teams please at least have one person watching matches to figure out who to pick? You don't even need to collect hard data. Just watching enough matches and scrawling down notes will give you an idea of the best robots in a division. Add in a couple strategy discussions and you can go down to the field with a pick list that gives you a great chance of succeeding.

Speaking of lists, I left ours somewhere and don't have access to it. Jeff may have his copy though. I remember that 33, 25, 148, 2054, and 180 were in the top 5.

[4057programmer]

I hate to sound like this, but on your scoring sheet is says that team 4057 only stopped 2 shots from being made. We spent most of our time playing defense and were able to stop shots every match that we played in save for our first 2 matches when our arm was almost broken. I know how whiny that sounds, but I felt that people should know we in fact did more work than shows in the scouting sheet. Okay I'm done on my little soapbox, I just felt the need to say something.

[diviney]

Defense --- Yes,

That is a tough one. We tried to scout defense, but it is a lot more subjective. How do you know for sure that a defender blocked a shot vs. it would have missed anyway? I don't know if our "system" was lacking here, or our scouts. We had 6 scouts at all times (one for each robot). We ran in roughly 2 hour shifts. Scouting is not the most glamorous job, and it is tough for them to remain fully engaged. Scoring baskets on offense is pretty easy but when it comes to judging defense I think our scouts / system fell down.

We have weighting factors for each category, and maybe we did not weight defense as high as we should have. I have a feeling it's more a problem of the scouts not paying close attention and kind of "ignoring" defense because it required them to make judgement calls...

Anyone else have some thoughts on how to effectively gauge defense?

[IKE]

2377 had a lot more points associated with 1538 than we did (and apparently team 25 as well). They were really good, but we figured they would be bubble second round (which agrees with the 25 data) as opposed to an early first round looking at the 2377 data.

Our system is paper based. I will see if I can get it into some sort of sharable format.

[Richard Wallace]

Originally Posted by diviney

Defense --- Yes, ...

Anyone else have some thoughts on how to effectively gauge defense?

It is water over the dam by this point, but I think the more important gauge for defense is balls collected in the backcourt, rather than shots spoiled. The offense can't shoot balls they don't have.

Watch a few of 16's elimination matches and see if you agree. Another good team to watch was 4334 on Archimedes.

[Gary Dillard]

See my post #11 in this thread for a brief discussion on how SPAM used its scouting method (the paper discussed in the thread) to make its decisions. I'll see if I can get them to post their completed spreadsheet to compare their data with 2377's data.
It's important to note again that your team's win-loss record and ranking have no bearing on detailed scouting; if you are performing then you will get recognized. Equally as important in marketing yourself is to be able to explain problems you have had and what you have done to correct them throughout the competition. The fact that 16 had run OK after they changed out a radio was a really big deal to us.
One other thing that came up in our discussions was robot construction; to get through division and championship elimination matches your robot is going to take a beating. Some teams got moved down on the list because of observations that their robot wiring and construction looked shoddy and we were concerned that they wouldn't last the battle. Not such a big deal at regionals but a huge deal when you're looking to go all the way.

[diviney]

Originally Posted by IKE

2377 had a lot more points associated with 1538 than we did (and apparently team 25 as well). They were really good, but we figured they would be bubble second round (which agrees with the 25 data) as opposed to an early first round looking at the 2377 data.

Our system is paper based. I will see if I can get it into some sort of sharable format.

Note: Our data shows 1538 with 16 "balance points" (a single balance worth 1, and a double balance worth 2). That is more than any other team in the division. We applied a weighting factor of 4 to this which boosted them up some. Even without this however, they were very good.