Complete scouting overhaul - where to start?

[ttldomination]

Phyrxes makes a good point about technology needs for an overhaul, and I will make a point about the determination needs.

Make sure that if you do this overhaul, everyone is on board. For the past two years, I've spent a lot of time and effort searching for scouting solutions, convincing the students to use this method, and then making sure the students have everything they need, only to see the blunder of one careless person cause the entire operation to go haywire.

This is a full time job, and the dedication required to do this at an all-star level is sometimes more than what a team can afford.

As for your questions.

(1) We've only recently started placing an effort in scouting. So we look for dependable veterans to pick up the scouting ticket. No one really steps forward to say that they're going to scout, and well, I think that's played a part in our short comings.

(2) There's a ton of useful software out there. My favorite has to be the Cheesy Poof system. It's a nice, comprehensive system which allows for the collection of quantitative and qualitative data, and it presents the data in a nice, manageable way.

Dr. Ed Law, goes by 'Ed Law' on the forums, also posts scouting spreadsheets of data for teams in the events. These spreadsheets are all numbers, but they give a person a good feeling about how things are moving.

- Sunny G.

[Basel A]

Originally Posted by ttldomination

Mr. Ed Law, goes by 'Ed Law' on the forums, also posts scouting spreadsheets of data for teams in the events. These spreadsheets are all numbers, but they give a person a good feeling about how things are moving.

- Sunny G.

Dr. Law, actually, and it's true, his often-updated spreadsheets are as good as you can get without actually watching any matches.

What Mr. Ross notes is very important. Scouting should not simply be gathering information, it should be gathering information for a defined purpose.

Purposes could be to decide match strategy, whether in qualifiers or eliminations, or to choose alliance partners. Once you know what you want from scouting, you can begin to discuss how you want to obtain that data.

Sometimes, categorising the robots as "defenders" and "scorers" is all you need. In other cases, you want accurate, precise scoring predictions for particular teams.

Most teams, to obtain data, use scouters in the stands filling in sheets of subjective (qualitative) or objective (quantitative) data. This data is sometimes compiled into spreadsheets and should be used for scouting meetings (used to decide team strategy, both for matches and selections). Other teams utilise match video or other resources. But, when it all comes down to it, what matters most is obtaining all the data you want to or can use and no more.


As for actually answering your questions, I would suggest that scouters be as prepared as they need to be. If they're analysing robots strictly by watching match video, they'd have to know strategy very well and be extraordinarily observant. However, if they're simply marking how much a robot scores, penalties, etc., they only have to be trained insofar as being able to fill out their sheet. The "training" for the former is usually just practice. Practice can be over years of watching FRC games or hours spent watching robots and strategies for a specific game. It's simply not an easy skill to teach or master, especially with how quickly strategies evolve over the course of a season.

We scout using a combination of methods. We have scouters filling in objective data about robots for all robots in every match. This is compiled into a spreadsheet and the resulting data is used for all match strategies. The data becomes more accurate as more matches are entered into the system (later in events). We also have a few scouters who watch matches and write subjective notes, but their role is limited due to the difficult nature of writing accurately from a subjective standpoint. In the past year and half, we've recorded matches and used these as references during scouting meetings, when necessary. Like in sports, game film can be used both to better your own play and anticipate that of your opponents.

[Talks Too Much]

That's an excellent point. Our main scouting goal is to gather info about how the other teams play, so we can tailor our strategy to who we're playing with and who we're playing against. I think one of our problems last season was that the information we were getting seemed directed towards choosing alliances, and we were very unlikely to be in a position to need that information.

[jwfoss]

We have been working in the offseason to document our scouting process and create a manual to train new members of the team. Our goal is to ensure that the students understand the importance of scouting. If you click on my name in my signature you will be directed to my blog, scrolling down a bit you will find our draft "Scouting Manual".

This year we also posted our excel based scouting system here on forums for use by anyone. I believe this worked out well for us and many teams. I am not 100% sure we will be doing the same thing again. Here's the link to that system: http://www.chiefdelphi.com/media/papers/2450?

[Phyrxes]

Off season events and non FRC robotics events you team is involved in also offer great opportunities to test scouting materials and/or train scouts.

[Talks Too Much]

Thanks for all of your help everyone. We have an off-season event in November, so hopefully we'll be able to experiment a bit there.

[Brandon Holley]

Theres a lot of information on this forums in relation to scouting so you are definitely in the right spot as far as that goes.

As for your specific situation, the key piece of advice I'll give is to make sure the system you are using fits your team. Some teams have the ability to use very elegant scouting systems, while other teams simply rely on pen and paper.

For us, we spend a great deal of time developing an easy to use and effective scouting sheet during the build season. It goes through many iterations and trials to determine if it will function as we hope. Usually the sheet ends up being rather small so we are able to fit 6 or so "sheets" on one piece of paper. This ends up cutting down the number of floating sheets quite considerably. You simply use one or two sheets for one team's entire event. We then take this information and compile it into spreadsheets and make various calculations from it using metrics that we develop for each game.

Examples of useful metrics would be, hurdles per match (2008), moonrocks scored per match (2009), goals scored per match (2010), tubes scored per match (2011), average minibot climb time (2011), % consistency of minibot deployments (2011) etc.

We combine this with our more subjective notes that we gather about a team throughout the match. These subjective notes are extremely important, and can only really come from experience. For example, this past year at the Boston regional we were fortunate enough to seed #2 and be able to choose our alliance. Our first pick was a no brainer, team 2648 had been performing fantastically all day, and their scoring metrics matched our own robots in many cases. For our 2nd pick, we had our eyes on a couple teams, but one in particular which was 222. They hadn't competed yet on the season (this was week 6) and were experiencing some issues throughout qualifications. Our scouts kept tally of how many tubes they scored and how their minibot performed, but we were really trying to look beyond that and see how their team was progressing. Fortunately, no one else seemed to pick up on their development and we scooped them up in the 2nd round. We managed to make it to the finals and lose by the skin of our teeth in the 3rd match. These subjective notes by our scout team are what made this decision possible, and what made our event so successful.

Thats just a few notes on how our team does things. Maybe it will relate to your team, maybe it won't, but it will hopefully give you an idea on how you can mold a system to fit your team.

Good luck!
-Brando