The Struggles of Scouting (And a Cry for Help)


#1

Scouting. The one word our team doesn’t like to hear until build season ends and we brace for the excitement of competition. The problem is, our team tends to struggle with scouting. For as long as I’ve been on the team, we have tried multiple different ways of scouting. From paper to electronics, nothing has been a solid way to scout teams. Some of the ways have been mediocre, but not one has been one good way that we can implement for years to come. So, members of Chief Delphi, does anyone have any easy and effective ways to scout? Help me Chief Delphi, you’re my only hope


#2

The easiest method to scouting that we’ve used for many years is paper match scouting. We usually ask our scouters (who are rotated throughout the event) to look for pre-defined things that will help us make our strategies for future qualification matches and create a picklist for alliance selection. Our scouting sheets used to be half pieces of paper that would be sorted into a teams specific folder at the end of the match. So whenever we have a match with or against someone we pull their folders and see the teams strengths and weakness and find the best way to play the match is.

Historically this has worked for us, and it is always the backup we keep for all our events we go to.

If you want any further information feel free to reach out.


#3

How many students would you have available for scouting this season and how well versed with FRC are they? How did you implement paper scouting and what were the specific issues? How did you implement electronic scouting and what were the specific issues?

I’d be happy to help you out but I need more information.


#4

We usually have about 6-10 kids that are able to scout each match. Usually a couple of those are in the pits talking to different teams there. For paper scouting, we made a document for each team with several areas for scouts to fill out (drive train, assets, etc.) and another sheet for each match that had different places for points, comments, etc. However, we found that for match scouting, scouts had a hard time keeping up with the match and writing at the same time, and it was challenging to keep track of match sheets. With electronics, we had no good, uniform way for scouts to record match stats or robot specs and file them onto one database. Our database ended up being an excel spreadsheet, which was difficult because it was a lot of data that was hard to interpret.


#5

This past offseason Team 503 has spent a lot of time transitioning our internal scouting stystem to being readily available to other teams for use called “SWAMP Scouting”. It’s as simple as heading over to our website and creating an account and anyone can scout out of a web browser on their mobile device. Data is then automatically processed into easy to read graphs to help show things as general as “Best Teleop Robots” to things like “Best Robot at Hatch Intaking”. The platform is ready but being refined for use with this years game. A more official release will happen in week 5 or 6 of build so look for details then! If you have any questions feel free to shoot me a DM!


#6

Do you mind sending me the link to that platform?


#7

http://frogforce503.org/SWAMP/info.html

The platform is not released completely due to modifying it for this years game. Feel free to join our mailing list for when we go live or for support. If you have anything you’d like to see in SWAMP please let us know, we’re always looking to improve


#8

Tally marks is the way to go for paper scouting. Have the scouts post match write the number of tallies for whoever is inserting the data to enter later.

Was it hard to input the data or to get meaningful things out of the data?


#9

Thank you!


#10

It was both hard to input and hard to interpret. We didn’t really have one strong way to organize things, and the excel doc was challenging


#11

Look into Microsoft Access. Its like Excel, but its used more in strictly a database type setting. It has a form feature which will take what you add from your top row into an easy to fill form for your data input person to easily use (all of this is assuming paper scouting) and keep track of data.

You can also do Google Form into google sheets if theres an issue of using MS products or the like. Access just works better for my next point, and is all offline, but both should be suitable for this.

For both paper and electronic scouting, Every team gets a free license of Tableau, thanks to Team 1983. Tableau is a data analytics program that basically will churn out graphs based on the data that you have. 4513 has been using Tableau since 2017 and its elevated the scouting of the team far more than anything done in the past. Tableau even have a great tutorial series on setting up both Access and Tableau to use with FRC scouting that id highly recommend looking into. Tableau supports a large range of sources (excel, access, Gsheets, and SQL which will probably be the main things an FRC team would export data to for analyzing but theres a ton more) and once you get the hang of the basics is a super powerful program.


#12

I’d just like to add that through Microsoft Forms you can pretty easily create a form to input data into excel without having to deal with learning how to use a Database system.

Also seconding using Tableau as an interpretation method. If you have any trouble importing data from access, excel, or gsheets into Tableau, CD or the general internet are pretty great resources and should work pretty well.
@maverick


#13

The biggest thing that makes scouting work is organization. you need to organize shifts for your scouters to take be able to take breaks and know when they need to be back to the stands to scout. you most importantly need to organize your data input, wether its paper, google forms, or an excel doc it must be easy to understand how input data. Then you need an organized output for data, whether you to use tableau, excel and graphs, or writing things down everything needs to be organized so you could take it to any person who knows how the games work and they can tell you what each thing means.

Finally if everyone hates scouting you need to explain why it’s important. how does this data help you? there is a reason 1678 has been on einstiens for the past 5 years, not because they always make good robots (though they are all very effecient) but because they make the best picks, they rank high because they go into each match already charting out what each team on their alliance needs to be doing during the match. They even try to guess what the other teams strats are to counter them.


#14

Hi maverick, first of all congratulations on your district win at Wilsonville this year.

Many below have recommended the usage of paper. This is by far the easiest way to scout if your team is lacking in resources and I would recommend it to anyone who has this problem during the season. However, from personal experience, a scouting app would be far better for a number of reasons (if you can manage it, if not, good old paper will work just fine). First of all, data compiling is way easier. Instead of looking through a file of 10 or so papers for each team, you can have their average stats (as well as min/max scoring) by simply compiling match data. This method also allows you to automatically sort for the best teams in each category, and you can compare teams based on what you feel is the most important.

Obviously this does take a lot of time, given that every year is a new game and therefore a new “scouting app” is needed, not to mention needing devices to record the data before it is compiled to a central source for review. You could always have students download the program to their own personal devices as well if necessary.

As for scouter numbers, I would recommend having 6 scouters active in the stands, along with 3 strategists that take detailed robot performance notes (2 red, 2 blue and the most experienced gets a red and a blue). Also if possible, a match video from the stands and one from a media student down by the field. Obviously if this is too much for your team, you don’t necessarily need the strategists or stands video, but you are losing out on information that might be the clear tie breaker between two bots that otherwise seem to have identical stats.

Hope this helps.


#15

I think maybe one problem you guys may be having that hasn’t been mention is possibly what you are scouting. For example, the parameters you use for Scouting could possibly to overelaborate or possibly your scouts are scouting too much, keep it basic and scout what is necessary. If you cut that off and it allows your scouts to spend more time watching the match and getting more accurate data, instead of trying to keep track of every detail that you desire.

Related to that it should be user-friendly so that when people are scouting that can easily locate what they need to fill in. Making it seem simple, but provide enough information for great choice, for your scouts can be the beginning of imporving the image of scouting for your team.


#16

Hi-

I just posted the first half of what’s below to a 2-day old potentially stale scouting thread, so I’ll repost it here. Apologies in advance for the (mostly) duplicate posting, as well as for posting anything here that may be slightly off-topic.

236 goes low tech on the front end and high tech on the back end. We employ simple (paper) scouting sheets that are both easy to fill out and easy to transcribe, and we collect only the truly relevant team-specific scouting data that we need to make decisions - admittedly, almost everything. We also pull FRC data real-time using python scripts calling on First’s API services - scheduling info, full match data, ranking data, and published stats (OPR, DPR, CCWM) - all via a cellular data connection. Depending on the year, some of these data get merged/manipulated - typically in MS Access - to provide additional key info and granularity that scouts may have missed (e.g. for Power-Up, did they complete a “cross-court” auto or how often did they score on the scale or the switch when it was lit on their side of the field?). All these data are then brought into Spotfire DXP (similar to Tableau) for data visualization and analyses. We plot all sorts of stuff…starting positions, specific actions, defensive skills, fouls caused, human player evaluations, trends over time, alliance strength evaluations for context, and more, and we do our best to identify the under-ranked teams that bring key complementary skill sets to potential alliances (should we only be fortunate enough to form one…).

We have tried scouting electronically on phones and tablets but we (presently, still) prefer paper over silicone. As long as the sheets are fool-proof and simple you can collect a rich and accurate data set to chew on. IMO, the main advantages of paper are a) you always have a hard copy backup; and most importantly b) it helps keep scouts’ eyes on the field. Flipping, swiping, clicking, and poking at a device can be distracting, even for the generation that grew up with them, but just about anyone can hit check boxes and make tick marks on a sheet of paper w/minimal effort and frequently w/o even looking. The only drawback (IMO) is that someone has to transcribe the data into Excel, and quickly, but with well designed sheets we’ve been able to keep up with ongoing matches with just one person doing the transcribing (of all 6 teams’ actions per match). Granted, that takes practice but any decent typist can do it.

Re: Motivating scouts, the most successful strategies we’ve employed include a) keeping it simple - the task has to be as painless as possible; b) going the extra mile with your entire team to make them all understand how important it is to get accurate team-specific scouting data; and c) rewarding your scouts by recognizing their efforts - especially the high performers - and encouraging others on your team, especially your drive team, to do the same. Sometimes this is just a matter of verbal recognition with a polite golf clap, sometimes it’s passing out sweets on the bus ride home (that you can typically find at a discount when purchased at the very end of the second day).

When I started mentoring a few years back (Recycle Rush) we might have collected 30% of match data, and it was a real chore to motivate our scouts and to track them down at an event when they were “no-shows” for their scheduled session. That was pretty stressful on everyone. In the past few years we’ve collected well over 80% of all match data, at some competitions over 95% of all match data. Remarkably, we have done this with self-motivated scouts - that is, we don’t ever set and enforce scouting schedules for our team. Instead, our dedicated team of scouts literally clamor over one another for their chance to contribute. It gets so bad that I have to police the situation - which is a really nice problem to have.

If you build it (correctly), they will come!


#17

1732 does a mix of both paper and electronic. Since we have such a big team, we are able to have 6 people scouting 6 teams per match and they rotate in and out every hour or so. We have an additional 7th person per rotation taking the data that the other 6 scouters took down and put it into a program we write that goes to an excel sheet which gives us the data from teams. We also have a select group of people called super scouts (amazing name). The super scouts will talk to teams in the pits the day before competition and ask them a few questions depending on the game of the year. The super scouts during qualifications watch a certain group of teams and take data down like They have a good driver and drive coach, the driver gets mad very easily, their robot is very shaky and moves very slow, their robot is super mobile, etc. We have a presentation on our website here about our scouting and other ways to do it https://www.team1732.com/frc-training-day Hope this helps!!!


#18

Team 48 has spent the last 2 years using the FIRES scouting system. It has been very useful. Its has a field layout that allows you to point to the area on the field where the robot you are scouting is performing an action and you can indicate what it has done. It then creates tables for sorting and such.

If you want to look at how FIRES scouting app looks and works. Here is where you can play around with last years 2018.

Use the test Regional.

Here is last years program.

https://fires-2018.azurewebsites.net/CS/#!/secure/login


#19

We’ve got a scouting development team, and they’ve made a webapp to input and view data. We assign one kid per robot, and we have them basically record individual scoring data (so we can determine they did well or were carried by a team mate) and more subjective information (robot stability, driver skill, etc…). Usually mid-competition we realize we need to add another metric though (turns out pilot competency was a major factor in Steamworks). We rotate about 20 kids (not everyone needs to be in the pits) through and have them drop-in/out using the webapp on their phones as they’re available.

Here is what last year’s report looked like (on a desktop that is, it adapts to phone screens)

And then we can get a big chart with sortable tables out of it. This is sorting by Average Teleop Scale cubes per match. There is a lot of downward and side scrolling for more info, but you get the idea.

When we’re picking an alliance, we’re are picking teams that fit the roles that we need them to fill (switch/exchange or high scale). So we use this chart to find the best available teams for those roles.

Pit scouting is basically useless (aside from gathering measurements and other objective details), because they may say they can climb 100% of the time, but why take their word for it, just see what their actual stats are.

Additionally, we find that we base alliance picking on just the last half-day of matches, because their robot might have been kicking butt at the beginning of quals, but if they have lost the past 4 matches by huge margins, then something is very wrong.

If you want more info on this, our team, 2102 Team Paradox, has done a presentation on it at Houston the past two years, and I think we’ll probably end up doing it again this year.

So, is this easy? Not to setup, but it’s pretty dead simple to use and look at data. It is very effective though. In San Diego in 2017, out first year doing this, we basically moneyballed our way to victory. We were the 3rd seed alliance, and 1 picked 2, and then we picked the 14th and 15th seeded robots (15th seeded robot had never even been to elims before). The next year, we picked the 5th and 45th seed team and got to finals where the whole alliance was wild-carded. So it’s been working pretty well so far.


#20

We used to use paper scouting but that quickly became chaotic and required twice the time (as the data is first added to paper and then a spreadsheet). Although I can see this working for large, well organized teams, we have a relatively small team so we usually have only 6-8 scouts total. Last year we switched to using google forms + google sheets. It’s really easy to set up and we have 6 scouts (1 per robot) fill out a form for each robot each match. The data feeds straight into a sheet (viewable on phone, computer, etc.) where you can perform tons of analysis on it that updates the instant a new form is submitted. It’s very easy to set up and has worked well for us the past year. The one downside is that I haven’t been able to find a way to add a timer and event timestamps to calculate cycle times.