Let’s talk scouting

Hello! I’m the scouting captain for FRC team 1625. This is going to be my second year in this position, and I’ve been researching all summers new ways for us to scout! We currently use gearscout by team 2338, and would like to continue to use that- but I am curious how do other teams scout! (Feel free to private message to chat about this too!) I am simply looking to see what other teams do.

2 Likes

3928 - which has ties back to your team :slight_smile: - creates our own application that doesn’t require wifi once cached (uses QR codes to transfer data) and backend analysis on Google Sheets. I’m hoping to get it opened up this year. Let me know if you’d like a demo and I can get one setup with our scouting lead.

You should connect with 1710 who has a more connected scouting system where they do more gamification - betting on matches and getting funny money for scouting well and betting well. @CoreyBrown

2 Likes

We currently use no specialized scouting apps. I’m curious to see if we can try one out this upcoming year.

1 Like

7525 (as well as previous teams I’ve been involved in) use a combination of Appsheet, google sheets, and tableau. Appsheet is amore user-friendly data entry portal than google forms and will cache data locally if you can’t find a network connection. Google sheets is mostly just an intermediary data warehouse, but can be used for simple calculations. Tableau is used for our actual reporting and analysis (you can get a key through the vKOP). Added bonus of Tableau is it’s one of the industry standards for BI tools, so your scouting team can slap that on a resume.

We’ve found that these 3 tools are lightweight enough that it’s pretty reasonable to rebuild year to year to tailor to the game. Gonna also tag in @MikLast, who’s the regional expert on the appsheet->Tableau pipeline.

2 Likes

Scout on vibes.

21 Likes

That’s a half serious, half joking recommendation.

Way too many folks focus on collecting too much data in complicated ways and then have no useful way to analyze it let alone do things quickly and properly at comp. The collection of data is only about a third of the process, but many seem to think that it is the only part of the process.

24 Likes

I wholly agree with this. In previous years we’d tally the score for each match and then graph all the values. After a bit, a lot of the data became kind of stale because people blink, get distracted, misclick, and so on. Not only that, most of the students really disliked that form of scouting and felt like a chore. Now the process is pit scouting which is basically a detailed description of the robot, confirmation of auto/tele/end game, and “vibes”. Then after the first day of comp we go home and do a video review of the teams and get a pretty decent ranking. We keep track of these robots and that’s the info we use in alliance selection. It’s surprisingly fast and the students want to participate…which is the best outcome for us.

3 Likes

4415 has been in the process of making an app the last 2 years, basically you input whatever’s happening in the game that we decide is important, (scores, how their auto is, if they break, etc) and then like @Skyehawk said they can include a description of just their “vibes” of the team they’re scouting. I think this next year is just going to be an optimization of that app, because we enjoy the format the data is presented to us, in a master spreadsheet with all the info we might need.

I’ve been looking into TigerScout developed by 4829 as well as CrowdScout developed by 4967. If anyone can recommend better apps or apps similar to these ones, feel free since I am also on the lookout

3 Likes

Something to remember , theres a ton of options out there. Nothing that stands out in terms of “the” solution. Every year there seems to be somebody trying to develop the answer but too many apps with not enough traction. It’s been like this for years!

Some great options for data are statbotics.io and and potentially scoutmachine.io ( new but looks awesome ) as well as the trusty ol’ thebluealliance.com.

3 Likes

I do know a LOT of teams in the Midwest use pwnage’s ScoutingPASS GitHub - PWNAGERobotics/ScoutingPASS: A FRC Competition Scouting Application which support is up in the air being that they are no longer a team, but is on GitHub, so hopefully someone will take over maintenance

The main components are all basically the same (data collection, data storage, and data visualization/dissemination), some people think theirs is the best :rofl: If you don’t mind putting in a bit of work, you could always go Abit alacarte and pick and choose what you like best from each of the 3 and integrate it together. And the easiest way to customize is to build ontop of what someone already slapped together.

Really all that matters is that what you have works for your team.

3 Likes

Is there an ongoing cost to using AppSheet? It looks like there is, but perhaps there is a practical way to stay in “test” mode and still have enough scale to use the scouting app.

1 Like

We use Appsheet for our scouting program. I believe if you have ten or fewer licenses, it is free to use. We usually have 30 or so licenses and costs us about $300 for two months of service. We use the education license which is half the cost of commercial license.
Please don’t quote me on any of this. One of the other mentors usually takes care of this for us.

We use a similar layout for collection to most. We used to do a google forms to google sheets and now we created a more streamlined app that feeds the same google sheet.

For visualization we use the google lookerstudio (former datastudio). You can see a few of the views we generate on the report I posted on this year’s newton field.

I would say that concept is way more important than implementation. First of all, just bin pit scouting. You will hardly ever get anything useful out of there. Teams are way overoptimistic about what they can do and, even if they weren’t, the data is too hard to process. Try to focus on hard metrics. Bored kids watching matches all day long will normally lead to very low quality data. If you are reliant on their qualitative observations, you will often find very conflicting data. You need something that is decently resistant to errors.

We really like trying to figure out average score per robot with a breakdown on where it comes from. Most years, by just adding up the averages you can already predict most matches. Those kinds of predictions can already inform decisions in a much better way than just reading a bunch of comments would.

For picklists it is really good to be able to break down the scores by source. This way you can find robots complimentary to yours. Also, keep track of breakdowns and fouls. You can draw some red lines on amounts of failures or fouls to avoid playing with robots that might hinder your alliance too much.

If you have any more questions. Feel free to ask

One piece of advice that I haven’t heard much is to work backwords when deciding what data to collect. A common mistake inexperience scouts make is to go to the scoring table in the manual and use that that determine what data to collect. Instead, you should look at what decisions will have to be made for qualification match strategy and picklist generation and collect data to inform those specific decisions. These decisions will change as robots improve and strategy shifts so what data is collected and metrics calculated and displayed will also change.

15 Likes

We are thrilled about all the teams using ScoutingPASS!

I will continue to maintain the ScoutingPASS software going forward.

You can read more about it in my other post:

2 Likes

literally this!!! I was from a super small team that didnt always have scouts, so my method of scouting sometimes only made sense to me as drive coach and was sometimes very based on vibes!!

my real method was to watch multiple match videos from every team attending, write down notes from at least 3 matches, rank their performance based on my team/others at the comp, then split the team list into groups of offensive picks and defensive picks. For primarily offensive robots, I drew out their pathing from every driver station to see how to best defend against them. For defenders, I came up with my own numerical measures of driver ability and robot durability.

Only pit scouting question I have ever used is “What have you changed or improved since your last comp?” and then just had a giant running notes doc on my phone of everything i saw during comp.

Something i sort of considered part of scouting was just interacting with members of their drive team, either during matches or in the pit, since I like to know a teams vibes before i play in elims with them.

(yes my method was absolutely insane and NOT generally recommended but i did get some pretty cool data from pathing!)

Mega post sorry this is really long lol but it’s got some important info you won’t regret the read.
My team does 3 different types of scouting pre-competition colloquially known as “pre scouting”, pits scouting, and match scouting.
When my team pre-scouts we scout all the teams going to an upcoming competition by looking to see if the team has; gone to previous competitions (if so we look at their match videos and match scout them or just gather general data), posted anything about their robot building, and collect generally any useful information we might want to know and put it all in the scouting spreadsheet for that competition to know what we are facing off against before we have any other data on the teams but is generally not used much after the first bit of competition.

As I mentioned earlier my team also does pit scouting which consists of teammates, usually 2 or 3, getting a list of teams and a list of questions and walking around the pits collecting information on the teams typically before matches start either load-in day and early on the first day of matches, before qualification matches start, we use AppSheet for this because I have it connected to our spreadsheet to automatically upload all of the information teammates upload to our spreadsheet. This information is typically referenced for matches when talking to drive coaches so we have information already and things go smoother, it’s also used for our scouting meetings to have information on robot specifics, autonomous period specifics, and strategy specifics.

For match scouting my team uses google sheets and a custom configuration I design on and for 2713’s QR scout so it’s entirely offline although it’s nice to have a hotspot connected to the computer (a cable-tethered hotspot so it doesn’t interfere with the field ) inputting data on the google sheets so other team members can access summarized data for strategy decisions. We have 6 tablets with qr scout pulled up and a computer with a USB-connected barcode/QR code scanner 6 people in the stands on those tablets collecting data and one person who takes notes on the matches, updates the drive team/pit on any important info (broken robots, upcoming match (this was before fc queue) and match predictions so they know what to expect) and scans the QR codes when the match ends to input the data into the spreadsheet, I also sometimes make changes to the spreadsheet if needed or input teams (in the spots where we aggregate data for them) the match schedule and anything else needed scouting related in the stands and their mostly there all competition (previously called “scout-master” but I’m trying to phase out that word for an alternative bc it sounds weird, I highly suggest teaching someone else to scan QR codes because you will need a break it gets tiring or you may need to use the restroom or whatever). We organize our scouting by shifts so we have a schedule and you show up at the stands to scout. Everyone that is not super busy doing stuff (everyone but the drive team, media, and our impact team on their judge presentation day pretty much) will have a few scouting shifts during the competition because it’s good to understand what’s going on and it’s hard for a few people to scout the entire competition. Of course, we have people that particularly enjoy scouting and do a lot more than their required scouting shifts, but everyone that helps is super helpful and I appreciate all the time they put in. Our google sheets have serval tabs, first off we have a raw data tab where we put in all of the data we collect for teams, a “totals” tab which contains a list of all the teams at that competition and automatically pulls all the data for them complied individually from the raw data sheet along with some averages of the data and interquartile means of the data, then we have a schedule tab which contains a schedule of all the matches and their alliances (which are automatically crossed off once we collect data for the match) it also has notable teams marked in distinguishing colors and using the data from the totals tab creates a prediction for our next match, we also have an individual team search tab where you can input a team number and get some more comprehensive data about the team over that competition such as improvement over time notes on driver skill and if they have had any robot issues over the duration of the competition. Related to this, on the night of the second to last day of the competition informed members of our team and members who need to be privy to the information presented have a meeting where we pour over the data and make a pick list of different categories of what teams are good at and discuss which teams are the best match for us and we try to predict who other teams will be picking and then make a comprehensive list of teams most likely to score the best with us.
We are looking into making and using our own custom app like QR Scout this coming year so we can not rely on other teams for things and have a more customized user experience for the app (plaster our branding all over it and darker dark mode along with all the other fun stuff).
if anyone wants I can link an old spreadsheet from last year with all of the Google sheet functions if anyone wants them, and also it’s just nice to have the physical thing instead of it described. If you want I can also share the custom config file I made for QR Scout and can do this every year from now on if anyone wants.

This is an expensive way to establish scouting at first because you have to buy all the tablets (these are the ones we use) but you don’t have to use the ones we have or you can use students phones if they are ok with that and considering it’s just having a cached website it’s not a large commitment and is a lot more cost-efficient. you also would have to buy a qr code scanner (yet again this is just the one our team uses I’m sure there are cheaper options) I don’t know of a way to set up a phone to read the data and input into a spreadsheet because the QR code from the app is just the solid chunk of data from the form but its probably possible. This method also requires a computer (I’m not sure if this is exactly what our team used but it’s very similar) no necessarily with access to the internet but I’ve found it to be helpful when communicating with the drive team about upcoming matches and prevalent information they need to know.
This takes a lot of time to do if you are making your spreadsheet from scratch because making enough formulas to interpret the data you collect well takes a lot of time (or at least it did for me but I was partially learning how to use Google sheets at the same time, and if you want to learn a really good way to do so is to ask chat GPT questions it helped me when I was about to explode several times because I didn’t realize what was wrong with my function, its very similar to programming in that sense. The Google spreadsheet function documentation is also very helpful.) If you want I can also in some way (probably a Zoom /discord call) explain how to use and setup AppSheet with a Google spreadsheet (The Appsheet documentation and Google sheets Appsheet combo documentation is also helpful).

Also just to note when you are making a scouting app it’s really important to keep your data points concise and simple for the user to understand and use. Too much data and your scouters will miss some data stuff will be inaccurate and you will have a bad time, too little data and your scouting is useless because you don’t have enough information for it to be useful. Also once you complete your scouting app try to train your whole team on how to use it its really important people understand how the app works, and what all your data points mean and you don’t need to stand over their shoulder correcting them during the competition, I made that mistake my freshman/sophomore year and it was really hard to maintain order in the stands and get good data because everyone was always asking me questions. It’s also good to gauge interest in who wants to scout and who doesn’t because some people might be interested and want to do more scouting or learn about the system you use and eventually when you leave do what you do.

Also talking with other scout leads from other teams around the end of the second to last day and exchanging contact info and data is really helpful because the more data you have the better and it’s also good to make friends it helps a lot when making alliances in alliance selection. For the last few matches of the second to last day I let someone else take my position in the stands (just another good reason to have someone able to do your job) and go check up on all the teams, their robot status (if they are having trouble or are broken) gauge their interest in an alliance and talk to their scouting leads.

As I mentioned a second ago it’s helpful to get data from other places so you should always look at the sites I have linked here (I was gonna just link them here but this post is long enough)
and this is just data and systems people have set up TONS of teams have guides about what scouting is and why it’s important

Those are just my tips and tricks as scout lead if you want any more in-depth information reply to this or shoot me a pm/dm and I’d be more than happy to share the knowledge I have coalesced thus far.

3 Likes

Team 1073 often makes an app on a Raspberry Pi by working with our software team and keeps all the data stored in a CSV format, so that we can export it into google sheets (or whatever equivalent that you are using) By keeping it on a Raspberry Pi, we often transfer data at the event through USB Sticks, and make sure that nothing is online. We have had situations in the past where online scouting has resulted in our loss of a lot of scouting data, and with the variability of venues in New England, we decided that it was much more worth it to just invest time into making a local app on the Raspberry Pi.

Something that I would definitely suggest to newer teams to scouting, or teams that haven’t built their own system to scout with yet is to avoid spending a long time just pre-scouting. Personally, I find that the time you use on pre-scouting could better be used on other things. In my experience, just having a general knowledge of teams before the season starts, as well as keeping yourself updated with how events in your region are playing out and what teams are doing well is much more helpful than spending meaningless time running through prescouting video matches, when you don’t know how a robot will perform at an event. My philosophy is to judge every robot based on how they perform at the event that I am currently at, so prescouting isn’t really that useful to me.

I would definitely suggest Tableau as a great software for graphing out data, and I used it in the past season for some great visualizations on different teams. The learning curve for Tableau is there, but once you start to understand it, it gets very easy to use, and there is a TON of data that you can visualize in very different ways very easily once you understand how to use it.

Anyways that’s just my two cents, if you are interested in any sort of Tableau use that I’ve done I have a bunch of graphs on my profile from last year, when I made the NE Scouting Newsletter.

You are correct that one downside from prescouting is teams will change, so the information collected may be a little stale. That being said, teams performances are stable enough that there is still significant value in pre-scouting.

Here are my pre-scouting tiers from the past two championships with where the teams ended up in alliance selection highlighted.

2022:

2023:

Sure, there are some misses in there but in general you’re going to have a very good idea of who the top contenders are and the general depth of the event (which is an underrated aspect of alliance selection and strategy). Teams don’t change that much.

I don’t think my team will do this, but if you’re a smaller team, I think it would be valid to scout the teams in the Bottom or DNP tiers less, if at all, as they are unlikely to make it to playoffs.

Pre-scouting can also be critical to match strategy. A really good example of this is match 70 at the championship last year, our toughest match per statbotics. Although 2614 did not have any traversal climbs in their first 5 matches, our pre-scouting indicated they did traversal climb that I was confident in. Given that they are a Hall of Fame team and were shooting just as well, if not better, than their previous events, I wasn’t concerned that they just lost the ability to climb or weren’t prepared for the championship. They ended up with a successful climb and we won by 5 points.*

My general approach to scouting is the more actionable information you can get before qualification matches and alliance selection the better. We are already dealing with very small sample sizes in FRC so anything you can do to increase the sample size the better even if the information may be a little out of date.

*Yes, Pit Scouting would uncover if they had a traversal climb or not, but it would also show many teams also had one that I’d be hesitant to trust