How does your team approach scouting?

Competition Scouting
1

Throughout the past few years, our approach to scouting has been very member heavy. We have a student record specific information from each team during each match of the event, and someone else put every student’s information into a spreadsheet. However, it has become increasingly difficult to keep students interested in scouting and to be in the stands, as well as allowing them to have breaks. This year we have a much smaller team than in years past, and we are afraid that we cannot use this form of scouting anymore.

So how does your team approach scouting? How can we make scouting more interesting, and how can we give members breaks while recording all the data we need?

2

In past years, we have had the same system. This year, we are trying to use a google form. The downside is that kids will have to use their own data so we can collect the information, but it will eliminate the problem of having a member manually input the data.

3

We can identify who are scouts are about halfway through the season. Scouts are students who (usually) don’t want to be in the pit or drive team BUT want to help the team do well. It’s important to remind scouts that scouting wins regionals. We have won and lost matches and regionals based off strategy choices. Scouting is NOT a throwaway position and it’s important to remind them of that.

We try to keep our scouting system simple by having our scouts look for things we think are most critical. Last year it was gear pickup, gear placing, and climbing. There were fuel shooters at our events, but not very many and not very many that were incredibly consistent. Thus, our scouts could make notes about those who shot well but then focus on who handled gears best. I also like to talk to my scouts in between matches by throwing out questions to keep them engaged and let them know their information is important. “What did you see that went well in that match?”, “Who did [task a] best and why?” or even “If [Team A] is so great, what sort of defense could we play to hinder them?”

If you don’t have a lot of scouts, you can do what we call reduction scouting. We scout everyone Friday morning then re-evaluate and scout less in the afternoon. When a regional has 56 teams you can start scouting everyone in the morning then during lunch decide who the top 26-30 are. Then on Saturday morning (if you are in a picking position), only scout those teams that have made your top 23 list (assuming you are the 24th team competing in eliminations). This helps alleviate scouting for a small group. It gives you some data on ALL teams but more data on those teams who are consistently performing in the top 24. And you can even scout less on Saturday because you won’t be able to select more than 1 of the top 8 so really you’re looking for that perfect 3rd alliance member.

We use all of the scouting information to make plans for qualifying matches and (hopefully) alliance selections. However, even if you don’t end up an alliance captain, that information can help your eliminations alliance.

EDIT: Sorry for all the edits; I’m never satisfied it seems.

4

Combine scouting with another team? (possibly a rookie team that doesn’t understand the advantages of scouting, or doesn’t have a system yet)

5

#1 Purpose for Scouting:
Give your drive team real-time intelligence on upcoming match partners and opponents. They will need this in order to negotiate match strategy. The most informed team has an advantage in those negotiations. Often the scouts can suggest strategies as well since they’ve had more time to analyze the data.

Typical Questions For 2017:
*Do we have the horses to go for a 4th rotor, or should we stop at 2 or 3 and play D.
*Are both alliances expected to be even on the climbing game? Is there a slow climber that could be in trouble if held up by defense?
*Would dumping the hoppers and flooding the floor with balls significantly affect a team’s ability to load gears? Are there some shooters without good floor pick-ups?
*Do the opponents have floor pick-up for gears? Do we need to be careful not to leave gears on the floor in our loading station for the opponents to snipe at the end of the game?
*Is there an injured robot or one where a simple modification could significantly improve the performance? (Most likely a climber modification to keep an off-center rope acquisition from walking off the spool. Also looking for radio resets, or brown-outs.)

If we are only scouting our partners and opponents we may need to only watch 1 or 2 robots each match. By the end of the tournament we have scouting notes on most of the field and can form a pick-list if needed. In Michigan they generally do a good job of posting match video. Re-watching a few matches that night can help fill in the data gaps.

Don’t feel like you have to watch every robot every match if you don’t have the man-power to accomplish that.

6

We use a custom spreadsheet with macro’s. Every student at the school has a school provided laptop, so we can have enough in the stands so everyone has direct input into the spreadsheet. The two macro’s we have are:

  • Input macro. We basically create a form for input that lets you put in info for a single team. This macro then takes that info and puts it into a single row on another sheet, identifying the team number and match number
  • Output macro - This takes all of the individual team/match data and collates it so we can get things like averages, ranges, and offensive/defensive calculations for each team. This macro usually ends up being a bit complicated.

From there, the spreadsheets on the various laptops can be collected on a flash drive and the match data easily copy/pasted from one to another to create a master spreadsheet with all the info.

The important thing to keep in mind with scouting: You have to be able to see timely results. Just collecting the data for alliance selection where it may or may not be used (depending on your ranking) isn’t enough. You need to make sure everyone knows the data is being collated every 10 matches or so. You have to let them know statistics on upcoming matches you are playing in are being sent to the drive team and are forming the basis of their alliance’s match strategy. Do that, and everyone knows that the scouting they’re doing influences every match and has a measurable impact on the team’s ranking.

You can also work to make scouting fun. If everyone has access to the full scouting data, you can play a game of “guess the score” every single match. Get them to do some analysis between matches while they’re in the stands, see who gets closest. And then have some candy available as a prize (think small trick or treat sized candy) for the top guesser every 10 matches.

Finally, make sure that, whatever your system is, you have an easy way to create a pick list. And then make sure that whoever your alliance representative is has an easy way to cross teams off that pick list as they are picked. We actually take a laptop up with us. As teams are picked their team number is typed into a “picked” column, and conditional formatting automatically blacks them out on our pick list. That way the rep doesn’t have to search through the list to find them in order to cross them out!

1 Like
7

I think this is the best solution if you only have a few scouts. If you cannot collect accurate data on every team, the first goal should be to seed high and then worry about the picklist.

I actually think an argument could be made that a significant number of teams would be better off studying their upcoming opponents/partners than passively collecting data on every team.

8

We partner up with other teams to scout. We develop a tablet-based scouting system (with some other teams that either provide programming support or input on what should be scouted) every year. We have a fairly small team year to year (I would guess maybe only 15-20 or so students), and it can become rather difficult to field a drive team, talk to judges, and scout. Some teams have the opposite problem, where they have an abundance of students looking for something to do, so we help each other out. At competitions, we have a student managing sign-up sheets where scouts can sign up for (I believe) 1-hour blocks. To help with data quality, we have metrics that track the accuracy of scouts, and the most accurate scout is rewarded with a gift card (for I think Amazon). We make the data available to any team that wants it. We also provide support to teams who want to use at competitions we aren’t even attending (and have done this several times).

9

Another thing to keep in mind is that lots of events are won and lost during the second round of selections. I personally like to keep an eye out for robots that I think would make a good 3rd partner starting from day 1, in my subjective opinion, based on factors like how they move about the field and play defense.

Raw scouting numbers and general knowledge will usually get you your first round ringers, but finding a quality third robot, especially late in a draft, can take a bit more digging. If you feel good about the top end of your list going into Friday night, definitely consider putting a few pairs of eyes on potential 3rd robots on Saturday morning* to make subjective decisions about who can maybe do some smaller things that aren’t going to appear in raw numbers to separate themselves from the pack.

*This is in the “Friday-Saturday morning quals at a district” world, adjust accordingly per event schedule where applicable

10

Some people above have mentioned this, but the biggest thing to me is about the culture of scouting. More often than not, scouting is where the kids not on drive team and pit crew are sent, often with little to no mentor and team support. In my experience this doesn’t go well. The data collected isn’t good and the analysis is poor, and it often reinforces the notion that scouting is not important. We try to create the culture where scouting isn’t just data collection, it is the entire cycle of analysis, reporting, and prediction.

Pit scouting is important to help excite students. We ask a few questions, usually about how many batteries a team has and the weight of the robot, but we encourage our kids to ask about what they find exciting with the robot or pit. This is often a moral booster, and gets them more excited to see how the teams do and how they can learn from what they saw. Additionally during matches, we have discussions about what we think the outcome of the match will be. Although superficial during the first 20 or so matches, everyone starts to think about strategy and some really interesting points and discussions pop up. Although not something we practice, I have heard of some teams that reward their scouts with ice cream during lunch if they correctly predict a certain number of matches. During the scouting meeting the night before alliance selections, we try to encourage all the scouts to provide their input by going around and asking everyone what their favorite robot was and why. We also match the data we gather with a scout, so we can target important questions we have at the shier students who might be too intimidated to talk.

There is more we can do to help our students, but I find that this helps create a culture where scouting is respected and the students are more than happy to wake up mentors at absurd hours to wait in front of event venues :rolleyes:.

Our head scout and I gave a workshop on scouting details at Beach Blitz. You might find the slides useful as well.

11

Google Spreadsheets for active data management and visualization. My students were introduced to it during Beach Blitz and said it changed their approach.

As far as Saikiran’s notes on the culture of scouting, I agree that it’s used as a place for students to be filed in if they do not have the skills for both drive and pit.

However, I emphasize that teams will win because of scouting directly. That is why I have made sure scouting is number 1 priority at events and that I’m always micromanaging them to ensure that they understand the importance. Additionally, if the understanding that scouting is first, then you’ll have more students eager to join over other positions.

12

Yeah the mentor / coach also scouting helped us keep kids in the stands. We also supposedly implemented a system where everybody had a button (red 1, red 2, etc) and you couldn’t leave the stands without giving somebody else the button to cover you. That helped the kids self police from taking long breaks.

We use AppSheets now, which was suggested on another thread here. I feel like you could make a super easy App that i feel like any team could use and find helpful. You guys might want to consider something like that and like discussed find another team who may want to pair up with you.

And other folks who are telling you to focus on 1-2 teams is probably your next best option. It turns out that finding the top 5-6 teams and ranking them is easy; we spend all day on Saturday scouting second picks and that list might be only 15 total teams to focus on.

13

Can you explain a little more about how you track this?

14

Unfortubarely I don’t know the exact algorithm used, but I seem to remember it checking with the FIRST API. I’m guessing it looks at each parameter (balls scored, gears hung, etc.). I’m not sure if it calculates the residuals over a period of time or what. I can ask the mentor in charge of it whenever I see him next.

15

Just wait until there are 15 or so qual matches left and figure out if you are going to be in the top 8 or not. If you are going to be, watch all the remaining matches and make a list of the best teams that you see in those matches, you probably only need like 10 teams on your pick list. If you aren’t going to make the top 8, then just be glad that you didn’t waste time collecting useless data on the earlier matches. An alternative strategy that I actually have used was just to copy the rankings list for your pick list. As a last resort, just pick famous and/or low numbered teams and hope they haven’t been picked earlier in the draft.

These are strategies a lot of teams use (with varying levels of success), so hopefully they have some merit.

16

As the head scout of my team, I think I’m qualified to talk about this matter.

Prior to our second regional in 2017, we used a paper system developed by whoever the head scout was (I only just started this year). Then, we would manually input the data into a document for match reports that we would print out and hand-deliver to the drive team for them to see.

This was quite cumbersome, and didn’t allow for a lot of in-depth analysis, because the time it took to input all of the data was more time than the head scout actually had. I personally know that a lot of our picklist at our first 2017 regional was based a lot more on qualitative assessment and personal opinion than quantitative data, and therefore we didn’t really have the most accurate picklist. However, the way we did it, which was new this year, all students at the regional were actively engaged in the making the picklist, and scouting seemed a lot more fun than it had prior. From what I had seen, scouting with the paper system just seemed like a chore to most of the members, without the carrot of helping with the picklist.

Then, the team decided it was time for a change. One of our outstanding programmers (a freshman at the time, too!) collaborated with the captain and mentors to create an electronic scouting system for our second regional.

The effects of this have only been positive. Our data is put into a Google Sheet for the drive team to see, rather than having to hand-deliver match reports to them. We also use the same Google Sheet for our picklist, which leads to the most informed picklist decisions possible (which is one of the main reasons we made a finalist run from the #6 alliance at that regional, which just goes to show that scouting can make or break your regional). And, our team went from the consensus that scouting was a competition chore to the consensus that it’s actually fairly cool.

The way we scout at competitions hasn’t changed that much. We try to use sign-ups for scouting shifts (typically an hour and a half), and then those members show up. We have had shirkers in the past though, although less since we introduced the electronic system, so plan ahead for that. We always tend to have a few extra people in the stands who can scout if need be.

If anyone has any questions about how this system works, send a PM my way and I will be happy to either answer questions or direct them to the app developer himself. We have also co-scouted with teams using this system to much success, so if you are curious about this, PM me.

Mythically sorry for the wall of text,
Danny

P.S. Don’t listen to Caleb.

17

Unless your team is INCREDIBLY low on people and ability to scout and you have contacted every team at the event who has scouting to maybe assist you, I don’t think this a good idea. The first pick if you are ranked decently high is easy, but where most events are won are with the 3rd robots. As we both know, even a team list sorted by OPR would probably be more representative than rankings.

If a team is that desperate, I would recommend getting the component OPRS for the event until 5 matches before alliance selections and using that to quickly create a picklist of 24 teams. It is easy, quick, and will mean a lot more than rankings/old teams.

18

I’m fairly sure he was joking.

19

I am too as Caleb has posted a TON of really cool statistics stuff, but there are always people here who can’t read between the lines.

20

Starting in 2017, my team has taken a really weird approach to scouting and totally loves it! Since we don’t have as many students as lots of other teams we often form a scouting alliance, opening up our ironed-out scouting system and all quantitative data we collect in exchange for a scout or two from each team. Once the match schedule is out, we assign matches to each person interested (usually about 10, was more like 20 at champs), and have one 1836 member availible to make sure people hit their matches, data is flowing correctly, and everything is running smoothly.

Beyond that, we have a few more scouts doing qualitative analysis of teams. This is not counting gears or balls shot, but more wordy notes about potential strategies, how we could work with or stop a team, etc. These scouts pose a strategy to me/drive team and love the role they’re playing-- it’s a lot more gratifying than just counting what a robot does for 60 matches straight.

Don’t hesitate to take a page from Saikiran’s playbook and buy your scouts Starbucks

When it comes time for alliance selection, these scouts (or super scouts in other team’s terminology) run the conversation with data they’ve collected in a notebook with their own two eyes, using concrete data from the scouting alliance to back up claims they’re making or settle disagreements. For example, “973 is a great shooter in auto” is typically followed by pulling up the data spreadsheet and seeing their auto kpa average. As another example, if we can’t decide if 973 or 3309 is a better shooter, we’d use tangible data as people see different matches etc.

With scouting, it’s incredibly important that the culture is maintained in a positive light. If scouts are just there cause they didn’t make pit crew or drive team, chances are they won’t care, won’t take data (or even worse, input false data), and tons of problems arise from this. On scouting alliance stuff, we typically hear back from kids who want to help their team but don’t have a formalized system, solving the problem because they are excited to do it. A scouting alliance is also a unique networking/t-shirt trading opportunity, and just a super fun thing to do because more kids means more networking and less brute numbers-driven scouting.

tl;dr we work with kids on other teams to do the quantitative scouting and focus the 3-4 available kids we do have on developing strategies driven by that concrete data.