Scouting Process for Other Teams

As this season has come to an end and I have become the new scout lead for my team, I find myself wondering about how other teams do picklists. As we are based in NC, I realize that we probably are in a different situation than a lot of teams in larger districts. That being said, here is how 4829 goes about scouting.

On Saturday night, after the first day of qualifiers, we take all of the scouts and drive team to the start of our scouting meeting. We then go through all of the teams in numerical order, asking drive team how they liked working with them and our scouts if they remember anything notable about them. Once we have this list of notes, we then send everyone to bed except for the head scouts.

We then assume that we will be ranked first for alliance selection and sort teams into 3 groups. 1st pick, 2nd pick, and prefer not. We try to keep a dnp list under 5(if not less), in order to be able to work with our alliance for a second pick. Once we have all of the teams sorted we go through and rank them by how compatible with our robot we think they will be. We look at what our robot can do well(we’re a great speaker bot and we’re good at passing) and what it struggles a bit with(our amp is kind of slow). We then look at who would be a really good bet to help us maximize our potential during playoffs. In this case we looked for a super fast amp bot who was also decent at speaker in case we broke or something. Our second pick we looked for a bot who could cycle speaker reasonably well, who also played good defense. Because of this we had two picklists. Both lists had all of the bots at the competition on them. We then spent some time trying to predict alliance selections(at least the top couple, assuming we wouldn’t be one). At that point we go to bed. Sunday morning we go and watch the rest of the matches, changing our picklists as we feel we should. One thing we do is we look at trends that might not be noticeable in an average. For example, if a robot has an average of two amp notes a match, but has gone from two to four to five to six, we take that heavily into consideration. That is usually a really good sign that they are on a roll and , and especially for second picks can be super useful. We have won competitions on scouting in the past, and I am wondering how our methods compare to other teams.

We usually have 6 scouts scouting teams[EDIT: we rotate scouts every 4/5 matches], and the scout lead watching the match and taking notes on anything notable they see. Most of the people schedule robots are freshman and we did a little bit of training with them, but not a ton. The data has been very accurate as we are sure to explain how we use it and just how important it is. The scout lead(s) have more experience, and are able to notice more complex parts of strategy bots might be doing. We have ALSO started pre-scouting teams for later events, going through and taking notes on the teams to see how they improve/adapt over the course of a competition. Come playoffs, the scout lead takes notes on how each alliance plays in their first match and sends it to the drive coach. Our data is very reliable and the bots at the tops of our lists always preform well in playoffs, whether we are with them or not. We made it to finals in our two regional events, but got unlucky at states with a rough second day of quals placing us in 11th for alliance selection. A scorched field made it clear we wouldn’t be alliance captains, and we had a rough time in playoffs. Had we been in a better position, I truly believe we could have won the competition, but alas, we didn’t.
One thing I’m curious about is how other teams go about editing picklists on day two(after they are made). When we make them originally, we have time to analyze the data a lot more, but with alliance selection being right after the last qualifier, we often have to use our best judgement and make minor changes as we see fit.

5 Likes

My advice, be more liberal with your DNP and prefer not groups, focus on consistency and scoring output on the last ~5 matches. Some of this can be on vibes. Get things down to about 30 teams (more at worlds). Rough sort then go to bed.

The actual sorting takes place the next day over the last 3 matches. Identify matches or back to backs in the schedule that allow you to make parewise cinparisons between teams and swap their positions or shift them if necessary.

If you do this right, meetings should max out at 45 minutes for a standard event.

I also avoid “dashboards” like the plauge. Generally you only care about 2 metrics anyway. Dashboards overload the user with info and all of a sudden you are back where you started scouting on pure vibes.

5 Likes

Some great suggestions here, some of these might not work in NC as a lot of events are under 30 teams. So you can’t have many dnp teams. (Fellow NC person) Usually how we do it is we first look at estimated rankings on Saturday night. Then based on these we look at who would be best to be on a alliance with. We reconvene after quals and modify based on final rankings and adjust on the fly as alliance selection happens.

What metrics do you care about this season? I feel like “dashboards” are the perfect way to sift, calculate and combine all the data collected to only show the metrics you care about.

5 Likes

I feel like going in vibes isn’t nessecarily the best way to approach this. We do take in to consideration what the drive team says, but ultimately they know that while their say is important, it isn’t the overriding factor. As for dashboards, do you mean like a way to view the data you’ve collected? I find it super helpful in finding issues in potential alliance partners. For example, if they drop a lot of notes per match(not feeding but actual drops), then they might not be the best choice for us. What metrics do you care about? Only 2 seems really small, even if it is epa and opr. There is so much more complexity to choosing a good alliance. One robot at our district championship, for example, really just doesn’t work well with us. They rammed into us a bunch and didn’t stick to the plan we made, costing us a match that should have been an easy win. We also look at compatible autos and a couple of other things.
We start out the meeting by defining what we want in our first and second picks, before even looking at the data. From my experience we can go through the data pretty quickly, it’s just the actual discussion about each robot amongst the scout lead(s) and mentors that takes a while. However, I think that it is one of the best parts of our process. As for DNP, NC is a small district, with only 40 teams attending the dcmp.

Thoughts on the “prepare as if you will be 1st, assume you won’t be” mindset?

1 Like

Prepare yourself for the highest seed you might be able to attain.

1 Like

The thing is, we were in first for a significant portion of day one at states. I know we have the potential to be first at all of the competitions we were at, we just fell a little short. I think having a Picklist as if you are first prepares you for the chance you do wind up in first.

Every team has a chance of being ranked one, you never know what will happen.

I agreed with what you said until this. You can figure out your highest possible seed by finding your rank if you get 4RP in every remaining match, and everyone above you gets 0. Unrealistic, but highest mathematically possible.

That’s a good point, and definitely something we will be doing from now on.

Honestly, I think it might be a waste of time in a lot of cases. A great resource is statbotics as they have a ranking prediction. It isn’t great and a lot can change but it should give you a good baseline. While it is great to be hopeful if you have a chance of being ranked one it is most important to know what to do if you are not ranked 1. Typically, the first pick from rank one will be pretty obvious but it is often much harder to find the best pick from rank 5 or something, especially at district events when there might not be much more than a few good robots. Typically the deep field is a lot more important as it can really change the outcome. This is especially true being a higher seed as there is typically not much to pick from. Statbotics predictions are a good way to roughly estimate how matches will go and ranks will end up. They are not super accurate but they are good for going quickly.

Yeah, that’s part of why our scout meetings take so long. We wind up with a pretty thorough list of bots and an understanding of why we want to pick them, and how to change that if we need to during selection. When I say that we always assume we will be rank one it’s because we have been in a position at every comp this year where we could have been.

I totally understand that. My personal opinion is to plan on not being ranked one strat-wise and it being a pleasant surprise if you are. That way you get more time on picks that would prob be much harder than being ranked 1. Other people may have different ideas but that is my 2 cents.

Honestly, I think the system you guys have is very smart and seems to work well for how your competitions for structured. If you are looking to cut down time, I would recommend sorting teams in to a top 24 (32 for worlds) instead of a first pick and second pick list. That way you can see how accurate teams follow the pick list you made when alliance selections occur. You make the list off of how compatible you would be with that robot, so not every team will follow it right in order. Our team does this in a meeting with only scout leads, and it tends to go 10 times faster than when we have had bigger groups. I would love to know how much feedback you guys get from scouters and drive team as that’s been a big discussion this season on whether we should ask for their feedback.

1 Like

I would absolutely reccomend getting drive team feedback! Knowing who they work well with, and who they don’t, can be a factor in how well your alliance meshes and works during playoffs. As for the scouters, they have a lot of knowledge about the robots after watching them all day, you’d be suprised at some of the details they can pick up on that might not nessecarily be written down in the notes for some reason. What we do is we rapid-fire down the list asking for anything that stands out to them(both scouters and drive team), and once we have gone over all of the teams, we send all of thescouts and drive team to bed. What would the advantage of one picklist as opposed to two be? From what I’ve seen, having two allows us to be more flexible during selection. Knowing that what we want in a first pick and second pick is pretty different, making one list seems a little under-comprehensive, imo. one more thing is i think having more than the top 24 teams is another benefit, as we will probably not have the same second pick as our alliance partner. Being able to choose a second pick with your first pick(or alliance captain) is a big part of strategy, and what usually takes us the most time. Sorting through the mid-to-low level bots and reviewing them with detail seems to me to be the most beneficial part of our meetings. A good(or bad) second pick can totally change the way that playoffs goes. If someone from our first picklist is still available for us as a second pick, we might decide to take them instead(because they might be an amazing speaker or amp bot, giving the alliance more room, but that seems like a pretty obvious point to me.

Ah that makes sense. I already talk to the drive coach before making pick lists for how well they work with other teams, but the scouters picking up stuff we don’t see would be pretty cool to add for us! The reason we do one list is because typically our second pick isn’t a defense bot as Michigan needs 3 offensive bots to really win a competition. Our system automatically ranks teams by different categories, and so we take that down along with scout lead notes to help when comparing lists. We really try not to overcomplicate things when making our list.

Our team creates a ranked list on Friday night. We start with a first order sort based on estimated expected points per match and put them in rough tiers, then we go back through and tweak the list by comparing teams to each other. We end up discussing a bit about every team. We make a watch list of teams that we want to know more about, and we focus most of our Saturday morning scouting on those teams, which tend to be teams in the middle performance-wise. On Saturday morning, we move teams up and down the list as we gather new information, and we print the final list shortly before alliance selection.

2 Likes

While you probably do need 3 offensive bots in michigan(im sure a super effective defense bot could alter the way things go, but that’s not the point here), i think differentiating between what your ideal first and second pick can do is important. For example, you might want a robot that can pass, and a robot that can score amp super well. I think having the different lists is a useful metric to have because during selection you can look at the lists and pivot based on who is available. Really the only difference I think I can see from what you’re saying is we move the all of the lists we make around a little bit based on who we work well with/who could maximize our potential, instead of just sorting by score. Our system lets us pull stuff up if we need to, so that part isn’t really an issue for us.

1 Like

Our competitions are only saturday/sunday, so we don’t have as much time, but, as mentioned, we make the picklist on saturday night. Sunday morning is when the last of the qualification matches are, so we watch the matches and adjust accordingly. The watchlist is sort of something we have done, but never something we really created an official list for, so we will definetly be doing that from now on!