Your scouts hate scouting and your data is bad: Here's why

This has been recommended by other mentors/teams. It is something we definitely need to look into.
We’re already using our data to confirm/deny alliance partner promises (“we can do 20 cargo…”, no the data says you score 1), once we trim down the data scouts collect, they can more easily communicate it to drive team.

1 Like

Although I personally have never scouted at a competition. I believe that the teams success is directly correlated with the amount of scouting that students are willing to put in. Years ago our team wasn’t as good and some of the students refused to scout no matter what. As we have progressed better through the past couple of years are kids are excited to scout. So what I’m trying to say is it’s about the attitude of the students that determine whether want to scout not just the complexity. But great Ted Talk

2 Likes

399 is a perennial good team, they are hard to miss :wink:

Teams all have character/class… they are what they are for the most part similar every year:

Us almost 90% elim participant after rookie trip never made finals (9/10)
399 solid one to highlight every event
Others good but guaranteed to break in quals / eliminations
Others fast strong flashy but cannot win many events
Pre-scout history flushes this out
I can guarantee teams pick teams much of the time based on previous years performance and character/class expectations

Same story every year, until it changes otherwise teams are mostly same year to year

We were on the receiving end of someone’s bad scouting this past week. Saturday morning one of our upcoming alliance members came up to strategize and said “I hear you play great defense…”. Our drive coach proceeded to spend the next 30 minutes telling them variations of “we haven’t been on the other side of the field once… we don’t play defense, and if we tried we would end up being penalized for being outside our frame perimeter”. 30 minutes for something that anyone actually watching matches would have known. That particular team wasn’t very happy with our response, but we ended up winning the match for them when they broke down… and yet I think they still aren’t happy with us because we didn’t bend over and do what they want.

Yes, I agree it’s an exception. However, I believe it needs to be pointed out. Most top tier teams do amazing scouting and do record an enormous amount of data.

Lots of smaller teams will try to emulate this because they see better teams doing it with success. It’s important to note that they are not setting the standard, they’re the exception.

I know how other teams scout and draft based on “reputation” which is why there are certain teams that end up on alliances that have no business being there, and why we are able to consistently get one of our preferred selections as our 2nd pick. It’s breaking out of exactly what you describe and looking at the data that helps us find the gems in the scouting data.

Our 2nd picks rotate around quite a bit, and we’re not relying on a long history of performance. We had little experience with the teams at Utah, so data was key. (And 1410 hadn’t played particularly well before then.) 399 has started disastrously at Utah which is why they had dropped off the radar for so many teams. At CVR this year we picked a rookie team that emerged from a group of about 6 teams we were comparing directly. So we come back to the scouting data, not individual observation.

5 Likes

We ran our scouting systems in 2013 and 2014 with 10 students total, which included two programmers maintaining the system. Our white papers for those systems are on CD. (However, you should use our latest uplink system which resolves many issues with the earlier communication mode.)

If those methods work for you, great! I found that having some stats (all ~10 matches of them)(that statistical significance tho) helped my team find “diamonds in the rough” - We were able to look at trends and say "their average isn’t great but they’ve been improving throughout the competition. If I had to pick between quantitative and qualitative, I would def take your side; but I don’t have to do that so I’d rather have both.

If you’re training scouts beforehand, you probably have a great culture around scouting!

This is probably the best mission statement of the scouting subteam I’ve seen. When you think of it this way, it changes your whole perspective on how you should scout.

11 Likes

This is great advice all around.

When we started our team we wanted to make sure scouting was treated as an important job from the get-go, it was not to be the job you are punished with if you’re misbehaving or the job you are stuck with if you didn’t make drive team. It also came hand-in-hand with a team philosophy of following Karthik’s strategic design process. We wanted to make sure we built a robot that could play the game well. The hard work paid off – in our first year of competitions we ranked well in quals and made playoffs, and have continued to be consistently “mid to upper tier” every season since. The scouts saw their value from day 1.

We also have students on our team who love strategy and numbers so they champion scouting. We have enough students that we can spread the load so everyone has time to take breaks, wander around, and not feel like they’re chained to the stands all day. For several years we worked on a custom Android scouting app which gave software students a practical application and helped excite them for scouting. One of the “perks” of scouting is you get to stay up later than the team curfew, although this year we’ve found new ways to streamline the picklist making process and the scouting meetings are done in under 2 hours.

Any suggestions on “comfy seating” perks we could provide for our scouting team?

That 3.5" thickness keeps my legs from becoming numb.

If you search for “bleacher seats” on the search engine of your choice, you’ll find a wide variety of options. If you attend events with bleacher seating, make sure to get chairs with backs (and for extra swank, padded with armrests). It’ll likely be an investment ($200+) especially if you get the nice ones - I thought it was worth it though

Yes! Ten thousand percent yes! Everyone needs to see the value in scouting. Too often I hear drive teams and pit crews from other teams talking down on their fellow teammates who are scouting as I’m walking around in the pits. If everyone saw the value in scouting, the overall morale and culture surrounding scouting would improve tremendously.

2 Likes

I was talking to a mentor from one of the top teams at our event (they would go on to win the blue banner). Their policy is to rotate everyone through all the jobs. At the particular moment I was talking to her, she was lamenting that they could really use their top mechanical guy in the pit right now because their robot was having trouble, but that he was currently on his scouting shift.

I was impressed.

2 Likes

I can’t imagine NOT having any scouts involved in these processes, especially the pick-list.

One thing that has improved scouting on my team recently (this year we had too many people that wanted to SuperScout at events!) was making the match strategy and pick-list process more public to the rest of the team. In the past, the scouting data for qualification matches was almost always shared privately in text messages / DMs / or paper handed directly to the drive coach. This year the pre-match strategy data sheet was shared publicly in our team’s slack so everyone could see their hard work actually being used (plus, people stopped asking me how our upcoming match looks like).

In terms of the sim-buck strategy, I think that works for some teams in some situations but isn’t a one size fits all approach. I’ve overheard other teams’ scouts be so excited to see if their bets won or not, that I’m sure they weren’t focused enough on their robot. It shouldn’t be the main focus of why your scouts are watching matches.

I’m not suggesting teams actively do this but…I think the most effective way to getting a team into scouting is to having it cost them an event. Nothing gets their attention more than having one of the best robots at the event, but pick the wrong teams because scouting wasn’t taken seriously enough. That is what motivated me to get involved way back when I was a student.

2 Likes

We had an issue last year where the best mechanical student was also on the drive team. They missed so many alliance strategy meetings that this year we made the decision as a team that the mechanical lead could not be on the drive team.

We have several student roles (drive team, mechanical lead, chairman’s presenters, safety captain, pit crew) that prevent them from being on the scout team. We’re a good sized team but not big enough to have backups at every position.

This year my team experimented with having each of our scouters get to take a turn working in the pit, either working as a full member of the pit crew for a couple hours or shadowing a more experienced member. Our message to the students was that scouting is extremely important, but can also be hard/boring at times, and we wanted to give them the opportunity to take a break and contribute in other ways at some point. This helped a lot with morale; getting to be “part of the action” in the pits was really exciting for new members, and I heard a lot less resentment about having to scout the rest of the time.

We also allow all interested students to attend, and speak up at, our pick-list meeting, and our scouting captain represents our team on the field during alliance selections.

It didn’t occur to us to share pre-match strategy with the whole team over slack. We were trying to shut down the excessive post-match back-seat driving (drive team being swarmed after each match and deluged with “Why did you do X? Why didn’t you do Y???”), and in retrospect I think everyone knowing the pre-match plan would’ve exacerbated the issue, since no plan survives contact with the enemy. But it would be nice to find a way to show our scouters how their data is used by the drive coach and strategist.

1 Like

Not to derail my own thread:
I try to encourage/enforce the rule of “all criticism for the drive team goes through the drive coach.”

9 Likes

My contributions to this thread:

  1. The scouts, not just the drive team, should get to see the aggregated data as it’s produced so they can see the cool thing they made.

  2. Partnering with another team for scouting is fun - this has helped us out with scouting burnout. Friday is a LONG day if you don’t get any breaks, so you want more people involved. If you have a system that produces data in a timely manner, you can offer value to another team (scouting data to support their match strategy and pick list) in exchange for people to help collect the data. Plus you can make some new friends. I’ve decided that I’m not going to make anybody scout if they don’t want to, because that contributes to bad attitudes and bad data. Partnering with another team makes that easier for our team.

  3. I have a notion to write a system like FiveThirtyEight’s probabilistic NFL predictions game into our scouting app. Similar to the SimBucks idea. This would be fast and fun, so I’ll have to see if I can write it for next season.

  4. I put “scouting stats” in our app. It tracks how many matches each scout does and their volume of comments. The scout with the best stats gets bragging rights, and some of the scouts definitely check their stats.

2 Likes

This is wonderful! I love to see those passionate and well-educated about scouting. I agree that it is important to give students a good experience when scouting and involve the match scouters in the process. I would love to see FIRST recognize scouting more - not for any particular team to get an award but because of the huge impact it has on students’ lives and subsequent careers. I’ve been a scouting mentor for 6 seasons - long enough to see our students go on to college and graduate in fields often outside of core engineering disciplines: data science, computer science, human-centered design engineering, social science research, forensics, etc. All of these disciplines require critical thinking skills that are data-driven rather than emotionally-based (i.e. picking the team they like or one that has historically performed well). I think we need to be louder in making our scouting efforts know to FIRST. My lead scout put together this booklet on our scouting efforts. https://chiptang.com/cpr The booklet is listed under 2019 Scouting Documentation. I would encourage all teams with good scouting to make their own booklet or system of sharing the information with others (like the original post above) so that we can all learn from each other. With enough online presence, I think we could slowly but surely transform how match strategy and alliance selections are done across the world.