How to effectively use/analyze day 2 scouting data?

One topic that came up on the Gettin' Picky: A Guide to Alliance Selections (Part 1: The Picklist) thread was how to best use data collected on day 2.

How does your team go about using data and adjusting teams on lists during day 2 at a district/regional? How do you make sure teams that really get it together for day 2 don’t slip through the cracks? How do you avoid getting burned by teams who fall apart?

On 2791, we keep a watch list of high-interest teams generated during the picklist session and make sure someone from our scouting leadership team 1) watched them in the match and 2) looked at the scouting sheet for that team in that match to make sure they get both a quantitative and qualitative look at the teams we’re interested in. This method generally works great for tracking the teams on the watch list, but sometimes falters on detecting new teams that we should consider.

One metric I’ve been toying with during the offseason is taking: (average game pieces scored in second half of their matches) - (average game pieces scored in first half of their matches) and seeing who has large/small changes over time. It seems promising, but has issues with the noisiness of averages, especially when defense enters the mix.

6 Likes

IMO There is a lot of data overthinking , in most scouting systems. We try to only collect useful data as to not overload scouts.

With a group of say 50 -60 teams or less at districts…the population and sample size is not large. Statistics to be valid require very large sample sizes. Dealing with say 50 teams and say 10 matches to rely on data pointing the way is problematic, add in “different alliances” for each team and the validly gets worse, a certain “favorable schedule” can alter the stats of quantitative data collected.

Qualitative does not relay on stats entirely and bodes well for days 1 and 2. Day 1 had the majority of matches, so an initial pick list can be made. Its easy to “verify” your qualitative data against , published stats on TBA etc. I almost always see a correlation in our pick list to those aggregated stats.

Day 2 likely adds a few extra teams that are "published stat " outliers. Our list captured almost all of the high stat teams just by watching them play the game, and tracking metrics we want to track. These metrics can change based on the game theme itself. By end of day 2 quals , we have a list of teams that help us. Plus, many notes of nearly all of those we are likely to face in eliminations.

Another key metric to watch is ranks (and win%) raising and falling by day 2 and look for the performance trend. Also confirming performance characteristics of specialists like in this past season “defenders”.

1 Like

This year, 1293 made a copy of the points-per-match data at lunch on Friday, then at the end of Friday, and put them in a spot on our spreadsheet. This let us see where the team stood in comparison to the current information (especially deep into Saturday morning) and spot a team getting hot.

This came into being after we completely missed Exploding Bacon at Palmetto, who gained nearly a full point at Palmetto between Friday night and the end of qualification rounds (going from 19.00 to 19.89, where the average team improved 0.29 points in the same time). We did save a copy of our scouting sheet on Friday night there, but we didn’t have an easy way to look at the numbers side-by-side.

Of course, this isn’t a panacea if you don’t look apply it correctly.

2556 was third-best cargo ship robot at Smoky (and eleventh in total scoring), all cargo. The Friday lunch average was a paltry 8.67 points/match, then shot up to a far more respectable 18.31 by the end of Friday. By the end of Saturday morning, they had tacked on another 2.45 points/match (almost a full cycle). We pushed them down our list because we were a pretty darn good cargo ship robot too (first at Palmetto, and tenth at Smoky despite serving up a lot of defense) and prioritized hatches on the rocket so we could cargo them, but then we shifted to a defensive role in the Smoky playoffs. Had the picklist meeting seen that decision coming, we would’ve made a very different picklist.

Suffice to say, 4265 pulled off the heist of the century getting them as the second pick of Alliance 4, and both of them (and MARS) totally deserve the banner. 2019 was the first and second time 1293 had been a captain in their 16 years (and even I had only had it happen once prior, at Palmetto 2007 with 1618), so we will continue to refine our process.