Team 1706 Scouting Summary

This year has been a massive leap for Team 1706. We made it to Einstein’s for the first time in our team’s history, and we went undefeated in a world-class division. One of the many reasons behind our success has been the scouting data. I am the head scout for 1706, The Ratchet Rockers, and over this most recent game, I have learned and created a lot.

The Scouting App

Our data is generated by a group of scouts using the Android application that I developed in-house. This app collects data, such as the scout’s name, the team number, the round number, and the alliance. Some other raw data is standard, but we also collect missed shots and auto accuracy, which plays a massive role in how our team selects alliance partners. We allow scouts to write notes to help us know why the data may seem different, such as a team playing defense in one specific match. I also added some Easter Eggs to keep things interesting for the scouts, but the Easter Eggs are so simple that they do not cause any noticeable data inaccuracies.

The Data Exporting

Our app exports a .txt file onto the Android device’s hard drive in a specific folder. From there, the data is grabbed by a Bluetooth file transfer program or a direct wire transfer to a host computer. We use a python script to convert the text file into a .csv form. Following that, it is a simple copy and paste into our data analysis tool.

The Data Analysis Tool

This raw data is then converted into averages and then parsed down to get the most crucial data. While that is the entire function, I added a lot more behind the scenes. To go through the tabs, the first one that is not just data or averages is the Match Schedule Strategy.

This tab forces us to look at a Google Script to understand the functionality. This script exports a team list into the setup tab as well as their match schedules. This is accessed through The Blue Alliance API and uses an auth key to access and download data from the API. This is also possible with =IMPORTHTML inside Google sheets.

Changing the match number on the first tab will cause all the teams to update from the setup tab, resulting in a match strategy that is often within 8-10 points per game (unless something catastrophic happens). The next tab is a Team Parser, which allows for the input of any team number, and it will give back the match by match data of any team, a trend line of that team and how they played across matches, and their average data, along with a rank, of each of the categories of scoring.

It also shows a list of comments, match by match, to see why data may be funky for a match (this is why we encourage our scouts to take sufficient notes). The next tab is the Picklist tab, which goes through the list of teams that we are contemplating inviting to our alliance, and then we compare them on all of their average stats.

This tab is not used as often, since we have the Team Parser, but it is good for a nice quick look at some data. The next tab is my favorite. The Team Comparer allows for the display of a radar chart (thing of an ability chart or spider chart) showing off a team’s strengths.

A good team will have a large area, and a smaller area indicates a team that is a little worse. The best part about it is that it can compare teams on each of the 5 traits at the same time in a comprehensive form. The next tab is a standard Match Strategy, which helps during eliminations as it allows for the custom importing of teams into the boxes, making the customization choices include the teams of the opposing alliance.

The next tab is a charts and graphs tab, a sort of visual aid so that the rest of the team can easily see what data matters most. This is filtered down to the top teams so we don’t get overwhelmed by a bunch of lines.

The bubble chart on this page is convenient to figure out the best cargo scorers by looking at accuracy vs. shots, and then for fun, I plotted all of the points on a standard graph and got the line of best fit, which looks like a natural log curve.

Next to that tab, there is the Driver Info tab, which is the data that we collect and send to the drivers before a match, alongside the Match Schedule Strategy.

The Validation tab is still a WIP and was intended to verify from TBA API what data was incorrect.

Using and Sending the Data

Having all of this data is nice, but making the data do anything involves sending it down to the pits. Before a match, our scouting mentor and I create a match strategy that takes the data we have collected thus far and we write a comprehensive plan from auto to teleop to the endgame. This being said, the data gets more and more accurate as the day goes on, so we scout practice matches beforehand to get a decent baseline. Once we scout enough qualifiers, we remove the practice match data points, since they are mostly inaccurate. In qualifiers, our goals were to get all of the ranking points, so we determined that climbing would be a big component of that, and wrote our strategy around that aspect of the game.

24 Likes

While this was posted by an alumnus of the 1706 team, this post was created by @xanderschulte1706 , the lead scout for team 1706.

4 Likes

Saying that a lot of work went into this is a terrible understatement.

Wow!

I am impressed, and the value is unmistakable. Scouting wins competitions.

2 Likes

I want to give a huge shoutout to Xander and the rest of the scouting/strategy students on 1706. It didn’t take long for me to learn that I just needed to suggest a half-baked idea for a new feature or way to view the data and they’d quickly come back with it fully implemented, exceeding my expectations for it.

(I’m just a little bitter that the only feature myself or any mentor coded – the data validation – wasn’t used due to inconsistent climb data and the cargo counting sensors :slight_smile: )

There was one team that used this google sheet to analyze data collected from one of the popular public scouting apps this past season. If there is interest, we’ll try to get it completed and fully tested by week 1 next season so other teams can use it to analyze data.

10 Likes

I absolutely love your guys’ system! Especially the ability spider charts and your ability to calculate the expected auto points for each different auto that a team could run. This is something that we will 100% be implementing next year!

@XaulZan11 Yes Please! We’re working on our own validation system so I’d love to collaborate and discuss with you for next year.

6 Likes

Since we’re on the topic of shout-outs, I really gotta hand it to you John, I was absolutely dumbfounded by the depth of knowledge you and some of the students had about every team in our division. Going down the list, the level of detail you knew off the top of your head felt like magic. You are an encyclopedia and I think all the citrus mentors will be talking about it for years. Looking forward to getting to work with y’all in the future, thanks again for taking us along the ride this year, it was great.

13 Likes

Huh… Would you look at that @CanadianDoggo :wink:

1 Like

I’ll make sure to tell Dan to get right on that for next year. After he learns Java of course :laughing:
I definetly agree they’re pretty cool way of visualizing that much data

I am really impressed on the spider charts that u made in google sheets. That is something like to look into at a further date
I also like your match strategy function where you can predict an upcoming match in google sheets. I have made an extremely similar search function on my own accord.
Overall, I am intrigued by this system.

2 Likes

Thank you for your comment. I intend on posting the sheet’s functions soon, but I want to recollect some data taht got lost and make sure that our scouting tool is in perfect condition before I send it out.

Not to take anything away from the work you’ve all done, but after we were back from champs I sat and dwelled on “what ifs” by going over the recorded matches of all the Einstein alliance offensive robots and a few other outliers that haven’t made it there and compared the offensive production or those alliances to ours to maybe increase my internal feeling of missed opportunity (because who doesn’t like to do that), and the stats you have regarding 1678 caught my eyes immediately since there’s a big difference between your numbers and mine - specifically during the first few matches they had.
It peaked my interest so I recounted all of their matches to see if I was maybe mistaken - and I did find a few mistakes, but overall it seems like you have quite a lot of missed high goals in their case:
image

You haven’t posted a breakdown of your teams’ numbers but I did once again compare your total number to my post event scouting and you’re selling yourselves short as well - I got an average of 64.4 and yours says 57.25 (when adding taxi points):
image

All of this once again isn’t to bash down any of your scouters - it mainly got me thinking about how inaccurate our scouting might have been as well, I’ll have to compare our database to my results and get back to you with this info, but it begs the question of whether we should all change our scouting tactics to using streams and match videos instead of doing it all live to get better and more accurate info out of it, at least after establishing a general idea about teams’ performances…

3 Likes

Yeah, we noticed data irregularities on Thursday night of the championship when starting the picklist process – it’s actually the main reason we haven’t posted the full google doc. We took a smaller group to the championship meaning the scouts were likely overworked over those long days of matches. (And teams like 1678 score really fast so it can be hard to keep up sometimes). I think the scouts understood the importance of scouting (1706 had a positive scouting culture before I joined) and put the effort into it, so it comes down to better training, finding ways to make their job easier and better processes to ensure data accuracy.

Regarding using match video / streams instead of live / in-person, I personally really prefer match videos as you can re-wind, skip ahead, or watch at 1.5x speed. I often watch match videos after our picklist meeting on Friday night of potential 3rd robots or where my opinion significantly differed from the group’s or data. But, I’d be hesitant to depend on match video since that adds another point of failure and we really depend on the live scouting data to inform qualification match strategy.

Reviewing match video could be a good option for data validation, however. Back in 2017 on 1732, we compared our scouting data (total number of cargo, hatch panels per alliance) with the data from TBA API and flagged matches where there was a significant difference. It became tedious as we waiting until the matches were over on Friday to review. But, if you assigned a couple of scouts to review flagged matches during the day, a system like that could really improve data accuracy. Maybe I shouldn’t have been scared off by the cargo sensor issues and finished implementing data validation this past season…

5 Likes

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.