This past season we started having pre-match data for drive team. Basically, we scout using an app I made using MIT App Inventor. The data collected goes into an excel sheet and is then copied into a google spreadsheet after a couple of matches. There are a bunch of different formulas and different sheets in the spreadsheet. The data from the app is pasted into a input sheet. Each team has their own sheet where the data then goes. Then a bunch of averages and percentages are calculated and then goes to an overall sheet. I then added a sheet where specific data like, do they dock and engage in auto, how many GPâs do they score, and their âdriver skillâ (observed by scouters). This data is laid out to make a little card. We then have a little printer that prints this little data card for the drive team and strategy team. I am the scout sub-team lead and the only one that knew how this works which was very stressful to handle at competitions and as we all know, Nothing goes right at competitions! I know it sounds very complicated and is very hard to explain.
Overall questions I have:
Does this seem like a good system to keep up?
Is it worth trying to make this system smoother and go better?
Donât know exactly what is on your other sheet with functions and stuff but make sure you have all your bases covered on what you want your match strategy to be in each phase of match you want to be able to get everything you want to know about auton, teleop and end game. And if you want to get into specifics when looking at your title cards to figure out whose the best for your picks definitely look at more in-depth data. When we do strategy for qual matches we just talked to the other teams before a match not really pulling from The scouting data because we only really use that for alliance selection. It also might be easier to just paste it directly into a Google sheet instead of from an excel sheet to a Google. You also want to make sure that your drive team is not fully relying on those little cards because your printer will inevitably run into errors or break at some point. I significantly recommend a training your scouts on your scouting app before comp for the most accurate data possible. My Google our team uses a scouting app where we input data to a form that gets turned into a QR code and then we scan that QR code from my computer that automatically puts it into a Google sheet, then all of that data gets turned into lots and lots of functions in a tab and then we make our pick list based on that data. Doing drive coach recently I feel like itâs much easier to talk to the people before Iâm at to get information from them instead of relying on a card from scouting data
Yeah thatâs an amazing system that I myself have been trying to push for my team for the next few seasons. Just be careful about your metric around driver skill as this is heavily subjective and you should probably do some scouting training so Scouters know what a âgoodâ driver looks like. I was never a fan of driver skill or defence rating as it is hard to gather data so Iâve just opted for a simpler scale of 0-2 and its very general. But in short the Auto data and especially GPâs and Endgame success are important.
Its always worth making something better, you can add the features to link to match that had endgame climb fails or display what match that was so you can know why it happened because numbers are good but knowing why is more important, numbers DO lie sometimes.
I wanted to look at more so for important matches or when I had time I did, but we normally just looked at GPâs and who will climb so when we went to a strat meeting we knew what to tell teams to do or to allocate what teams can do best based off GPâs and what endgame should look like. I looked at What GPâs werenât scored in auto for a out of auto plan, who will probably play D on the other team and have average score attributed per team to know on average is this match is close so I know how risky I need to make it, If its close DONT TELL YOUR DRIVE TEAM just tell them what they need to know and thatâs it you can tell your coach but not your drivers they will get in their heads and not play their best.
Keep it up, scouting is so much fun and there are so many things that are yet to be learned by everyone. My tip is for your team to take it seriously. I you have bad data or like put the students that donât want to be there you will get bad data, make it fun. if you need to and just make the scouting culture a place where scouters want to be there.
We use our own custom app, we also our brains and intuition lol. We just played with it all season and added everything we found helpful to know, having useless facts is pointless so we had minimal features and added the ones we realized we needed.
While it is hard to gather objective data on driver skill, in my experience, its been very helpful to know. Knowing if a particular team is poor at driving or poor under defense, and more specifically, what they donât respond well to is helpful for determining if and how defense should played. Similar things can be said for how a particular team tends to play defense. Knowing how to take the opposing alliance out of their comfort zone is one of the most important data points in my experience that scouting can collect and can completely change the outcome of some matches.
One system (which my mind seems to attribute to 2056 if I recall correctly) is to have your scouters have a discussion after the match and rank an allianceâs driving abilities in order of their strength. The best-driven robot gets a 3, the second best a 2, and worse a 1. This data gets aggregated over the event. This removes the need to quantify how well one can drive and rather quantifies how well one drives compared to others, which is as useful a metric while being easier to scout.
I have made a variety of drive team reports for pre-match over the years on different teams and for different drivers remember:
Drive team has a lot going on. Seemingly small things like physically getting from point A to B take time
Drive coach (or whoever is calling the shots) is a managerial role, not a technical one. The people and the overall event experience come first. There have been way too many stories on CD about âThat jerk-wad drive coach made my students feel like Cr*pâ
With that in mind very pointed information that is useful to everyone and will not cause issues is key. Some info can be shared some cannot. The idea is to give a little more info so the honor system isnât the only thing at play here.
With three different drive coaches on what I would consider to be relatively good teams only the following has been really necessary for the amount of time they have to analyze info, irrelevant of season:
Auto start positions and auto gampieces
Preferred location to get gampieces
How many teleop gampieces are scored
Specific short notes for that team (i.e. drops gampiece if rammed head on; driver likes to lock bumpers and get into pushing match; human player can launch pieces midfield; robot always clears out midfield pre-stage pieces in auto from left to right; etc.)
Anything beyond the above is wasted effort in my experience. âDriver skillâ is subjective, situational, likely to change, and there is no guarantee there is one driver at a comp. The high quality short notes REALLY matter in shaping strategy.
1678 had a version of this a few seasons ago, 111 was running it last year, not sure off the top of my head who was playing with it âfirstâ.
There is a lot that can be done with ranking recovery from limited pairwise comparisons, both in absolute analytic terms and in general âaverage rankâ terms.
Here are a few suggestions I would give based on the data viz courses I have taken in undergrad and grad school. IâllLimelight, an integrated vision coprocessor include a graphic that shows how I would change it over time:
Original:
Avoid Encodings and Abbreviations - While it may seem to save space and time, youâre drive team may need to show this data to other teams on the alliance who need to understand the data. You want your data to be as clear as possible.
Remove Alliance Color - Your alliance color may seem important, but alliance color actually doesnât affect your data. This becomes chart junk that isnât necessary for the graphic. We can add color back in later as an extra way to encoder information.
âBut Carl! We need the color to let our alliance partners easily determine which numbers are ours and which arenât.â Donât worry your little lighting bugs, I got you covered in point 5.
Reorder Teams - Instead of ordering by alliance color, which doesnât actually have a bearing on the data, instead put your alliance on the top, and your opponents on the bottom. This allows your drive team to easily see which is their alliance versus their opponents.
The ordering of the 3 teams within their respective groups can be based on a metric such as a robot strength metric. That way you can redundantly encode some data within your table and better help your drive team understand their strongest opponents.
Standardize the Data points - While not always possible (such in driver skill), you may put everything in terms of âpointsâ so itâs easier to compare values. This differs a lot from game to game, so sometimes just doing game pieces is better.
Add a Scientific Color Scale - Since we can kind of suck at actually understanding numerical data, it can be beneficial if we redundantly encode this data with color schemes. There are a lot of studies that have gone into what types of color maps allow for people to accurately understand data. Here are a couple of good ones. Iâm going to parrot one of my professors and say that Virdis is a pretty great one. Here are some images of different schemes below compared to rainbow:
I was able to approximate Virdis pretty well but using Google Sheets conditional formatting rules and inputting the below values from min, 50%, and max.
If you are able to accurately guess the top end of scores for these values, you can make a unified color scheme for the entire event, which would allow the drive team to start to get an intuition of a teamâs strength compared to the entire competition instead of just the other robots in the match.
Hopefully, this adds some more to your visualization. I tried to stay within the confines of a matrix-based table without diving into charts or other visualizations. Let me know if you have any further questions!
A lot of the things I wanted to say have already been covered, so Iâm just gonna link to other CDChief Delphi threads I think you should read if you havenât yet!
Firstly to the op, data visualization specifically in a match per match scope is what I consider to be a fundamental must have for any team serious about strategy, so I think you are on the right path!
We view data in this view for every match planning session for three main reasons:
To gather an approximate strength of each alliance letting us know how close we expect the matches to be.
A quick at a glance view of robots strengths/weaknesses/preferences of game actions (no scroll, see everything on one page)
How close we are to achieving a ranking point
Here is an example of our 2023 system (which is far from perfect, but can give you an idea of data layout)
This whole system is getting revamped for us actually right now, but there is a lot to like here.
Since these are our specific teams goals, we pretty universally divide up our view by game piece and location scored. Game piece division is important to be able to see goal 2 for our system. From the above image you can clearly see who is scoring where, and what type of game piece when.
A big part of this specific view is the summation of what each alliance is capable of doing on the bottom. we know approximately how many game pieces the alliance is capable of scoring, as well as what rows that exist.
Going into this match just some examples for things strategically we would say:
We can see that we are slight underdogs in this matchup, both in total # of game pieces scored, as well as scoring locations.
From these numbers we can see red alliance is super cone heavy, so contesting any cubes in neutral zones is beneficial to preventing them from creating links is beneficial.
We are a bit cube heavy, but the vast majority of that comes from a low only cube scoring robot, so we will want to make sure we are completing links low to maintain optimal piece to link ratios
We know based on what an ideal cone to cube ratios (2:1, and they are more 1:1) that 5719 is much stronger at cones than cubes, so we can have them primarily focus on cones, and us cubes to play to their strengths
RP wise we should have a decent buffer due to the volume of scoring by both alliances, we just need to make sure we finish out links efficiently.
Now obviously when we are making strategic decisions for this match these examples are just scratching the surface of the decision-making that we go through for an average match plan for us, but you can get the idea of the utility here.
All this relates back to this possible suggestion, which most of this analysis/takeaways wouldnât be possible with a sum of points. I do think that there is some value in including summed extra points to get extra context for the match, and thatâs something that would be included in an overhauled version of this particular view.
There is a balance to this as well. Making things readable without making things confusing is a bit of an art. IMO find what works best for your team.
I believe 1678 was the first to really pioneer the system. I think itâs reasonable to implement, but the key part is actually having the backend to make use of the data. Donât scout things that you canât adequately use!
â
Also the big one that Bryce posted is utilizing the FIRST API for things. Minimize your scouted data points, we only count game pieces on our team, and pull the rest of the data from the api. Yes sometimes refs mess up attributing things to teams, but trust me⌠they mess up a lot less than 99% of scouts do. The api is a powerful tool that you can leverage to validate your data and get a good idea of how well you can trust your scouts. We have been using api data to supplement/validate our data since 2020, and every year we get new ideas on how to integrate it into our scouting system to specifically tell us how good our data points are, and what kind of error is in that.
In terms of what and how much data to include for the drive team, I think that depends on your specific team and isnât a one size fits all type of thing. Some teams have scouts collect the data and pass the ârawâ data to the drive team to strategize upcoming matches. Others have their scouts analyze the ârawâ data and send the drive team an already formed match strategy.
Lots of good info and I agree with most things, but this is one that I canât get onboard with. Specifically for Auto Docked and Engaged, average points leaves a lot to be desired. Does a 8.0 average mean they docked every match (which is good as it helps us get the ranking point!) or does it mean they engaged 2/3rd of the time (which isnât good as there is a 33% chance weâllLimelight, an integrated vision coprocessor need a triple engage for the ranking point)? A better solution, in my opinion, is the percentage of matches they docked or engaged in auto. An even better solution, however, is to record docked/engaged attempts and display the success rates (# of successes / # of attempts).
Yes, this is very true! We do a lot of reflection and work with our drive team and strategy. I was just interested in what other teams do as this is only my first season doing scouting! Thank you for your reply!
This is all very true! My spreadsheet covers everything we look for and I am always upgrading it and adding more! Your guys system seems pretty reliable and efficient! I am looking into QR codes and stuff but I donât think the program we use is able to do it. Thank you for your response!
Thank you so much! This is my/ teams first app. We are pretty new as a team, this is only our 3rd season. This was my first build season and I became the scout lead. Drive skill is just a basis, like an estimate and isnât fully relied on but it is fully true that it is not fully subjective.
Improving is very important to our team, which is why I made this post! I wanted to see how other teams operate and if anyone had any ideas! Numbers do lie sometimes, but each team has multiple matches so if something is off I can sometimes figure out what happened and if it was just a typo, which happens a lot!
We have a pretty similar pre-match plan. We mostly look at the autos, gps, and endgame, and make a plan for each point. This has worked very effectively for us! And we try to keep the pressure off our drive team. The operator and driver are the kings/queens of the day, especially our driver! We really appreciate them! it is a lot to handle!
Scouting is very interesting and I am so glad I had the perfect opportunity to do scouting! It was an amazing season and I love that it can grow so much! I really hope to improve in the upcoming 4 years of high school in front of me! (I started in 8th grade) Our team definitely has some issues with bad data, so I am trying my best to make it fun and will definitely keep it in mind next season!
This is really cool! Like I already mentioned, we use MIT App Inventor, where I basically designed and programed an app. This app is downloaded on 7 tablets, as we an extra to switch out with other tablets during competitions. I am very proud of the app I made this past season! It was my first app and really my first programming! I learned a lot from my mentor and on my own! I was only in 8th grade this past season so what I did was pretty huge for me! I have gotten a lot of support and hype though, which I really appreciate! I am super glad you replied with this awesome information as I am looking to grow in my scouting knowledge! Thank you for your post!
The very first time I presented color-coded data at a meeting as a professional engineer, one of the very senior engineering managers looked me dead in the eye and said âIâm colorblind. Which line is which?â
And I never made that mistake again
Color Oracle is a helpful colorblindness simulator that shows you what your computer screen would look like with different kinds of colorblindness. Itâs also helpful to put together a decent palette in Excel (or wherever youâre visualizing data), for a smoother workflow.
I wonder if it would be effective to designate one experienced scout to be responsible for evaluating driver skill, and nothing else. Itâs hard to make qualitative evaluations while also keeping meticulous track of scoring; at the same time, you donât have to watch every team in every single match to form decent impressions of driving skill.
We have super scout(s) who make qualitative notes about every team and itâs super helpful. Typically we pick top students who maybe werenât ready / didnât have space to be on the pit crew or drive team but deserve a larger role at competition. So they take really good notes and weâre fortunate where we now have a strategy group who handles the pre-match discussions and those students are usually involved in that process.
Ideally, I look at rolled up data similar to what the OP posted (several people gave specific pointers on things Iâd change) and I reference that super scout data because itâs usually really helpful and at times entertaining to read.
