2020 Scouting Database/Event Simulator Metrics

Hello everyone! This post is confirmation that my scouting database and event simulator will be returning yet again for 2020. If you are unfamiliar with these, links to the 2019 versions are in the names above.

FIRST has already released a preliminary version of their 2020 year-specific api documentation. This nominally remains subject to change, but we can probably expect minimal changes. Thanks FIRST for getting this out so early this season! You can find the documentation here, scroll down to “Response Details 2020” under “Score Details” to see exactly what we have available to us. You can also find parallel documentation in TBA’s api docs under the “Match_Score_Breakdown_2020” field here.

Here is the 2019 version of this thread as well for reference.

FIRST is being kind enough to provide essentially all of the raw scoring data, so now we have to find the best way to turn these into useful metrics. I’d like to use this thread to bounce around ideas for metrics and to make sure I’m capturing as much of the raw data as possible while at the same time not adding anything superfluous. Please provide feedback either if you think I’m missing a metric, or if any of my suggested metrics seem utterly useless. Remember I can only work with the data provided to me in the linked documentation, so don’t ask me for a metric like “missed power cells”.

With that in mind, here is my first pass at a list of metrics I’m planning to develop. “CC” means calculated contribution (or component OPR), and “rate” means I’m just counting the number of times something happens for a specific team and dividing by all of their matches:
winning Margin Elo
total Points CC
unpenalized Total Points CC
winning Margin CC
win CC
ranking Points CC
auto Points CC
teleop Excluding Endgame Points CC
endgame Points CC
total Scored Power Cells CC
shield Energized RP ILS (more info on ILSs here)
shield Operational RP ILS

auto Init Line Points CC
auto Init Line Rate
auto Cell Points CC
auto Cells Bottom CC
auto Cells Outer CC
auto Cells Inner CC
auto 3-pt Efficiency CC (define as 0 for matches in which no high goals are scored in auto)

teleop Cell Points CC
teleop Cells Bottom CC
teleop Cells Outer CC
teleop Cells Inner CC
teleop 3-pt Efficiency CC (define as 0 for matches in which no high goals are scored in auto)
control Panel Points CC
activated Stages CC
stage 1 Activated CC
stage 2 Activated CC
stage 3 Activated CC
stage 3 Is Blue Rate CC
stage 3 Is Green Rate CC
stage 3 Is Yellow Rate CC
stage 3 Is Red Rate CC

parking Points CC
parking Rate
hanging Points CC
hanging Rate
level Rung Points CC
level Rung Rate
level Rung With 1+ Hangs Rate

shield Energized RP CC
shield Energized Completion Percentage CC
shield Operational RP CC
shield Operational RP Completion Percentage CC
total 3-pt Efficiency CC
no Show CC
no Show Rate
foul Count CC
tech Foul Count CC
foul Points CC
fouls Drawn CC
tech Fouls Drawn CC
foul Points Drawn CC

My thoughts:
I wish that we could see how much capacity was scored in each of the stages. At the very least, it would be nice to know how much capacity is remaining on the last stage. Unfortunately, this is not provided to us at the moment.
I thought about not including color information as these should just be random and evenly distributed, and therefore not useful datapoints. However, I had the exact same thought process in 2018 for the plate assignments, and when I opted not to include those, it took until week 3 for someone to realize the plate assignments were not evenly distributed. So I’m leaning toward keeping these in so we can more quickly identify something like this if the colors are for some reason not evenly distributed in 2020.
One of the most interesting dynamics this year is that the inner goal is surrounded by the outer goal. Teams will be interested in knowing how many successful “high shots” get into the inner goal. I’ve already committed to building a tool do help out with this when combined with scouting data. What can we do with the publicly available data though to better determine a team’s 3-pt vs 2-pt scores?

  • My first thought is to calculate 3-pt efficiency CCs similar to what I did last year for cargo efficiencies. For example, define “auto inner goal efficiency” to be (auto inner goals)/(auto inner goals + auto outer goals) if (auto inner goals + auto outer goals) > 0, otherwise 0. Plug this efficiency into the standard CC equation and you’ll get a crude estimate of how frequently a team scores in the 3-pt versus the 2-pt goal
  • Another idea would be to find the aggregated 3-pt rate over all matches, essentially just summing all 3-pt shots in the event and dividing by all high goal shots. This would require some new code in my simulator, but would be doable.
  • Another option would be to set up a process to calculate 3-pt percentages in two stages as outlined by Eiim here. I have concerns that this methodology will blow up if there are teams at the event that don’t score many balls, but that threat may be overstated, difficult to know without testing.
  • Open to other ideas as well on this front, seems like there should be a better way to get at this kind of insight that I haven’t thought of yet

shield Energized Completion Percentage will end up being just 1/3 of activated Stages CC, so I’m considering dropping it. Adds some formatting consistency though.

Let me know what you all think! I’m happy to take feedback.


This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.