Adambots Automated Scouting Kit

[mmkbenbray]

Howdy, everyone!

Several of us on Team 245 have been developing a nifty scouting application for FRC, hosted on the Adambots website, that does OPR calculations and other event analysis on the fly. It's cross-browser compatible, and it works on most Javascript-enabled mobile devices.

Here's the link: Adambots Automated Scouting Kit

We've also made the source code available here.

AASK will begin displaying results for the Michigan State Championship as soon as the match schedule is posted. We hope to integrate global OPR calculations by Friday morning, which will be used in predicting matches and displaying information about specific teams. As of this post, those features have not yet been published.

I hope you all find it useful as the World Championship approaches! Please respond to this thread with any bug reports, suggestions, or comments you may have!

Below is a description of the tool, copied mostly from the website.

About This Tool

Below is a description of each of the calculations provided by AASK:

Rank: The team's rank at the event, in terms of Qualification Points, as reported by FIRST.
Autonomous Rating: An estimate of the number of points a team scores, on average, during the autonomous period.
Teleoperated Rating: An estimate of the number of points a team scores, on average, during the teleoperated period.
Climb Rating: An estimate of the number of points a team earns, on average, by climbing.
OPR: Offensive Power Rating. An estimate of the number of points the team scores overall, on average. This number represents the offensive utility of a team. Comparable to the OPR reported by other teams.
DPR: Defensive Power Rating. An estimate of the defensive utility of a team. May be interpreted as the number of points that a team takes away from its opposing alliance, on average.
CCWM: Calculated Contribution to Winning Margin. An estimate of the number of points a team contributes to the winning margin of its alliance.

Should My Team Still Scout?

Yes! AASK is meant only as a secondary source of information, either as a basis for comparison or to replace missing or faulty information. We do not guarantee that the results of our estimates will accurately reflect the capabilities of each team.

How Does This Work?

For each point category, our system solves a system of linear equations for the "average contribution" of each team per match. Each equation corresponds to a single team and expresses the total accumulated points earned by that team as a linear combination of that team's average contribution and the average contributions of every other team that has competed on an alliance with that team. Naturally, we represent the system of equations with a single matrix equation of the form Ax=b

• Vector b contains the aggregate point value (one of AP, CP, TP, or their sum) for each team.
• Each element A(i,⁣j) of matrix A represents the number of times team i has played with team j. Each element on the diagonal, therefore, is the total number of matches played by the team represented by that row and column. As a result, our matrix has the following properties:
  • It is symmetric.
  • It is irreducibly diagonally dominant.
• We solve for the vector x, which contains the average contribution of each team.

Because of the special properties of our matrix, we can easily find an exact solution when the matrix is nonsingular using LU Factorization (without pivoting!) followed by forward- and back-substitution. When the matrix is singular, we approximate the solution iteratively with the Gauss-Seidel method and display a warning to the user.

The Defensive Power Rating (DPR) for a team is calculated by iterating through the list of completed matches and using the calculated OPR values to predict the outcome of each match. For each match, for both alliances, the difference between this expected outcome and the true outcome of the match is credited to the defensive utility of the opposing alliance. For each team, we sum up these differences and solve a linear system similar to the one above using this new tabulated data.

The Calculated Contribution to the Winning Margin (CCWM) for each team is calculated by summing up the alliance score difference for each team for each match and solving our favorite system of linear equations for the x vector with these margins in our b vector.

AASK is dependent on the match schedule and rankings reported by www.USFIRST.org. Occasionally, the FIRST website will experience an outage, rendering our tool temporarily incapacitated. We have implemented a simple caching system, designed to reduce the frequency of such failures, but ultimately the status of the FIRST website controls the functionality of this tool.

[joelg236]

Holy crap! That is actually extremely impressive. Thank you so much for making such an awesome tool.

[Austin2046]

wow, very impressive, thanks a lot for putting this together.

[joelg236]

Just a thought, is it possible for you to make a page that combines all teams together? I wanted to do some predictions of crazy matches (IRI type stuff), but there's no regional with enough of those teams to do something like that.

[mmkbenbray]

Originally Posted by joelg236

Just a thought, is it possible for you to make a page that combines all teams together? I wanted to do some predictions of crazy matches (IRI type stuff), but there's no regional with enough of those teams to do something like that.

Absolutely, that's one of our plans for the immediate future. What makes such a task hard is the number of requests to the FIRST site necessary to gather the info for every team; all the processing is done client-side, so the number of requests to the FIRST site would be enormous, which is undesirable.

Luckily, my partner in this project came up with a better way to fetch and store so much information, and he began work on it this afternoon. We'll begin writing code tomorrow for global rating calculations and a corresponding match prediction tool, which we'll hopefully publish before the end of the day. (Last minute coding in the stands, as always!)

[orangemoore]

So cool

[hiyou102]

It's interesting to see the teams with a DPR of -24. It makes me wonder if that's a failure in the data or in the team.

[Grim Tuesday]

Originally Posted by hiyou102

It's interesting to see the teams with a DPR of -24. It makes me wonder if that's a failure in the data or in the team.

I suppose it just means that in matches those teams played in, their opponents scored more than usual.

'DPR' might actually be another measure of schedule strength!

[AdamHeard]

Will this be updated to include IRI?

Would it be possible to incorporate support for offline and unofficial events? The user uploads a table of match results and you calc off that?

Thanks

[Mr. B.]

Awesome job 245! I will make sure that my team's scouting and strategy teams see this link. This is very impressive, keep up the great work! Thanks again for a great run at the Palmetto Regional!

[JB987]

Any idea why so many of the OPR values differ so much from Ed Law's and FRC Tracker's numbers?

[Siri]

Originally Posted by JB987

Any idea why so many of the OPR values differ so much from Ed Law's and FRC Tracker's numbers?

I spot-checked a few teams and didn't find any differences. Are you sure you're comparing the kit with Ed's per-event OPRs rather than the World versions?

This is fantastic, Adambots! Thanks.

[JB987]

Ed Law's Championship V4, Archimedes tab (and FRC Tracker, Archimedes data) has 987's OPR listed as 106.9 while Adambots lists OPR as only 85 and change for the division. Several other teams are listed lower as well same and other divisions.

[Ed Law]

Originally Posted by JB987

Ed Law's Championship V4, Archimedes tab (and FRC Tracker, Archimedes data) has 987's OPR listed as 106.9 while Adambots lists OPR as only 85 and change for the division. Several other teams are listed lower as well same and other divisions.

There is an issue that was identified and discussed in another post. The problem arises when there are surrogate matches.

Currently, OPR is calculated using all matches and scores. Even surrogate matches for teams are included in the calculations. However when we calculate OPR for autonomous, teleop and climb, the total points used in vector b of Ax=b does not include surrogate matches, but the matrix A contains all the matches. This makes your team's auto, teleop and climb OPR smaller because you played 9 matches but only have scores for 8 matches. It is like you get zero for everything for one match.

Hence 106.9 for overall OPR is correct. 85.4 came from adding the auto, teleop and climb OPR and it is wrong.

This is one of my summer project to do. I have a few different ideas how to compensate for it, but I want to pick one that is most mathematically sound.

[JB987]

Thanks for the clarification, Ed! And please know that all of your effort is deeply appreciated.