paper: 4536 scouting database BETA

[Caleb Sykes]

Thread created automatically to discuss a document in CD-Media.

4536 scouting database BETA by Caleb Sykes

[Caleb Sykes]

This is a beta test of a scouting database which calculates component calculated contributions (OPRs) using the data from the FIRST API. As this project is still in its infancy. Please report any bugs or potential improvements to Caleb Sykes (calebsyk@gmail.com). Each sheet currently contains data from a distinct week 1 event. Starting weekly on 3/21, a new database will be published which will contain data from all events up to that date.

Be extremely careful when using the individual defense crossings (columns J-Q on each sheet). At a given event, if a defense is chosen fewer times than there are teams at the event, a #NUM! error will appear. If a defense is chosen less than twice as many times as there are teams at the event, place limited faith in the numbers.

See the "instructions" sheet for more detailed information on what each category represents.

[Caleb Sykes]

I have just uploaded version 1 of this database. It is populated with all week 1-3 events.

In addition to adding the additional data, the two main changes since the BETA include:
An alternative calculation of eOPR (elimination OPR) is now included for each team. So there are now two eOPR calculations, which I have dubbed eOPR1 and eOPR2. Details on how these are calculated can be found in the "instructions" sheet. Although I have not verified this, I expect eOPR1 to provide better elimination predictions at weaker events where captures are more infrequent, and eOPR2 to provide better elimination predictions at stronger events where captures are more frequent.

A new "world results" sheet has been added, which allows for component comparisons for every team at every event in which they have competed. Be aware that this list will have duplicates for teams that competed at 2+ events. Also, don't compare individual defense crossing data unless you know what you are doing. For example, team 5114 has a drawbridge contribution of 1722968039259170.00. 5114 is not that good at crossing the drawbridge, this just means that the drawbridge was not chosen frequently enough at Midland for there to be meaningful results for drawbridge contributions. As a rule of thumb, you can almost always trust the rock wall, sally port, and cheval de frise contributions, but be wary of the others.



Remember, this project is still quite young, and there are very likely errors in places (especially since I have not yet automated everything, and have to do some copying by hand). If you see any errors, please let me know and I will look into it.

[Nathan Streeter]

First: very cool spreadsheet! I'm glad to have a resource that looks at the component OPR for pretty much every possible condition! It has all the usual OPR caveats, but it does seem useful for establishing some trends and making some comparisons.

As a sidenote, thank you to FRC HQ for making this data more available for capture. The API certainly provides much better data than the twitter feed over recent years.

I have some questions about the "units" of some columns... I'm pretty sure they're my initial guess for most of them, but I wanted to double-check.

For columns H and I (teleop Capture or Breach), I presume a "1" would indicate a successful Capture/Breach?

For columns J - V and AM (defense crossings), is "1" a single defense traversal (5 pts) or a weakened defense (10 pts, 2 traversals)?

Also, how are eOPR 1 and eOPR 2 calculated? What's the difference? They differ dramatically from the OPRs based solely on match scores.

[Caleb Sykes]

Originally Posted by Nathan Streeter

First: very cool spreadsheet! I'm glad to have a resource that looks at the component OPR for pretty much every possible condition! It has all the usual OPR caveats, but it does seem useful for establishing some trends and making some comparisons.

Thank you. I'm glad to see people are benefitting from it.

As a sidenote, thank you to FRC HQ for making this data more available for capture. The API certainly provides much better data than the twitter feed over recent years.

Agreed. Also thanks to Ether and team 2834, I have built off of both of their work, and this spreadsheet would not be possible without them.

I have some questions about the "units" of some columns... I'm pretty sure they're my initial guess for most of them, but I wanted to double-check.

For columns H and I (teleop Capture or Breach), I presume a "1" would indicate a successful Capture/Breach?

For columns J - V and AM (defense crossings), is "1" a single defense traversal (5 pts) or a weakened defense (10 pts, 2 traversals)?

Also, how are eOPR 1 and eOPR 2 calculated? What's the difference? They differ dramatically from the OPRs based solely on match scores.

The "instructions" tab has more detailed descriptions about each category, let me know if anything there is unclear and I can revise it. I'll summarize here anyway though.

"teleop Tower Captured" and "teleop Defenses Breached" both have units of ranking points. A 1 in either of these would indicate that the given team contributes an average of 1 ranking point each match.

All categories that have "crossings" in their name have units of crossings, not weakenings. That is, a 2 in any of these categories would indicate that the given team contributed 2 scored CROSSINGS over this defense each match.

eOPR1 and eOPR2 are my rough attempts to compensate for different scoring methods in quals and elims. Since breaches and captures provide points in elims, but not in quals, "normal" OPR probably does a poor job predicting elimination match scores (although this is as of yet unverified). eOPR1 essentially makes boulders and crosses scored in quals worth more, and eOPR2 takes breaching/capturing contributions and assigns them point values, and then adds those to the "normal" OPR.

[Caleb Sykes]

Week 4 data has been added.

Additionally, I deleted the unnecessary whitespace that was beneath most of the event sheets' data. This will allow sorting to make much more sense and cause the scroll bar to be more appropriately sized.

Also, I hadn't realized that excel saved the position of the last cell selected, which is why seemingly random positions on each page were previously selected upon entering them for the first time. I have now selected the top-left corner cell on each sheet.

As always, I appreciate feedback and/or error reports.

[Caleb Sykes]

Week 5 data has been added.

I will include the data for the Western Canada regional in the week 6 update.

[Ben Martin]

Thanks for producing this every week! It is very interesting how the results from this data aligns very closely with scoring averages by type in our scouting data (not a perfect match, but very close)--we'll definitely be using it for Championships scouting.

[Caleb Sykes]

Week 7 data has been added.

Per request, I have also added a "championship preview" sheet which contains data on the best event (by OPR) of every team registered for championships as of 5PM CST on 4/18/2016. There is no new information on this sheet, all data are copied directly from the "world results" sheet. I am not planning to release updates if/when the championship team list changes, so you will have to update this sheet yourself.

If someone could check the data from the Michigan State Championship against scouting data to see that they roughly correlate, I would appreciate it. When I originally made this database, all of my calculations assumed that no event would have more than 100 teams or more than 200 matches. Thus, I had to modify a few things to accommodate MSC, which makes me nervous that I may have introduced one or more small errors somewhere.

Unless someone notices an error, I will not be releasing another update until after championships.

[Caleb Sykes]

Originally Posted by Caleb Sykes

If someone could check the data from the Michigan State Championship against scouting data to see that they roughly correlate, I would appreciate it. When I originally made this database, all of my calculations assumed that no event would have more than 100 teams or more than 200 matches. Thus, I had to modify a few things to accommodate MSC, which makes me nervous that I may have introduced one or more small errors somewhere.

These values have been verified to be correct. Ether ran calculations independently that provided results which matched the results in this database.

[Caleb Sykes]

I have ran the calculations for championships, but there seems to be a bit of a discrepancy between my data and what is posted on TBA. Could someone independently run OPR calculations for Carson to see if the error is more likely on my side or on TBA's side? I will be investigating more on my own, the discrepancy seems like it might be related to qualification match #1.

Here is Carson's top 15 OPR according to TBA:

1024	67.95
868	64.40
973	59.73
225	57.22
2052	57.07
610	56.95
2122	56.15
2590	54.44
5895	54.13
41	52.82
2067	51.82
3824	50.56
2137	49.56
3538	47.72
2474	47.07

Here is the top 15 OPR according to my calculations:

1024	67.75374586
868	63.84195886
2122	59.32732553
973	59.02368093
225	57.49356224
610	56.71770781
2052	56.52062215
2590	54.57781003
5895	53.93177915
41	52.86710115
2067	51.99822564
3824	50.64961634
2137	49.26312626
3538	47.7814427
2474	47.15846249

[Ether]

Originally Posted by Caleb Sykes

Could someone independently run OPR calculations for Carson

1024	67.94699488
868	64.39937013
973	59.72708602
225	57.21656682
2052	57.06823258
610	56.94982187
2122	56.15330503
2590	54.44241751
5895	54.13186804
41	52.82497403
2067	51.82418186
3824	50.55946533
2137	49.55923262
3538	47.71879089
2474	47.06784238
1918	47.06371556
4028	45.59624045
904	45.41944435
1718	45.29367738
1625	43.90561045
4362	43.58606725
2996	43.53157842
2655	43.28570403
525	43.20450209
2403	43.1175526
2771	42.92284815
3970	42.72891288
135	42.57181239
5907	42.13118592
1987	41.79576813
4264	41.54625885
2486	41.10297812
3688	41.04522675
5167	40.89402704
319	40.27903932
4131	40.17573866
1619	39.56191031
2485	37.35922029
1533	36.90397632
1137	36.73476528
6098	35.99548457
233	34.91407358
6144	34.34974789
5663	32.20977958
5913	31.95709232
60	31.8850688
1156	31.25624534
1258	30.90491415
5084	30.58091357
5332	28.5437442
5454	28.51064153
1126	28.40910065
2761	28.31012272
2445	28.30861263
1159	26.93097019
2202	26.2979391
5879	25.36524886
5712	25.16147429
6025	24.98551622
2978	24.36624945
3352	22.86652503
4592	22.59269553
11	22.58569118
4121	22.17481602
296	22.09641455
1939	21.41832877
4026	20.96318342
4135	20.9368346
5572	20.35016881
3021	20.24024128
2526	19.82585398
5897	19.74217565
746	18.84443104
51	17.41725192
1369	7.854537579

[Nathan Streeter]

Is a post-CMP update coming? :-)

[Caleb Sykes]

Originally Posted by Nathan Streeter

Is a post-CMP update coming? :-)

I normally update within a day of the 2834 scouting database being updated, because the "world results" page uses event information from it. So if the answer to this question is yes, I will update within a day after the 2834 update. If there will be no update to the 2834 database, I will have to rework some things, so it will take me a bit longer.

If the 2834 database isn't updated by Friday, I'll rework my database and publish an update no later than Saturday.

[Caleb Sykes]

I have uploaded a final update to this database. This update has tabs for each of the championship divisions. I have also removed all of the championships preview tabs.

I hope everyone who downloaded it found it useful. I am planning to maintain this effort in the upcoming years. I am also planning to spend some more time developing the interface to reach the level of the 1114 and 2834 databases in future versions. I will also be looking to develop new metrics next year, depending on what the game is and what data the API provides. Keep an eye out for a thread near the end of build season next year where I will be asking for feedback on what everyone would like to see calculated.

Thanks to teams 1114 and 2834 for providing my inspiration for creating this. Special thanks to Ether for providing the CSV files on which my entire database is founded. None of this would have been possible without him.