paper: 4536 scouting database BETA

[Caleb Sykes]

Thread created automatically to discuss a document in CD-Media.

4536 scouting database BETA by Caleb Sykes

[Caleb Sykes]

This is a beta test of a scouting database which calculates component calculated contributions (OPRs) using the data from the FIRST API. As this project is still in its infancy. Please report any bugs or potential improvements to Caleb Sykes (calebsyk@gmail.com). Each sheet currently contains data from a distinct week 1 event. Starting weekly on 3/21, a new database will be published which will contain data from all events up to that date.

Be extremely careful when using the individual defense crossings (columns J-Q on each sheet). At a given event, if a defense is chosen fewer times than there are teams at the event, a #NUM! error will appear. If a defense is chosen less than twice as many times as there are teams at the event, place limited faith in the numbers.

See the "instructions" sheet for more detailed information on what each category represents.

[Caleb Sykes]

I have just uploaded version 1 of this database. It is populated with all week 1-3 events.

In addition to adding the additional data, the two main changes since the BETA include:
An alternative calculation of eOPR (elimination OPR) is now included for each team. So there are now two eOPR calculations, which I have dubbed eOPR1 and eOPR2. Details on how these are calculated can be found in the "instructions" sheet. Although I have not verified this, I expect eOPR1 to provide better elimination predictions at weaker events where captures are more infrequent, and eOPR2 to provide better elimination predictions at stronger events where captures are more frequent.

A new "world results" sheet has been added, which allows for component comparisons for every team at every event in which they have competed. Be aware that this list will have duplicates for teams that competed at 2+ events. Also, don't compare individual defense crossing data unless you know what you are doing. For example, team 5114 has a drawbridge contribution of 1722968039259170.00. 5114 is not that good at crossing the drawbridge, this just means that the drawbridge was not chosen frequently enough at Midland for there to be meaningful results for drawbridge contributions. As a rule of thumb, you can almost always trust the rock wall, sally port, and cheval de frise contributions, but be wary of the others.



Remember, this project is still quite young, and there are very likely errors in places (especially since I have not yet automated everything, and have to do some copying by hand). If you see any errors, please let me know and I will look into it.

[Nathan Streeter]

First: very cool spreadsheet! I'm glad to have a resource that looks at the component OPR for pretty much every possible condition! It has all the usual OPR caveats, but it does seem useful for establishing some trends and making some comparisons.

As a sidenote, thank you to FRC HQ for making this data more available for capture. The API certainly provides much better data than the twitter feed over recent years.

I have some questions about the "units" of some columns... I'm pretty sure they're my initial guess for most of them, but I wanted to double-check.

For columns H and I (teleop Capture or Breach), I presume a "1" would indicate a successful Capture/Breach?

For columns J - V and AM (defense crossings), is "1" a single defense traversal (5 pts) or a weakened defense (10 pts, 2 traversals)?

Also, how are eOPR 1 and eOPR 2 calculated? What's the difference? They differ dramatically from the OPRs based solely on match scores.

[Caleb Sykes]

Originally Posted by Nathan Streeter

First: very cool spreadsheet! I'm glad to have a resource that looks at the component OPR for pretty much every possible condition! It has all the usual OPR caveats, but it does seem useful for establishing some trends and making some comparisons.

Thank you. I'm glad to see people are benefitting from it.

As a sidenote, thank you to FRC HQ for making this data more available for capture. The API certainly provides much better data than the twitter feed over recent years.

Agreed. Also thanks to Ether and team 2834, I have built off of both of their work, and this spreadsheet would not be possible without them.

I have some questions about the "units" of some columns... I'm pretty sure they're my initial guess for most of them, but I wanted to double-check.

For columns H and I (teleop Capture or Breach), I presume a "1" would indicate a successful Capture/Breach?

For columns J - V and AM (defense crossings), is "1" a single defense traversal (5 pts) or a weakened defense (10 pts, 2 traversals)?

Also, how are eOPR 1 and eOPR 2 calculated? What's the difference? They differ dramatically from the OPRs based solely on match scores.

The "instructions" tab has more detailed descriptions about each category, let me know if anything there is unclear and I can revise it. I'll summarize here anyway though.

"teleop Tower Captured" and "teleop Defenses Breached" both have units of ranking points. A 1 in either of these would indicate that the given team contributes an average of 1 ranking point each match.

All categories that have "crossings" in their name have units of crossings, not weakenings. That is, a 2 in any of these categories would indicate that the given team contributed 2 scored CROSSINGS over this defense each match.

eOPR1 and eOPR2 are my rough attempts to compensate for different scoring methods in quals and elims. Since breaches and captures provide points in elims, but not in quals, "normal" OPR probably does a poor job predicting elimination match scores (although this is as of yet unverified). eOPR1 essentially makes boulders and crosses scored in quals worth more, and eOPR2 takes breaching/capturing contributions and assigns them point values, and then adds those to the "normal" OPR.

[Caleb Sykes]

Week 4 data has been added.

Additionally, I deleted the unnecessary whitespace that was beneath most of the event sheets' data. This will allow sorting to make much more sense and cause the scroll bar to be more appropriately sized.

Also, I hadn't realized that excel saved the position of the last cell selected, which is why seemingly random positions on each page were previously selected upon entering them for the first time. I have now selected the top-left corner cell on each sheet.

As always, I appreciate feedback and/or error reports.

[Caleb Sykes]

Week 5 data has been added.

I will include the data for the Western Canada regional in the week 6 update.

[Ben Martin]

Thanks for producing this every week! It is very interesting how the results from this data aligns very closely with scoring averages by type in our scouting data (not a perfect match, but very close)--we'll definitely be using it for Championships scouting.

[Caleb Sykes]

I believe there is a good chance that I am currently calculating "tech foul count" and/or "tech fouls drawn" improperly. I will be investigating more tonight, but for the time being assume that these metrics are erroneous.

[Dancin103]

Caleb, thank you for pulling this together every week! Our team has been using it as a "pre-assessment" of teams before each event. We will for sure being using it for CMP!

Thanks again!

[hutchMN]

Thank you so much for putting this together. It's been a great tool so far.

[Caleb Sykes]

Originally Posted by Ben Martin

Thanks for producing this every week! It is very interesting how the results from this data aligns very closely with scoring averages by type in our scouting data (not a perfect match, but very close)--we'll definitely be using it for Championships scouting.

The results align reasonably well with our data as well. We got the most similar results when rounding all negative values up to 0 and rounding all positive values down to the nearest 0.1 or 0.2.

Originally Posted by Dancin103

Caleb, thank you for pulling this together every week! Our team has been using it as a "pre-assessment" of teams before each event. We will for sure being using it for CMP!

Thanks again!

Originally Posted by hutchMN

Thank you so much for putting this together. It's been a great tool so far.

Thanks for the compliments, I'm glad to hear it is getting some good use.

[Caleb Sykes]

Originally Posted by Caleb Sykes

I believe there is a good chance that I am currently calculating "tech foul count" and/or "tech fouls drawn" improperly. I will be investigating more tonight, but for the time being assume that these metrics are erroneous.

Never mind, I think these are still being calculated properly.

The reason I thought these were wrong was primarily a result of me forgetting how exactly tech fouls are scored.

[mitchellzone]

Caleb, this is great stuff, very helpful. The world results sheet makes it great to use for Worlds scouting.

I do have a question on how you've calculated these numbers, though. I'm assuming for a given event you're taking averages, but how do we end up with negative numbers for things like teleop high boulder points, etc.? There are several fields with values like this that I don't understand as the minimum value should really be zero.

Can you explain this? Thanks.

Also, I'm wondering if you'll be producing a sheet that contains only the teams going to Worlds in St. Louis? That would be helpful for those who are going. Thanks for doing this work!

/mike

[mitchellzone]

Originally Posted by mitchellzone

I do have a question on how you've calculated these numbers, though.

Thought of another couple:

Does this data just reflect qualification rounds or also playoff rounds? I assume the latter, but wanted to verify.

Also, how are you calculating total points? This seems to be a really low number...

/mike