Is OPR an accurate measurement system?

[Ether]

Originally Posted by CJ_Elliott

I don't know exactly how OPR is equated

Would you like to learn? I can post some links to discussion threads here on CD that are written at an accessible level.

... and if you have any questions I -- and others I'm sure -- would be glad to answer them.

[gblake]

To get the right answer, you first have to ask the right question ....

And

"... all models are wrong, but some are useful."

George Box

OPR is what is it is; and the OPR equations compute OPRs 100% accurately.

You need to ask/determine whether OPR is a useful tool for your purpose (or ask what things OPR is useful for).

I personally think that Chairman's Award submissions are a better (but still imperfect) tool to use than OPR is, if I'm (quoting the OP) searching for "... the best teams in FRC."

Blake

[Louisiana Jones]

Originally Posted by tindleroot

Since OPR is calculated under the implication that every team is playing at their normal ability every match, any situation where a team is playing below (or above) their ability is going to mess up OPR calculations not only for them but for other teams in their matches.

Some teams have systems where scouters track irregularities like broken robots/ penalties and include that data in the OPR calculations to make the calculation more accurately reflect the performance of robots.

[Caleb Sykes]

Originally Posted by throwaway

Simple answer: It's not a great measurement this year but it's certainly better than the rankings

This. Too many people bash on OPR, and not enough people bash on the rankings.

[Caleb Sykes]

Originally Posted by saikiranra

Finding component OPR numbers can be useful depending on what you are looking for.

Is anyone actually going to be computing component OPRs this year? I believe Ed Law is not doing that this year, so I need to find a new master scouting database to reference.

[Jon Stratis]

With the data that FIRST provides through the FRC Event API, we can certainly do much better than your typical OPR. For example, I can pull down that data and know exactly which defenses were on the field and which of those were crossed and damaged in every match any team played in. I can know exactly how many balls were scored in which goals, how many robots challenged, and how many robots climbed. Proper statistical analysis (think OPR, but for each individual category instead of just overall score) can get you much more detailed and specific data. It won't be the whole story, but I would be willing to bet it would be more accurate than just the overall OPR. And more useful in assembling an eliminations alliance with the capabilities you want.

[Louisiana Jones]

Originally Posted by Caleb Sykes

This. Too many people bash on OPR, and not enough people bash on the rankings.

Taking Northern Lights as an example the first robot that was picked had the Highest OPR, but was ranked 16th.

While 5232 helped their alliance win and probably was a good bargain for when they were picked, it appears that most teams at the event didn't think they were a top 15 team, otherwise they would have been picked sooner.

[hutchMN]

Originally Posted by Louisiana Jones

Taking Northern Lights as an example the first robot that was picked had the Highest OPR, but was ranked 16th.

While 5232 helped their alliance win and probably was a good bargain for when they were picked, it appears that most teams at the event didn't think they were a top 15 team, otherwise they would have been picked sooner.

But they probably had the best challenge we will see all year in the double DECCer . That was insane

[waialua359]

OPR is no worse or no better than how they rank teams in sports.
Any given Saturday or Sunday?
NCAA March Madness tournament? 5 vs. 12 seeds? etc...

[BrennanB]

Originally Posted by Caleb Sykes

This. Too many people bash on OPR, and not enough people bash on the rankings.

Because no ranking system is perfect, and people (usually) don't go around praising rankings like they are the god of all numbers.

People do that with OPR

[Chris is me]

Originally Posted by Louisiana Jones

Some teams have systems where scouters track irregularities like broken robots/ penalties and include that data in the OPR calculations to make the calculation more accurately reflect the performance of robots.

If you're actually scouting matches, you have basically no use for OPR. Actual objective data will beat it every time.

[Libby K]

My favorite statement on OPR will always be this comment from Karthik's 2012 Effective FIRST Strategies presentation.

Take OPR with a couple grains of salt and understand, within that year's game, why it might not be accurate. (Scroll back to ~50min in that video if you want to hear the whole spiel. Better yet, watch the whole thing.)

[Richard Wallace]

Originally Posted by Libby K

Better yet, watch the whole thing.

Or you could just go directly to 57:44 .

[Louisiana Jones]

Originally Posted by Chris is me

If you're actually scouting matches, you have basically no use for OPR. Actual objective data will beat it every time.

Its easy for a single scout to track dead robots or any other measure throughout a competition and apply it into an OPR calculator. While this isn't as good as having experienced scouts watching every match some teams might not have enough experienced students to scout every match.

I have found that actual objective data collected and used by our students to make decisions sometimes yields what some would consider questionable decisions.

Sometimes teams don't know what data should be collected or overlook a key feature of the game. OPR can help account for a team's contribution even if they aren't the robot actually scoring the points or they do something significant that isn't accounted for in the collected data.

[themccannman]

This comes up pretty much every year, the best answer is "it's okay", but the most important part is to understand how it's derived which Jared explained very well. If you understand what it actually means then it's a very useful tool.