OPR after Week Four Events

[Hoover]

My question is badly stated. How soon can projected results be generated once the qualifying schedule is published. How easy is this to do? I can do it myself if possible.

[efoote868]

Originally Posted by animenerdjohn

Is it more likely to be over the expected result or under the expected result.

Averages out to zero, but that's to be expected.

[dodar]

Originally Posted by animenerdjohn

Is it more likely to be over the expected result or under the expected result.

I'd say that the OPR match results are more often than not higher than the actual scores.

[Ether]

Originally Posted by efoote868

Can't find an easy way to say it, but:
Predicted OPR match results match actual match results 82% of the time (3142 out of 3833).

Interesting observation, and I got the same answer you did.

In 82% of the matches, the outcome of the match (win or lose) was the same according to the actual score and the OPR "expected" score.

[JohnSchneider]

Originally Posted by Ether

Interesting observation, and I got the same answer you did.

In 82% of the matches, the outcome of the match (win or lose) was the same according to the actual score and the OPR "expected" score.

Id love to see some confidence intervals on this. With a .95 CI it might be neat to try and apply it to matches this weekend before they happen and see to what accuracy an event could be predicted.

[Ether]

Originally Posted by dodar

I'd say that the OPR match results are more often than not higher than the actual scores.

3,713 (48.4%) of the actual alliance scores were greater than the OPR "expected" alliance score. 3,953 (51.6%) were less.

But the sum of all the actual alliance scores is equal to the sum of all the OPR "expected" alliance scores.

[Ether]

Here's a spreadsheet in a format which may be somewhat easier to use for various statistical analysis endeavors.

Column Headings & meaning:

E	event
M	Match
r1	r2	r3	b1	b2	b3	red & blue alliance teams
rs	red score
bs	blue score
ors	OPR "expected" red alliance score
obs	OPR "expected" blue alliance score
drs	rs-ors
dbs	bs-obs
rgo	rs>ors?
bgo	bs>obs?
rw	red win?
orw	red win based on OPR scores?
ocp	OPR correct prediction of match outcome?

[Ether]

Here's a spreadsheet using CCWM instead of OPR.

Column Headings & meaning:

E	event
M	Match
r1	r2	r3	b1	b2	b3	red & blue alliance teams
rs	red score
bs	blue score
crs	sum of red alliance teams' CCWMs
cbs	sum of blu alliance teams' CCWMs
rw	red win?
crw	crs>cbs?
ocp	rw==crw?

[markmcgary]

Originally Posted by Ether

Here's a spreadsheet using CCWM instead of OPR.

Column Headings & meaning:

E	event
M	Match
r1	r2	r3	b1	b2	b3	red & blue alliance teams
rs	red score
bs	blue score
crs	sum of red alliance teams' CCWMs
cbs	sum of blu alliance teams' CCWMs
rw	red win?
crw	crs>cbs?
ocp	rw==crw?

Correctly predicting 83.3% of the outcomes. (I'm pretty sure.)

[Ether]

Originally Posted by markmcgary

Correctly predicting 83.3% of the outcomes. (I'm pretty sure.)

Yes, I got the same number.

[Ed Law]

Originally Posted by Hoover

Hi Ed,

Forgive me if this has already been asked.

For an upcoming event the spreadsheet is of course mostly empty. Is there a way we could find the current OPR rankings for the teams coming to that event? I'd like to cut and paste that ranking order as a list into something I am working on.

Please read the instructions tab. The answer is there. The information you are looking for is available in one of the tabs. Filters are set up for you so you can select only the teams that is going to a particular regional/district.

[Ed Law]

Originally Posted by Hoover

My question is badly stated. How soon can projected results be generated once the qualifying schedule is published. How easy is this to do? I can do it myself if possible.

Starting from row 154 are the qualification matches. Column L and M is the prediction of each match. However it cannot predict match results until there are data. It needs a few matches before it will even be able to solve the equations. To get accurate and meaningful numbers, I think you need at least 4 to 5 matches.

The over 80% accuracy is after all the qualifying matches are available. A least square fit is used which is why the error is minimized. The number is even more impressive if you consider the close matches that can swing either way. During the qualifying matches, even after 4 to 5 matches, the prediction is not as accurate. But it is the best method there is and that is what most people use. I have not done any studies but the accuracy should be over 66% this year and the later matches will get more and more accurate.

Ether, this is another interesting challenge for you.

However there is another way to do this. It is by using historical OPR or projected OPR. But it only works with all 6 teams in that match has played in another event.

[Chris Hibner]

What I would be interested in is a comparison between OPR for an event, and actual average points scored for each robot. This, of course, means that a very accurate log of all robots for all matches is needed. Anyone have that data? (Before we get flamed for poor scouting, we don't keep track of ALL robots in ALL matches. The majority, yes - but not all.)

We make videos of all of our matches so we can go back and look for improvements in the robot and the driving. Using the videos, I've kept track of all of the points our robot scored during qualifications at both of our districts so far. OPR has been within 10% of our actual average at both events.

[IKE]

Originally Posted by Chris Hibner

What I would be interested in is a comparison between OPR for an event, and actual average points scored for each robot. This, of course, means that a very accurate log of all robots for all matches is needed. Anyone have that data? (Before we get flamed for poor scouting, we don't keep track of ALL robots in ALL matches. The majority, yes - but not all.)

We make videos of all of our matches so we can go back and look for improvements in the robot and the driving. Using the videos, I've kept track of all of the points our robot scored during qualifications at both of our districts so far. OPR has been within 10% of our actual average at both events.

We are starting to do that analysis for Grand Blanc. I have done it on a piece-meal basis for many other teams and the +/-10 is a pretty good rule of thumb this year, but there have been a handful of outliers (as to be expected). At Waterford, on firday night scouting, a team had an OPR between 15 and 20 pts (after 8 matches), but they were not even on the field for about 6 of those matches.

And of course our "actuals" are only as accurrate as the scouts. In general, they do a really good job, but I do frequently find errors (one scout recorded 33 putting up 8 discs in auton )

[stingray27]

Originally Posted by Hoover

My question is badly stated. How soon can projected results be generated once the qualifying schedule is published. How easy is this to do? I can do it myself if possible.

I know some people have already seen this, but this forum has responded exactly to what my app does. Discussion on the topic can be found here:
NEW: OPR FIRST 2013 Android App
OPR and CCMW applications

Either way, I use the OPR's to calculate a predicted score in real time. Like stated earlier, it is very accurate in terms of who wins/loses. As of scoring, the percentage varies. It would be cool to use past OPRs to predict the results. My app uses data specific to that regional and only that regional. So the longer the regional goes on, the more accurate the data is.

Originally Posted by Chris Hibner

What I would be interested in is a comparison between OPR for an event, and actual average points scored for each robot. This, of course, means that a very accurate log of all robots for all matches is needed. Anyone have that data? (Before we get flamed for poor scouting, we don't keep track of ALL robots in ALL matches. The majority, yes - but not all.)

In addition, it has information about the average alliance score and OPR right next to each other to compare. I know, its not the same as a robots average points, but it would be hard to get that information for every team in a regional that hasnt been played yet because no one can keep track of individual scouting data for each team at each regional. Our OPR was 42.7 ish (I dont remember the actual number) but according to our scouting data, our average scoring points were around 43-44 points.