OPR after Week Five Events

[Ed Law]

The OPR/CCWM numbers up to Week 5 events have been posted, please see

http://www.chiefdelphi.com/media/papers/2174

All events up to Week 5 are now included. I fixed a small bug for each event page in column T for "Record". The formula was wrong for rows beyond the first 37 teams.

If you find any error or have any questions, please let me know.

[Ether]

Weeks 1 thru 5 OPR & CCWM correlation to actual match results.


CCWM Column Headings & meaning:

E	event
M	Match
r1	r2	r3	b1	b2	b3	red & blue alliance teams
rs	red score
bs	blue score
crs	sum of red alliance teams' CCWMs
cbs	sum of blu alliance teams' CCWMs
rw	rs>bs?
crw	crs>cbs?
ccp	rw==crw? (CCWM correct prediction of match outcome?)

OPR Column Headings & meaning:

E	event
M	Match
r1	r2	r3	b1	b2	b3	red & blue alliance teams
rs	red score
bs	blue score
ors	OPR "expected" red alliance score
obs	OPR "expected" blue alliance score
drs	rs-ors
dbs	bs-obs
rgo	rs>ors?
bgo	bs>obs?
rw	rs>bs?
orw	ors>obs? (red win based on OPR scores?)
ocp	rw==orw? (OPR correct prediction of match outcome?)

[efoote868]

Once again, thank you Ed and Ether for posting this.

Summary of global OPR and CCWM match win/loss predictions: 81.72% and 82.47% respectively.

Does anyone have an idea of the number of teams yet to compete in week 6?

[Basel A]

Originally Posted by efoote868

Once again, thank you Ed and Ether for posting this.

Summary of global OPR and CCWM match win/loss predictions: 81.72% and 82.47% respectively.

Does anyone have an idea of the number of teams yet to compete in week 6?

278 teams in Ed's database haven't competed yet.

[Ed Law]

Originally Posted by Basel A

278 teams in Ed's database haven't competed yet.

That would be a rough estimate. Some teams may have dropped out and not compete in any events.

[MikeE]

Thanks Ed & Ether for this great resource.

However I do need to quibble about how OPR/CCWM is being discussed, as exemplified by several earlier posts. Picking one of these:

Originally Posted by efoote868

Summary of global OPR and CCWM match win/loss predictions: 81.72% and 82.47% respectively.

Those numbers are misleading since OPR/CCWM are calculated from the same data which are being used to test their predictive power, i.e. the training & test sets are the same.

It's analogous to (although not as extreme as) stating that final qualification ranking is a good predictor of the performance in earlier qualifying matches. Whereas obviously qualification ranking is a consequence of performance in earlier matches.

Good practice would use disjoint training and testing sets. I'm sure this analysis has been performed in previous seasons but I didn't see it from a brief search of CD.

Interestingly the simple baseline heuristic of "alliance with lower team numbers" has 59.1% predictive power for qualification matches this season. I'm assume that OPR and CCWM are better than that, but not as good as the ~82% claimed above.

[Ether]

Originally Posted by MikeE

Those numbers are misleading since OPR/CCWM are calculated from the same data which are being used to test their predictive power, i.e. the training & test sets are the same.

That's a valid criticism, which could be addressed by using Weeks 1 thru 4 OPR numbers to predict Week5 outcomes. That would take a little bit more work, since Week5 may have teams which are competing for the first time, so they would have no OPR values. Those matches would have to be omitted from the analysis.

[Ether]

Originally Posted by Ether

That's a valid criticism, which could be addressed by using Weeks 1 thru 4 OPR numbers to predict Week5 outcomes.

OK, here's qual matches of Week5 events being predicted by OPR World rank based on weeks 1 thru 4 data only. 72% correct

This is a pessimistic estimate, since data gathered during the Week5 events for already-played qual matches could be used to supplement the weeks 1 thru 4 data to predict future qual matches in the Week5 events.

Based on the earlier post by efoote868, future qual matches at any given event are best predicted by OPR of already-played matches at that event, once a sufficient number of matches has been played.

Forgot to mention: columns V thru AA list the teams for each match for which no week 1 thru 4 OPR data is available.

[efoote868]

Originally Posted by MikeE

Good practice would use disjoint training and testing sets. I'm sure this analysis has been performed in previous seasons but I didn't see it from a brief search of CD.

Interestingly the simple baseline heuristic of "alliance with lower team numbers" has 59.1% predictive power for qualification matches this season. I'm assume that OPR and CCWM are better than that, but not as good as the ~82% claimed above.

At Boilermaker Regional I calculated the OPR of teams using their Friday match results (about 8 or 9 matches), and tracked the qualification results on Saturday. OPR predictions were 20 for 24.

I'm guessing the easiest place to track predictive power would be the Championship event.

Originally Posted by Ether

That's a valid criticism, which could be addressed by using Weeks 1 thru 4 OPR numbers to predict Week5 outcomes. That would take a little bit more work, since Week5 may have teams which are competing for the first time, so they would have no OPR values. Those matches would have to be omitted from the analysis.

I'd be interested in substituting unknown values with world averages. I'll probably do this during Crossroads until each team has played 5 matches.

[George1902]

OPR and CCWM are calculated using only qualification match results, right?

So, one could test their predictive value using the elimination results of that event.

[Ether]

Originally Posted by Ether

Weeks 1 thru 5 OPR & CCWM correlation to actual match results

Oh, and here's a link to updated Twitter data for weeks 1 thru 5:

http://www.chiefdelphi.com/forums/sh...53&postcount=1

[MisterJ]

As a second-year non-engineering mentor, I was wondering all off-season what my role on the team would be now the the team was able to mostly sustain itself. I discovered OPR and related statistics this year and they have kept me busy with scouting every week.

Thank you for all the work you do correlating this data each week! It has made FRC so much more interesting for me this year! I love watching the webcasts of high-OPR teams like 1114, and it doesn't hurt that OPR has made my own team look pretty darn good this year, (4th and 2nd in OPR, respectively, in their two district events, even without a banner to show for it).

So from a statistics geek from PA... thanks again!

[I-DOG]

It's bittersweet that we have the 5th highest OPR in the world and didn't qualify for Championship...

[AlexC]

Thanks for this. It was a pleasure to meet and win with you last Saturday.

[Ether]

Originally Posted by I-DOG

we have the 5th highest OPR in the world

What data are you looking at ?