Incorporating Opposing Alliance Information in CCWM Calculations

[wgardner]

Originally Posted by AGPapa

I ran the numbers on the 2014 St. Joseph event. I checked that my calculations for 2015 St. Joseph match Ether's, so I'm fairly confident that everything is correct.

Here's how each stat did at "predicting" the winner of each match.

OPR: 87.2%
CCWM: 83.3%
WMPR: 91.0%

AGPapa's results are using the full data as training data and then reusing it as testing data.

On the same data doing the "remove one match from the training data, model based on the rest of the data, use the removed match as testing data, and repeat the process for all matches" method, I got the following results:

Stdev of winning margin prediction residual
OPR : 63.8
CCWM: 72.8
WMPR: 66.3

When I looked at scaling down each of the metrics to improve their prediction performance on testing data not in the training set, the best Stdevs I get for each were:
OPR*0.9: 63.3
CCWM*0.6: 66.2
WMPR*0.7: 60.8

Match prediction outcomes
OPR : 60 of 78 (76.9 %)
CCWM: 57 of 78 (73.1 %)
WMPR: 62 of 78 (79.5 %)

Yeah! Even with testing data not used in the training set, WMPR seems to be outperforming CCWM in predicting the winning margins and the match outcomes in this single 2014 tournament (which again is a game with substantial defense). I'm hoping to get the match results (b with red and blue scores separately) for other 2014 tournaments to see if this is a general result.

[Edit: found a bug in the OPR code. Fixed it. Updated comments. Also included the scaled down OPR, CCWM, and WMPR prediction residuals to address overfitting.]