Quote:
Originally Posted by wgardner
Stdev of prediction residual of the winning margins:
OPR: 34.6
CCWM: 30.5
WMPR: 30.4
(note that on the testing data from a few posts ago, WMPR had a Stdev of 15.9, so this is an argument that WMPR is "overfitting" the small amount of data available and that it could benefit from having more matches per team)
|
For data that is "overfit" you can sometimes improve the prediction performance on the testing data by simply scaling down the solution.
For fun, I computed the standard deviation of the prediction residual of the testing data not in the training set using the WMPR solution, 0.9*WMPR, 0.8*WMPR, etc. The standard deviation of the prediction residual of the winning margin for the test data for this particular tournament was minimized by 0.7*WMPR, and that standard deviation was down to 28.4 from 30.4 for the unscaled WMPR. So again, more evidence that the WMPR is overfit and could benefit from additional data.
This doesn't change the match outcome prediction that some folks are interested in, since scaling all of the WMPRs down doesn't change the sign of the predicted winning margin which is all the match outcome prediction is.