[wgardner]

[Edit: my previously posted results had mistakenly reported the values for the scaled versions of OPR, CCWM, and WMPR as the unscaled values (!). Conclusions are somewhat changed as noted below.]

So my summary of the previous data:

WMPR always results in the smallest training data winning margin prediction residual standard deviation. (Whew, try saying that 5 times fast.)

WMPR is also very good at predicting training data match outcomes. For some reason, CCWM beats it in 1 tournament but otherwise WMPR is best in the other 3.

But on the testing data, things go haywire. There are significant drops in performance in predicting winning margins for all 3 stats, showing that all 3 stats are substantially overfit. Frequently, all 3 stats give better performance at predicting winning margins by using scaled down versions of the stats. The WMPR in particular is substantially overfit (look for a later post with a discussion of this).

BTW, it seems like some folks are most interested in predicting match outcomes rather than match statistics. If that's really what folks are interested in, there are probably better ways of doing that (e.g., with linear models but where the error measure better correlates with match outcomes, or with non-linear models). I'm going to ponder that for a while...