Thanks Ether!
I'd love to see the residual of the predictions of the winning margins using OPR, CCWM, and whatever you want to call the new thing (how about WMPR, if you don't like CCWMOA)? It would be interesting to see the average squared winning margin prediction residual and the distribution of the prediction residual (like you did with your L1 v. L2 comparison) for both 2015 FRC tournaments (where defense was essentially non-existent) and 2014 FRC tournaments (where defense mattered more).
It might also be interesting to see if tournaments with lots of matches per team are different from tournaments with few matches per team.
I'm puzzled by AGPapa's finding that the match outcomes (such as they were in 2015) are not predicted as well with the new measure. While minimizing the prediction error in the winning margin isn't the same as predicting the match outcomes, I'd expect the match outcome results to be fairly similar. Thoughts? (BTW, I haven't verified AGPapa's finding, so I suppose there's a chance that there's a bug in the code he used to predict the match outcomes?)
[Edit: AGPapa later found an error with his initially reported results.]
And if you had a lot of time and/or processing power on your hands, I'd also love to see how well the winning margins are predicted for matches that aren't in the training data. Given that we're so low on data, I'm reluctant to suggest the "model with the first 1/2 of the data, then test with the second 1/2 of the data" proposals as we may not have enough data to get a reliable model as it is. Instead, I'd suggest the "model with all of the data except for match 1, then test with match 1, then remodel with all of the data except match 2, then test on match 2, etc." approach as then the data size is almost the same but you're testing on data that's not in the training set.
I'd be happy to do this in scilab too, especially if you could get some 2014 tournament data in your nice formats.
BTW, I computed the new metric this morning using the data from the MISJO tournament you provided and got the same results for the new measures (using the new A and b that you provided), so that confirms that we're talking about the same thing.
