Overview and Analysis of FIRST Stats

[wgardner]

Originally Posted by AGPapa

I did some tests on how apriori information can improve the estimation. I wanted to see how these stats would do in predicting the outcomes for the last day of an event (to project the standings). The following data is from the four 2014 Championship Divisions. For each division 150 qual matches were played on the first two days, leaving 17 on the last one. Since the event was basically complete, LS OPR did pretty well. I suspect that for an event with more matches on the last day (like a District Championship) it would perform worse.

I used the LS “World OPR” as the prior for one of the tests. For another I averaged Oavg and World OPR (essentially regressing them to the mean). All of the MMSE calculations were done with VarN/VarO at 2.5

LS OPR (no prior): 69.12%
MMSE OPR (Oavg prior): 69.12%
MMSE OPR (World OPR prior): 72.06%
MMSE OPR (regressed World OPR prior): 72.06%

Adding prior information improved the predictions by a couple of matches, I’ll try this again later for a District Championship and see how it turns out.

Thanks, AGPapa! I wonder if you could do the following:

1. Could you report the average squared error in the scores and/or winning margin, in addition to the probability of correct match outcome?

2. Your runs were using 150 values as the "training set" and 17 as the "testing set". Could you run them with other splits, like using 50 or 100 matches as training and the remaining part as testing? It would be interesting to see how the different techniques perform as the tournament progresses.

Cheers.