Paper: Comparison of Statistical Prediction Models

CD-Media: Papers
, , , ,
1

Thread created automatically to discuss a document in CD-Media.

Comparison of Statistical Prediction Models
by: Caleb Sykes

This is a presentation of 9 distinct statistical prediction models of FRC matches from 2008-2016. The purpose is to determine which metrics have the most predictive value and where future efforts on predictive models should be focused.

This paper provides a comparison of common statistical prediction methods in order to determine which methods have the most predictive power. To see each model’s predictions for each match during the period 2008-2016, as well as each team’s rating before each match during this period, go to its corresponding workbook. The “Data Summary and Methodology v3” workbook contains details on each model, a FAQ, a summary of predictive capabilities of each model, and a side-by-side comparison of each model for the year 2016.

Elo winning margin.zip (23.9 MB)
adjusted winning record.zip (18.4 MB)
average score.zip (20.7 MB)
average winning margin.zip (20.7 MB)
calculated contribution to win.zip (21.8 MB)
calculated contribution to WM.zip (21.8 MB)
calculated contribution to score.zip (21.7 MB)
Ether power rating.zip (21.8 MB)
Data Summary and Methodology v3.xlsx (1.98 MB)
WM Power rating.zip (16 MB)

See primary discussion thread here.

2

Caleb this paper provided an unexpected result. The range of predictive accuracy in 2016 of the 10 methods is from ~66% to ~75%. However the combination of predictors made a slight improvement of ~1% to ~76%. One would expect a combination of predictors to improve more than ~1%

3

Primary discussion thread can be found here.

4

It really isn’t too surprising to me. All of these models are looking at the same data (raw match scores). Because they all analyze the same data, just with different methodologies, there isn’t any particular reason to expect a drastic performance boost by combining models.

5