"standard error" of OPR values

[Citrus Dad]

Originally Posted by wgardner

There are two types of error:

The first is the prediction residual which measures how well the OPR model is predicting match outcomes. In games where there is a lot of match-to-match variation, the prediction residual will be high no matter how many matches each team plays.

The second is the error in measuring the actual, underlying OPR value (if you buy into the linear model). If teams actually had an underlying OPR value, then as teams play 10, 100, 1000 matches the error in computing this value will go to zero.

So, the question is, what exactly are you trying to measure? If you want confidence in the underlying OPR values or perhaps the rankings produced by the OPR values, then the second error is the one you want to figure out and the prediction residual won't really answer that. If you want to know how well the OPR model will predict match outcomes, then the first error is the one you care about.

The first error term is generally reported and Ether produced a measure of those residuals.

It's the second error term that I haven't seen reported. And in my experience working with econometric models, having only 10 observations likely leads to a very large standard error around this parameter estimate. I don't think that calculating this will change the OPR per se, but it will provide a useful measure of the (im)precision of the estimates that I don't think most students and mentors are aware of.

Also, as you imply, a linear model may not be the most appropriate structure even though it is by far the easiest to compute with Excel. For example, the cap on resource availability probably creates a log-linear relationship.