[Ether]

Originally Posted by asid61

So it's not possible to perform a statistically valid calculation for standard deviation?

We're discussing standard error of the model parameters, also known as standard error of the regression coefficients. So in our particular case, that would be standard error of the OPRs.

Standard error of the model parameters is a very useful statistic in those cases where it applies. I mentioned one such situation in my previous post:

In a situation, unlike FRC OPR, where you know the variance of each observed value (either by repeated observations using the same values for the predictor variables, or if you are measuring something with an instrument of known accuracy) you can put those variances into the design matrix for each observation and compute a meaningful standard error for each of the model parameters.

An example of the above would be analysis and correction of land surveying network measurement data. The standard deviation of the measurements is known a priori from the manufacturer's specs for the measurement instruments and from the surveyor's prior experience with those instruments.

In such as case, computing standard error of the model parameters is justified, and the results are meaningful. All modern land surveying measurement adjustment apps include it in their reports.

Are there no ways to solve for it with a system that is dependent on other robots' performances?

That's a large (but not the only) part of the problem.

I briefly addressed this in my previous post:

Or if, unlike FRC OPR, you have good reason to believe the observations are homoscedastic, you can compute the variance of the residuals and use that to back-calculate standard errors for the model parameters. If you do this for FRC data the result will be standard errors which are very nearly the same for each OPR value... which is clearly not the expected result.

In the case computing OPRs using only FIRST-provided match results data (no manual scouting), the data does not meet the requirements for using the above technique.

In fact, when you use the above technique for OPR you are essentially assuming that all teams are identical in their consistency of scoring, so it's not surprising that when you put that assumption into the calculation you get it back out in the results. GIGO.

Posting invalid and misleading statistics is a bad idea, especially when there are better, more meaningful statistics to fill the role.

For Richard and Gus: If all you are looking for is one overall ballpark number "how bad are the OPR calculations for this event" let's explore better ways to present that.