View Single Post
  #5   Spotlight this post!  
Unread 01-07-2015, 20:15
Ether's Avatar
Ether Ether is offline
systems engineer (retired)
no team
 
Join Date: Nov 2009
Rookie Year: 1969
Location: US
Posts: 8,104
Ether has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond reputeEther has a reputation beyond repute
Re: "standard error" of OPR values

Quote:
Originally Posted by Citrus Dad View Post
That one can calculate a number doesn't mean that the number is meaningful.
I'm glad you agree with me on this very important point. It's what I have been saying about your request for SE estimates for each individual OPR.


Quote:
Without a report of the error around the parameter estimates, the least squares fit is not statistically valid
Without knowing your private definition of "statistically valid" I can neither agree nor disagree.


Quote:
and the meaning cannot be interpreted.
The meaning can be interpreted as follows: It is the set of model parameters which minimizes the sum of the squares of the differences between the actual and model-predicted alliance scores. This is universally understood. Now once you've done that regression, proceeding to do inferential statistics based on the fitted model is where you hit a speed bump because the data does not satisfy the assumptions required for many of the common statistics.

The usefulness of the fitted model can, however, be assessed without using said statistics.


Quote:
I wasn't quite sure why you dug up my original post to start this discussion.
I had spent quite some time researching the OP question and came back to tie up loose ends.

Quote:
It seemed out of context with all of your other discussion about adding error estimates.
How so? I think I have been fairly consistent throughout this thread.

Quote:
That said, my request was more general, and it seems to be answered more generally by the other computational efforts that have been going on in the 2 related threads.
Your original request was (emphasis mine):
Quote:
I'm thinking of the parameter standard errors, i.e., the error estimate around the OPR parameter itself for each team. That can be computed from the matrix--it's a primary output of any statistical software package.
During the hiatus I researched this extensively. The standard error of model parameters (regression coefficients) is reported by SPSS, SAS, MINITAB, R, ASP, MicrOsiris, Tanagra, and even Excel. All these packages compute the same set of values, so they are all doing the same thing.

Given [A][x]=[b], the following computation produces the same values as those packages:
x = A\b;
residuals = b-A*x;
SSres = residuals'*residuals;
VARres = SSres/(alliances-teams);

Av = A/sqrt(VARres);
Nvi = inv(Av'*Av);
SE_of_parameters = sqrt(diag(Nvi))
The above code clearly shows that this computation is assuming that the standard deviation is constant for all measurements (alliance scores) and thus for all teams... which we know is clearly not the case. That's one reason it produces meaningless results in the case of FRC match results data.


Quote:
But one point, I will say that using a fixed effects models with a separate match progression parameter (to capture the most likely source of heteroskedasticity) should lead to parameter estimates that will provide valid error terms using FRC data. But computing fixed effects models are much more complex processes. It is something that can be done in R.
That's an interesting suggestion, but I doubt it would be successful. I'd be pleased to be proven wrong. If you are willing to try it, I will provide whatever raw data you need in your format of choice.



Reply With Quote