View Single Post
  #29   Spotlight this post!  
Unread 15-06-2015, 17:13
wgardner's Avatar
wgardner wgardner is offline
Registered User
no team
Team Role: Coach
 
Join Date: Feb 2013
Rookie Year: 2012
Location: Charlottesville, VA
Posts: 171
wgardner is a splendid one to beholdwgardner is a splendid one to beholdwgardner is a splendid one to beholdwgardner is a splendid one to beholdwgardner is a splendid one to beholdwgardner is a splendid one to beholdwgardner is a splendid one to behold
Re: Overview and Analysis of FIRST Stats

Quote:
Originally Posted by AGPapa View Post
Yes, still with VarN/VarO = 2.5. This was chosen fairly arbitrarily, better numbers probably exist.

Here's Ed Law's database. He has the LS OPRs for every regional/district since 2009. He's also has the "World OPRs" for each team in each year in the Worldrank tab.



Thanks, that makes a lot of sense.



I'm having some difficulty understanding what you're asking for here, but here's what I think you're looking for.

std dev of Previous OPR (LS) - Champs OPR (LS): 18.9530
std dev of Champs OPR (LS): 25.1509
std dev of match scores: 57.6501


I want to point out that the previous OPR is not an unbiased estimator of the Champs OPR. Champ OPRs are higher by 2.6 on average. (In my MMSE calculations I added a constant to all of the priors to try and combat this).


EDIT: I think we can use this to find a good value for VarN/VarO.

Var(Match Score) = Var(O) + Var(O) + Var(O) + Var(N)
Assuming that Var(Champs OPR) = Var(O) then we can solve the above equation and get that Var(N) = 1425.8, so Var(N)/Var(O) is 2.25. Now this is only useful after the fact, but it confirms that choosing VarN/VarO to be 2.5 wasn't that far off.
Yes, that all makes sense and looks good. The ratio of the stdev of the opr guess/ the stdev of the oprs themselves was 18.95/25.15 = 0.75 which is between the values of 1.0 and 0.3 that I used in my sims, so it seems consistent that your plot shape is sort of between those two.

I also renormalize the oprs to agree with the new average, as you described.

I found that the variance of the ls oprs was often a bit above the variance of the true underlying o values because of the slight overfitting, so if anything you might be slightly underestimating the variance of n. 2.5 - 3.0 is probably a good guess.

Thanks for all of the great data!
__________________
CHEER4FTC website and facebook online FTC resources.
Providing support for FTC Teams in the Charlottesville, VA area and beyond.
Reply With Quote