I've added the dataset used for these calculations to the paper. Feel free to use it to do any more analysis that you'd like.
Now, to answer specific questions:
Quote:
Originally Posted by Joe Ross
Do the results change if you only look at the top 24 or 30 teams at an event (the teams you would be considering when forming a pick list)?
|
Yes. By selecting the top 30 teams (in terms of Average Score), the least squares line becomes
Code:
y = 1.0486x - 0.7729
with an R^2 of 87.32%. The model actually moves further away away from the line we're expecting (y = x) when compared with the overall combined model, though it has a much higher R^2 value (a change of 7.02%).
In terms of the percent error model, the new mu is 1.11% with a sigma of 45.10%. The new table for the probability a team will fall within a given percent error is as follows:
Code:
10% 17.543%
20% 34.248%
30% 49.396%
40% 62.475%
50% 73.230%
60% 81.649%
70% 87.927%
80% 92.383%
90% 95.396%
100% 97.336%
110% 98.525%
120% 99.219%
130% 99.605%
140% 99.809%
150% 99.912%
160% 99.961%
170% 99.984%
180% 99.993%
190% 99.997%
200% 99.999%
Quote:
Originally Posted by Joe Ross
Is there an OPR at which it becomes more accurate? Looking at chart 13, OPRs above 15 seem much better then those below 15 (obviously game dependent).
|
Here's a table of the different averages and standard deviations for OPRs greater than or equal to the OPR listed. I see large increases in standard deviation from 10 to 20 (as you observed), and from 30 to 40.
Code:
OPR mu sigma
80 6.88% 15.60%
70 13.02% 13.92%
60 12.57% 21.20%
50 18.15% 25.74%
40 20.00% 25.31%
30 23.33% 55.35%
20 20.56% 51.77%
10 24.36% 92.58%
0 20.76% 100.66%
-10 3.08% 131.97%
Quote:
Originally Posted by Joe Ross
Can you quantify the percent chance that Team A is better then Team B, given a specific OPR difference (IE, Team A has an OPR that is 1 higher then Team B, and has a 55% chance of being better then Team B, but Team C has an OPR that is 10 higher then Team B, and has a 90% chance of being better then Team B.
|
This table has the mu and sigma for all teams within the OPR bin.
Code:
OPR mu sigma
80 6.88% 13.51%
70 17.12% 11.27%
60 12.12% 26.05%
50 26.12% 29.06%
40 24.18% 23.79%
30 28.43% 81.58%
20 13.57% 40.66%
10 36.60% 165.01%
0 4.63% 129.47%
-10 -287.83% 216.42%
Let Team A be represented by a Normal model with mu OPR, and sigma equal to the sigma in the table above multiplied by the team's OPR. Follow the same pattern for Team B. Subtract the two normal models (subtract the two averages; A-B, and add the variances to find the new sigma). Integrate underneath this curve from 0 to infinity to find the probability A would score more than B.
This is a method to approximate a prediction strategy I detailed in this post:
http://www.chiefdelphi.com/forums/sh...23&postcount=1
Quote:
Originally Posted by Joe Ross
Does the percent error histogram still look normal if you discard the outliers and put more bins between -100% and +200%?
|
Yes.
https://drive.google.com/file/d/0B4t...it?usp=sharing
Quote:
Originally Posted by Basel A
If there were such a correlation, you would have noticed it in the residual plot for your OPR-True Average linear regression. I didn't notice a residual plot in your pdf; they are essential for determining if your model is a good fit for the data.
|
We're not creating a model for the data we have, which is why I removed it; this wasn't about creating a regression that was supposed to model the data. Instead, this was about checking one of the properties of OPR: ideally, it should have a 100% correlation with True Average score, with an intercept of 0 and slope of 1.
That being said, here's links to the residual plots:
As a function of OPR:
https://drive.google.com/file/d/0B4t...it?usp=sharing
As a function of True Average Score:
https://drive.google.com/file/d/0B4t...it?usp=sharing
Quote:
Originally Posted by Michael Hill
This thread combined with watching Searching for Bobby Fischer earlier today got me wondering how to apply the Elo Rating to FRC, and I think I figured it out. This is the resulting data from the 7 weeks of the 2013 season (Sorted by rating)
Not sure how useful the Elo ratings are, but there is some statistical significance with having teams like 469, 67, 1114 and 2056 near the top. I'm nowhere near good enough with statistics to determine if this is enough data to work with (my instinct says no), but it's just another interesting way to look at wins/losses
|
Elo predictions of matches on Saturday, based on the previous day's matches for Archimedes were around 65-80% accurate, if memory serves.
It was more accurate than the method we used where a team was represented by a normal model with mu equal to true mu and sigma equal to true sigma, and integrating underneath to find the chance for each alliance to win. See thread here:
http://www.chiefdelphi.com/forums/sh...23&postcount=1