2015 Championship division simulated rankings

[Citrus Dad]

Originally Posted by Spoam

We have all this data available here if anyone wants to use it. Click the download button to get a local copy in csv format. We'll run our own monte carlo analysis later today using various models and post it here, along with some regression analysis on which model gives the most accurate predictions, just for fun.

Wow, thanks!

[forbes]

Originally Posted by Spoam

We have all this data available here if anyone wants to use it. Click the download button to get a local copy in csv format. We'll run our own monte carlo analysis later today using various models and post it here, along with some regression analysis on which model gives the most accurate predictions, just for fun.

Just to verify: the "Highest OPR (no co-op)" column includes auto?

[nuclearnerd]

Originally Posted by CVR

I think a better method would be to use the following:

Red Score =
Red1_(toteOPR + binOPR + litterOPR)
+ Red2_(toteOPR + binOPR + litterOPR)
+ Red3_(toteOPR + binOPR + litterOPR)
+ MAX(Red1_autoOPR, Red2_autoOPR, Red3_autoOPR)
+ MAX(Red1_coopOPR, Red2_coopOPR, Red3_coopOPR)

I think that method, while slightly more complex, will give more accurate results.

There's an issue with this method that may skew predictions for alliances with more than one team that does a lot of co-op. Lets say:

R1 has a "platform OPR of 40, and a co-op OPR of 30
R2 has a "platform OPR of 40 and a co-op OPR of 28
R3 has OPRs of 0 for the sake of argument

By the above method, the red alliance would be predicted to score just 110 points, since we would use R1's co-op average, but not R2's. But if R2 can usually score 40 platform points *and do co-op most of the time*, surely they wouldn't put up their 40 and then spend the rest of the match twiddling their thumbs. They would use the time they normally use on co-op to score more platform points!

(There's also the issue of whether the opposing alliance has a high enough Co-op OPR to ensure co-op will be successful, but now we're getting really complicated.)

[Louisiana Jones]

Originally Posted by nuclearnerd

There's an issue with this method that may skew predictions for alliances with more than one team that does a lot of co-op. Lets say:

R1 has a "platform OPR of 40, and a co-op OPR of 30
R2 has a "platform OPR of 40 and a co-op OPR of 28
R3 has OPRs of 0 for the sake of argument

By the above method, the red alliance would be predicted to score just 110 points, since we would use R1's co-op average, but not R2's. But if R2 can usually score 40 platform points *and do co-op most of the time*, surely they wouldn't put up their 40 and then spend the rest of the match twiddling their thumbs. They would use the time they normally use on co-op to score more platform points!

(There's also the issue of whether the opposing alliance has a high enough Co-op OPR to ensure co-op will be successful, but now we're getting really complicated.)

Is the max coopOPR used for the same event as the Max OPR? If not I see some problems where it may appear that teams score more than they really do, if they coop more at some events, and then stacked more at other events. They would be getting the max points from both.

[Citrus Dad]

On the coop OPR I think you need to take a sum of the maxBlue(coopOPR,20)+maxRed(coopOPR,20) as the closest approximation and apply the total to both alliances.

And watching the regionals, the coop points increased with overall scores so that the max coop OPRs should be in an event close to the max OPR.

[MechEng83]

Originally Posted by Spoam

I had the exact same idea actually. The problem with the OPR residual, however, is that it gives you information about the accuracy of the regression with regard to each match, not each robot.

If you took the set of residuals from the matches a robot played in, it makes intuitive sense that that data should contain some level of information about that robot's deviation from their OPR. But is a data set of only 8-12 elements enough for this value to dominate the noise generated by their alliance partners' deviations (and therefore produce a meaningful standard deviation itself)? I dunno.

If some statistics wiz would like to chime in on this, I'd love to hear it.

Originally Posted by themccannman

The problem with trying to use variation or standard deviation with OPR is that the number it spits out pretty much just tells you what their match schedule was like. OPR is already a calculation of how much an alliances score tends to change when certain teams are playing, calculating standard deviation for that basically just going backwards. OPR tries to determine how one robot affects an alliances score, where as SD (with unique alliances) would give you how each alliance affected that robots score.

Unfortunately it's not very useful unless you have actual scouted data for each team to use, in which case you can make much more accurate predictions about rankings. Our scouting system had a little less than an 80% success rate guessing the winners of each match in our division the last two years, and those games were very defense heavy. I would bet on this system approaching a 95% success rate guessing match results this year since the game is much more consistent.

Good points. Thanks for pointing out the flaw in my idea. what I surmise is that this calculation of stdev would be marginally useful at best. This reminds me of a mantra I hear at work quite often: "All models are wrong. Some models are useful."

[Joe Ross]

What probability distribution did you use for the random terms?