pic: Archimedes Scouting: OPR vs. Robot Performance (Scatter Plot)

[Ether]

Originally Posted by Ether

In years past, I have posted plots and charts and analyses showing the Ax-b residuals, which is the difference between the actual alliance scores and the "OPR-computed" scores for those same alliances, and makes no assumptions about the error distribution being normal. If this is of general interest, I can generate and post that analysis for 2015.

I already had a script which required only minor modification to generate this, so I went ahead and did it for all 117 events in 2015 with qual match data.

http://www.chiefdelphi.com/forums/sh...88&postcount=6

~

[Citrus Dad]

Originally Posted by evanperryg

Some teams, mine included, had no ability to actually score co-op, but would assemble the co-op stack and pass it to an alliance partner (recycle assist?) to score. We originally had a mechanism that allowed us to score co-op, but we found that it was more of a burden than anything. In the end, disregarding co-op is essential when assembling a proper picklist.

I just submitted the component graphs, they should be up soon. I will not include a litter OPR graph because there is no practical way to accurately scout litter point-for-point. (it's very difficult to see whose human player is throwing what litter where, while still tracking the robot.)

We also ignored coop points. For the highest performing robots in the Newton, the only robot for which it was an effective strategy was 118 who could put up a coop AND 3 stacks. For others the tradeoff is an almost certain 36-42 pt stack vs. a probability weighted 40 points (after the first 2 weeks, this probability was less than 1/3.)

The litter OPR includes putting noodles in the cans which is why the noodle scores are so high. 3-5 stacks noodled are worth 18-30 points.

[Ether]

Originally Posted by Ether

I already had a script which required only minor modification to generate this, so I went ahead and did it for all 117 events in 2015 with qual match data.

I added another attachment with Ax-b residual statistics for each of the 117 events:

http://www.chiefdelphi.com/forums/sh...53&postcount=7

~

[Ether]

Originally Posted by Citrus Dad

It's been known that OPR doesn't reflect actual scoring ability. It's a regression analysis that computes the implied "contribution" to scores. Unfortunately no one ever posts the estimates' standard errors (which I imagine to be enormous with 10 or so observations.)

I'm thinking of the parameter standard errors, i.e., the error estimate around the OPR parameter itself for each team. That can be computed from the matrix--it's a primary output of any statistical software package.

The second quote above is from a dialog I've been having with Citrus Dad, and I have his permission to post it here.

I'd like to hear what others have to say.

In order not to hijack this thread, I've created a separate discussion thread here:

http://www.chiefdelphi.com/forums/sh...d.php?t=137223