Chief Delphi - An improvement to OPR

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)

- Scouting (http://www.chiefdelphi.com/forums/forumdisplay.php?f=36)

- - An improvement to OPR (http://www.chiefdelphi.com/forums/showthread.php?t=116791)

Re: An improvement to OPR

Thanks for the information Ether. It spurred me to check the details in the Octave documentation

Quote:

Originally Posted by Ether (Post 1276717)

I was wondering what was the theoretical basis for assuming a normal distribution.

There is no theoretical or empirical* basis for assuming a normal distribution, it's just a matter of convenience and convention. For the purposes of estimating mean, minimizing the squared-error will give the right result for any non-skewed underlying distribution.

Unfortunately I don't have access to reliable per robot score data otherwise we could establish how well a Gaussian distribution models typical robot performance. (I did check my team's scouting data but it varied too far from the official scores to rely on.) If anyone would like to share scouting data from this season I'd be very interested.

In my professional life I work on big statistical modeling problems and we still usually base the models on Gaussians due to their computational ease, albeit as Gaussian Mixture Models to approximate any probability density function.

* In fact we know for certain that a pure climber can only score discrete values of 0, 10, 20 or 30 points.

Re: An improvement to OPR

Quote:

Originally Posted by MikeE (Post 1276757)

In my professional life I work on big statistical modeling problems and we still usually base the models on Gaussians due to their computational ease...

Yes, computational ease... and speed.

Speaking of speed, attached is a zip file containing a test case of N and d for the normal equations Nx=d.

Would you please solve it for x using Octave and tell me how long it takes? (Don't include the time it takes to read the large N matrix from the disk, just the computation time).

PS:

N and d were created from the official qual Match Results posted by FIRST for 75 regional and district events plus MAR, MSC, Archi, Curie, Galileo, & Newton. So solving for x is solving for World OPR.

Re: An improvement to OPR

Quote:

Originally Posted by Ether (Post 1276104)

As a short-term solution that sounds like a reasonable approach to try to make the best out of the data that is available.

Going forward, perhaps someone who has Frank's ear and is interested in statistics could make an appeal to him to resolve the Twitter data issues. At the very least, store the data locally (at the event) and don't delete it until it has been archived at FIRST. Then make the data available to the community.

Ether, I changed my mind about scaling numbers that has surrogate match in the vector b before solving Ax=b. I now propose to scale x(auto), x(tele) and x(climb) for each team proportionally so they will add up to the overall OPR.

We can test it afterwards and calculate the b and see how close it is to the missing subscore of the surrogate match.

Re: An improvement to OPR

Quote:

Originally Posted by Ed Law (Post 1276032)

Thank you for pointing out the issue with sum of individual categoty OPR do not add up to total OPR. I don't know exactly what you mean. You made it sound like I do the calculations by hand. I can ask the computer to run it 100 times and I can guarantee you that I will get the same answer every time. :)

I see your explanation about inclusion of the surrogate match scores. I think one check would be to see if the deviations of the total OPR vs the sum of the individual components is larger with the inclusion of more surrogate matches. You may be able to derive a correction factor based on the number of surrogates. (I estimated the average foul scores with a correction factor against our scouting data.)