An improvement to OPR

[Frenchie461]

Just a thought I had recently while working on some interesting linear algebra problems. Given that OPR is generated is generated by This
(thanks to Ether for this)


All of the operations performed to compute OPR are completely functional for complex numbers, so there appears to me to be no reason why OPR could not be solved for complex numbers where the real part of the element in the matrix is the teleoperated score and the imaginary part of the complex number being autonomous score. This should yield an OPR matrix containing complex entries, which theoretically should have a least squares average for both teleop and auton.

[SoftwareBug2.0]

What advantage does this have over simply calculating independently with just auto scores, and then just teleop scores, and then just climb scores? I've actually seen people do that.

And the results are fun sometimes because you get teams where their climb OPR is like -2.

[RyanCahoon]

Originally Posted by Frenchie461

This should yield an OPR matrix containing complex entries, which theoretically should have a least squares average for both teleop and auton.

Originally Posted by SoftwareBug2.0

What advantage does this have over simply calculating independently with just auto scores, and then just teleop scores, and then just climb scores?

Since the OPR calculation boils to down to

P = (A^-1) * S

where P is the OPR, A is the binary matrix denoting teams in each alliance and S is the alliance scores, then Frenchie461 is essentially advocating

Pt + Pa*i = (A^-1) * (St + Sa*i)

and since matrix multiplication is distributive

Pt + Pa*i = (A^-1) * St + ((A^-1) * Sa)*i

So you'll end up with the same result as calculating each OPR component independently. You'll get least-squares best fit for each component (as you would otherwise), but there won't be any additional interaction gained between them. This makes sense, because the least-squares fitting part of the operation happens when taking the inverse of A, and isn't affected by the value of S (whether real or complex) that it is post-multiplied by. Performance-wise, I would guess they would take about the same amount of time, assuming you're not re-calculating the value of A^-1 when doing the calculations independently.



note: the inverse operation written ^-1 above becomes the generalized inverse for non-square cases of A

[efoote868]

I'm not sure how this is an improvement; your code might be more concise but only if you're working with a computational package like MATLAB.

[Frenchie461]

Originally Posted by efoote868

I'm not sure how this is an improvement; your code might be more concise but only if you're working with a computational package like MATLAB.

It's only an improvement so far as it's more data than most teams usually compute, and it should be computationally faster than a pair of OPR calculations. Hypothetically, let's say that OPR is a O(N^3) operation, with this method it's N^3 to find auton and teleop rather than 2(N^3).

[Ether]

Originally Posted by Frenchie461

it should be computationally faster than a pair of OPR calculations.

In the formula for OPR, namely [A][OPR]~[SCORE], [OPR] and [SCORE] need not be vectors - they can be matrices.

So instead of [OPR] being a Nx1 column vector, it can be an Nx2 matrix... and [SCORE] can be a (2M)x2 matrix.

The first column of [OPR] and [SCORE] can then be for TeleOp, and the second column for Autonomous.

This can be extended to any desired number of columns. For example use 4 columns for TeleOp, Autonomous, Climb, and Foul points.

Adding extra columns to [OPR] and [SCORE] increases the computation time only minimally, since the lion's share of the computation is spent factoring [A]^T[A].

[Siri]

Originally Posted by Ether

This can be extended to any desired number of columns. For example use 4 columns for TeleOp, Autonomous, Climb, and Foul points.

Minor detour: does someone have comprehensive foul point data broken out from teleop scores? (because I would love them forever )

Other than that, it's a cool idea and would be a fun way to team the concept to new students around that level. In terms of data though, I don't know that it brings something new. It'd actually complicate my work to do it that way, because the matrix case Ether describes allows the simultaneous calculation of endgame OPR by the same method (i.e. it's not limited to 2, and we're looking for at least 3 basically ever year).

[AGPapa]

Originally Posted by Siri

Minor detour: does someone have comprehensive foul point data broken out from teleop scores? (because I would love them forever )

Other than that, it's a cool idea and would be a fun way to team the concept to new students around that level. In terms of data though, I don't know that it brings something new. It'd actually complicate my work to do it that way, because the matrix case Ether describes allows the simultaneous calculation of endgame OPR by the same method (i.e. it's not limited to 2, and we're looking for at least 3 basically ever year).

Ether has created a spreadsheet with all of the twitter data from every match.
http://www.chiefdelphi.com/forums/sh...t=twitter+data

It has everything you need.

[Ether]

Originally Posted by Siri

Minor detour: does someone have comprehensive foul point data broken out from teleop scores? (because I would love them forever ).

A few weeks ago I posted a least-squares analysis of foul points using Twitter data:

http://www.chiefdelphi.com/forums/sh...53&postcount=1

As with any analysis using Twitter data, caveat utilitor.

[Siri]

Originally Posted by AGPapa

Ether has created a spreadsheet will all of the twitter data from every match.
http://www.chiefdelphi.com/forums/sh...t=twitter+data

It has everything you need.

Originally Posted by Ether

A few weeks ago I posted a least-squares analysis of foul points using Twitter data:

http://www.chiefdelphi.com/forums/sh...53&postcount=1

As with any analysis using Twitter data, caveat utilitor.

Sorry, I should have been more specific about comprehensive (Twitter this year is missing a good chunk of MAR data). Thank you though, you're correct, I'll use this. Awesome analysis as always, Ether. Thanks!

[/and now back to your regularly scheduled thread]

[Citrus Dad]

Note that the OPR parameters are like any other statistical measure and have standard errors associated with those parameters. I haven't seen the SEs posted with the OPR parameters, but I can tell you that the SEs are likely to be VERY large for so few observations--only 8 per team at the Champs, at most 12 in any of the regionals. The OPRs are useful indicators, and probably can be used in a pinch if you lack other data, but they are highly unreliable for live scouting. I wouldn't bother calculating them on the fly at a competition. (The CCWM and DPRs appear to be even less reliable).

Our team had an OPR of 43 on Curie, yet our own scouting data showed we were scoring over 62 per match--quite a discrepancy. 4814 showed a CCWM ranked only 33rd yet went undefeated, and our defensive scouting data confirmed that they had substantial value added.

[SoftwareBug2.0]

Originally Posted by Citrus Dad

Note that the OPR parameters are like any other statistical measure and have standard errors ... I wouldn't bother calculating them on the fly at a competition. (The CCWM and DPRs appear to be even less reliable).

Obviously having your scouts get better data is the best solution, but there really is value in having the numbers as unreliable as they may be. Even if OPR for each phase of the game is +/- 50% from what that teams scores, that's often close enough to figure out what your match strategy should be.

Also, DPR may be less reliable than OPR, but I've been told too many times that it's totally meaningless. It isn't. It just doesn't mean what people think that it means. It's purely how many points your opponent scores. So last year, when we were a robot that scored just enough points at championship to make it so that we were better off playing offense than defense our DPR was awful, while this year our DPR was great because opposing alliances were forced to defend us, and therefore have one less robot scoring.

[Citrus Dad]

Originally Posted by SoftwareBug2.0

Obviously having your scouts get better data is the best solution, but there really is value in having the numbers as unreliable as they may be. Even if OPR for each phase of the game is +/- 50% from what that teams scores, that's often close enough to figure out what your match strategy should be.

I agree that the OPR is better than nothing--it still had a 0.91 correlation with our offensive scouting data. However, it tends to miss the outlier teams that may be hard to pick out otherwise. Our OPR was 27 points less than our actual offensive average--more than 50% off. But if your team has sufficient resources to calculate the OPR on the fly then you probably have enough to do full scouting. On the other hand, you might be relying on the OPR calculated with one of the apps tracking the competition, in which case then that's the best you have.

Originally Posted by SoftwareBug2.0

Also, DPR may be less reliable than OPR, but I've been told too many times that it's totally meaningless. It isn't. It just doesn't mean what people think that it means. It's purely how many points your opponent scores. So last year, when we were a robot that scored just enough points at championship to make it so that we were better off playing offense than defense our DPR was awful, while this year our DPR was great because opposing alliances were forced to defend us, and therefore have one less robot scoring.

4814 is a great case in point in Curie. Their OPR was only two points higher than their actual offensive output and their CCWM was only 9 points less than their OPR. They look like a defensive dud. Yet deeper digging shows that their defense was probably worth at least 30 points a match, and perhaps twice that.

[Ether]

Originally Posted by Citrus Dad

Their OPR was only two points higher than their actual offensive output

How are you calculating their actual offensive output? Do you have six scouts monitoring every match, one scout recording the scoring for each team?

[T^2]

Originally Posted by Ether

How are you calculating their actual offensive output? Do you have six scouts monitoring every match, one scout recording the scoring for each team?

Yes.