Week3 cumulative Twitter stats & OPRs

[Ether]

Originally Posted by GearsOfFury

The bi-modal nature of this graph is really very surprising to me. I cannot think of why there would be such a discrete break around 0... What am I missing?

It looked a bit strange to me at first, so I re-created the graph manually directly from the Twitter data and got the same result. I can post the spreadsheet later this evening if you're interested.

[Ether]

Originally Posted by myork

Is the twitter data any different from the FRC-Spy data?

Same thing.

[Ether]

Originally Posted by dodar

You are reading that wrongly. That shows me posting after him.

No, I am reading it correctly. The threaded view is a hierarchy. The intent is to show which post you are responding to.

All I did was click Post Reply at the very bottom of the page.

That's the problem. When you do that, it links your post in the threaded view to whatever is the most recent post in the thread, even if that isn't the post you are responding to. That might be OK if you include context in your post like quoting a section of the post to which you are responding or use screen names instead of pronouns. Otherwise there is no way to tell who you are addressing.

If you look at the threaded view, you will see this post linked to your post in the hierarchy, even though your post is not the most recent one as I am typing this.

The way to get the links correct is to use the "Reply with quote" or "Quick reply to this message" buttons on the post to which you are responding.

[Ether]

Originally Posted by dodar

Then this truly isnt the correct OPR calculations

I can assure you that the algorithm I use to compute the L2 norm of the simple linear combination of team scores (what we're calling OPR) is correct.

But that leaves two questions:

1) What is the "correct" data to use, and

2) Is the L2 norm of a linear combination of team scores the "correct" algorithm to get the most meaningful and useful metric?

Question 2 has been discussed in various threads here on CD in the past. I won't beat that horse here.

Question 1 is especially problematic this year because of the high value and erratic enforcement of fouls (I am not blaming the refs: this is a difficult game to ref and score). To get a truer measure of performance arguably requires that the foul points be removed from the score before computing the OPR. The problem is, you can't do this with the official data. You need to use the Twitter data to remove the foul points.

Ed Law maintains a spreadsheet in which he makes every effort to "repair" the Twitter data whenever possible and deciding when and when not to integrate it with the official FIRST Match Results and Team Standings data. It's apparently a labor of love for Ed and he spends many hours getting it right, for which we are in his debt. If I tried to do that, I'd probably introduce more errors than I corrected.

So if you're looking for the "official" OPR, rather than the "correct" OPR, I would say that Ed Law's spreadsheet is the de facto standard.

[Mike Bortfeldt]

Originally Posted by GearsOfFury

The bi-modal nature of this graph is really very surprising to me. I cannot think of why there would be such a discrete break around 0... What am I missing?

Mark,

I suspect the break around zero occurs due to penalties. All the matches with negative win margins were decided by penalties points and the number of matches where this occurs is a small subset of all the data. All the data with positive win margins include matches with no penalties, offsetting penalties or matches where the penalties did not affect the final outcome.

Mike

[Ether]

Originally Posted by Mike Bortfeldt

I suspect the break around zero occurs due to penalties.

Correct.

All the matches with negative win margins were decided by penalties points...

Correct.

...and the number of matches where this occurs is a small subset of all the data.

Some would say "Not small enough."

All the data with positive win margins include matches with no penalties, offsetting penalties or matches where the penalties did not affect the final outcome.

Correct.

I think Mark's question was why the curve changes so abruptly at the Y axis. I've been trying to come up with a good intuitive way to explain it. Any takers?

[Ether]

Originally Posted by Joseph1825

I thought I understood what these numbers meant, but I'm confused, so could someone explain somthing to me? I'm with team 1825 and the spreadsheet says we have an OPR of over 80 points (the blue alliance agrees.) but our CCWM is -40, could someone explain what those number mean? Thanks.

I'll try. What spreadsheet are you referring to?

[Ether]

Originally Posted by Joseph1825

I thought I understood what these numbers meant, but I'm confused, so could someone explain somthing to me? I'm with team 1825 and the spreadsheet says we have an OPR of over 80 points (the blue alliance agrees.) but our CCWM is -40, could someone explain what those number mean? Thanks.

I believe the spreadsheet you are referring to is OPR based on Twitter data 3-14-2014 203053 ET revB.XLS.

The numbers are calculated correctly. However: 1) that spreadsheet was prepared from Twitter data from Friday night before you had finished quals (on Saturday), and 2) Twitter contained only 5 of the 8 matches you played on Friday. For such a small data sample size, the OPR may be misleading.

To answer your questions:

Column B is the "standard" OPR, based on final score (which includes awarded foul points).

Column E is unpenalized CCWM... "unpenalized" in this context meaning the awarded foul points are removed from the final score, and CCWM meaning the opposing alliance's score (with awarded foul points removed) is subtracted from your alliance's score for each match. Many folks think removing the foul points gives a better metric for this year's game because of the high value and erratic enforcement* of penalties.

Note: AIUI, Ed does not use the Twitter data to remove foul points if it is not complete for that event. So you won't see unpenalized CCWM for MOKC in his spreadsheet.

update:

Originally Posted by Ed Law

4) Quite a few regional/districts this week have incomplete twitter data so adjusted OPRs cannot be calculated for those events..

* I am not faulting the refs. This is a very difficult game to ref and score.

[MikeE]

Originally Posted by Ether

I think Mark's question was why the curve changes so abruptly at the Y axis. I've been trying to come up with a good intuitive way to explain it. Any takers?

I thought Mike Bortfeldt explained that well enough.

We're not seeing a break so much as a graph representing two populations of different sizes.
Everything to the right of the Y-axis shows the fairly expected distribution of winning margins. This represents just under 90% of all matches.

The data to the left of the Y-axis shows a similar albeit reflected pattern. It is scaled down in frequency since it comes from the ~11% of matches that would have a different winner without the penalties.

There are some other effects due to penalties being larger and more quantized than the point value of scoring objectives, but the main cause is due to sub-population size differences.

[GearsOfFury]

Originally Posted by Ether

It looked a bit strange to me at first, so I re-created the graph manually directly from the Twitter data and got the same result. I can post the spreadsheet later this evening if you're interested.

I would be interested, yes, thanks.

In the meantime, can you overplot this graph with the penalized version? And (separately) the histogram of penalties?

[GearsOfFury]

Originally Posted by MikeE

I thought Mike Bortfeldt explained that well enough.

We're not seeing a break so much as a graph representing two populations of different sizes.
Everything to the right of the Y-axis shows the fairly expected distribution of winning margins. This represents just under 90% of all matches.

The data to the left of the Y-axis shows a similar albeit reflected pattern. It is scaled down in frequency since it comes from the ~11% of matches that would have a different winner without the penalties.

There are some other effects due to penalties being larger and more quantized than the point value of scoring objectives, but the main cause is due to sub-population size differences.

The plot is showing frequency, not percentage. In a "normal" world, the winning margin would be normally distributed about some mean, and so would the penalties. When the penalties are applied, the winning margin would "shift" to the right - but there would not be a large, discrete jump at 0.

In our "less normal" world, we probably have a large skew to the left - the winning margin is far more likely to be small than large. But, I would expect it to still be more or less continuous (as you point out, there are quantization effects because of the scoring objectives... just as scores of say 4 or 5 in football are unlikely). I would expect the penalty point distribution to be continuous, as well, but with even larger gaps between likely values.

When the penalties are subtracted from the penalized score, I expect the resulting distribution to be continuous. The jump right at zero is not expected.

I'll withhold my tin foil hat theories as to why this is until I can take a look at Ester's raw data.

[Mike Bortfeldt]

Originally Posted by MikeE

...

Mike,

Thank you for a better explanation. Perhaps a different way to look this graph is to plot the data as two separate sets. The first being all matches where there were no foul points. The second with the matches that had foul points.

Mike

[Ether]

Twitter winning margin histograms

As requested, attached is an Excel XLS spreadsheet using non-tied qual matches from Twitter data 3/16 16:44:46. Included are histograms of frequency (counts) versus:

wm: winning margin (with awarded foul points included)

wmu: unpenalized winning margin (with foul points removed)

wfm: winning foul margin (winner's awarded foul points minus loser's awarded foul points)

All the necessary raw data is in the spreadsheet, as well as the derived data and the formulas used to compute it. You can play around with it to see if there's a better way to present it.

Other than removing ties, I made no further effort to modify the Twitter data.

The usual Twitter data caveats apply.

[Andrew Schreiber]

Ether, maybe I'm just blind as a bat (likely) but have you posted raw data for twitter? I'm trying to avoid going to the effort of scraping it if I can.

[Ether]

Originally Posted by Andrew Schreiber

Ether, maybe I'm just blind as a bat (likely) but have you posted raw data for twitter? I'm trying to avoid going to the effort of scraping it if I can.

http://www.chiefdelphi.com/forums/sh...7&postcount=11

Thank you, Brandon


Here's an XLS version with practice matches removed