Week3 cumulative Twitter stats & OPRs
 

Stats and OPRs based on Twitter Qual match data as of Fri 3-14-2014 22:30:53 ET.

The usual Twitter data caveats apply.


NOTE: LOOK FOR UPDATED DATA TO BE POSTED LATER IN THIS THREAD

Re: Week3 cumulative Twitter stats & OPRs
 

Excerpt from the above post:

win foul points awarded:
20 average (mean)
220 max

lose foul points awarded:
5 average (mean)
110 max

The above is a very simple assessment of how the foul points affect this game. 4X as many foul points to the winning alliance....this means that fouls swing a large percentage of matches.

50 point foul is effectively a automatic loss most of the time.
Tragic.
Worst game design since 2003, yet no real action to correct....
Tragic.

Re: Week3 cumulative Twitter stats & OPRs
 

20 foul points on average for the winning alliance. That is insane. What makes it worse, is that by the look at average scores, they are not a ton different then last year, but the fouls are worth a ton more. I think the game designers expected higher average match scores to make up for higher foul scores.

Re: Week3 cumulative Twitter stats & OPRs
 

Agreed that the impact of foul points is tragic... and that foul points swinging the outcome of the match is likely a significant contributor to this statistic; however, I do think there is another possible/probable cause.

Fouls "take advantage" of less experienced, lower-performing teams. The teams that are more likely to get careless HP fouls, struggle to pickup a ball when inbounding, play hard defense that involves a manipulator flipping out and getting inside an offensive bot, or pin for too long are the teams that are less familiar with the rules, haven't kept up with the Q&A, and haven't designed as effective an offensive bot.

The fact that the teams most likely to suffer from the heavy fouls are also the teams that FIRST is having the hardest time retaining isn't a good thing for FIRST growing at or above average next year.

Consider me more and more on the side of "bring foul values down to 10 and 30." Keep in mind this is still 333% and 150% the value of fouls in 2013.

Big, Bad, Bobth Post! (#319)

Re: Week3 cumulative Twitter stats & OPRs
 

Agreed. Quite sad to see some really good strategies play out and then that one tech foul swings it the other way.
I think a 30 point value should be the max assigned to the tech foul. Hopefully, based on average win/loss margins it will have a smaller swing effect on match outcomes.

Re: Week3 cumulative Twitter stats & OPRs
 

Twitter Qual Match Stats

based on Twitter Qual match data as of Sat 15 Mar 2014 20:13:09 ET.

The usual Twitter data caveats apply.

I've removed the Quartile data from the stats report. In their place I will be posting histograms later this evening. Also Twitter-based OPRs. So check back later.

Re: Week3 cumulative Twitter stats & OPRs
 

Re: Week3 cumulative Twitter stats & OPRs
 

OPR for Final, Foul, Auto, & TeleOp

based on Twitter Qual match data as of Sat 15 Mar 2014 20:56:33 ET.

The usual Twitter data caveats apply.

Data is shown for teams which have played at least 6 matches.

Re: Week3 cumulative Twitter stats & OPRs
 

Very well presented, your picture is worth a thousand words! Would you mind if I combine your charts with my questionnaire results for submission to FIRST tomorrow?

The questionnaire can be found here.

[would it be possible to calculate the percentage of games finished within +/- 50 points? i.e. a tech foul call or non-call would have swung the results of that game?]

Re: Week3 cumulative Twitter stats & OPRs
 

^^^ This says it all. Thank you Nathan. ^^^

Re: Week3 cumulative Twitter stats & OPRs
 

You have my permission.

See attached chart. Of the 3022 non-tied qual matches in the Twitter data dated 3/15/2014 20:13:09 ET, there were 1619 matches with final score (including awarded foul points) margin less than or equal to 50.

The usual Twitter data caveats apply.

Re: Week3 cumulative Twitter stats & OPRs
 

Alliance Score Residuals

based on Twitter Qual match data as of Sat 15 Mar 2014 20:56:33 ET.

Example graph interpretation:

81 - 18 = 62% of Alliance Unpenalized Scores were within +/- 20 points of the "OPR" predicted value.

66 - 32 = 35% of Alliance Unpenalized Scores were within +/- 10 points of the "OPR" predicted value.

The usual Twitter data caveats apply.

Re: Week3 cumulative Twitter stats & OPRs
 

This can be a double whammy for District teams on the cusp of making it to the District Championship. The points for winning qualifying matches can function as the tie-breaker for determining who makes the cut-off and who doesn't. Your team can do everything right, but if a partner makes a mistake or doesn't know the rule, there goes 2 points. (And vice versa, you can do everything wrong and still get 2 points!) We do not have the manpower or experience to pull a 51 and record every foul and coach other teams through their problem areas. At our next event we will make a point to bring it up before every match so everyone knows.

At the end of the season I would be curious to see how many teams make or do not make the DCMP by foul points.

Re: Week3 cumulative Twitter stats & OPRs
 

I agree that would be an interesting metric, although the match data doesn't show how deserving the fouls were or how consistently they were called.

As you are probably aware the twitter data (caveats apply) shows 1778 lost one match and won another at Mt Vernon due to fouls, so no net impact.

Re: Week3 cumulative Twitter stats & OPRs
 

Yep, and that make me wonder if they are uniformly enough distributed so that it is a net zero effect. I think the safest thing to do is assume that they are deserved. At Mr. Vernon I thought the refs did a great job.

Re: Week3 cumulative Twitter stats & OPRs
 

Red wins 1935; Blue wins 1722; Ties 26

based on Twitter Qual and Elim match data as of Sun 16 Mar 2014 15:32:18 ET.

The usual Twitter data caveats apply.

Re: Week3 cumulative Twitter stats & OPRs
 

Does this include eliminations? If so, that would make sense.

Re: Week3 cumulative Twitter stats & OPRs
 

The long term trends have been that Qual performance for red is 50% and elimination is ~65%.

Re: Week3 cumulative Twitter stats & OPRs
 

Elims only: red wins 415 (66.8%); blue wins 206 (33.2%); ties 3

Quals only: red wins 1520 (50.1%); blue wins 1516 (49.9%); ties 23

Twitter data 3/16 15:32:18

Re: Week3 cumulative Twitter stats & OPRs
 

I thought I understood what these numbers meant, but I'm confused, so could someone explain somthing to me? I'm with team 1825 and the spreadsheet says we have an OPR of over 80 points (the blue alliance agrees.) but our CCWM is -40, could someone explain what those number mean? Thanks.

Re: Week3 cumulative Twitter stats & OPRs
 

I dont think you took out duplicate matches from replays.

Re: Week3 cumulative Twitter stats & OPRs
 

Are you addressing Joseph1825 or me?

Re: Week3 cumulative Twitter stats & OPRs
 

You.

Re: Week3 cumulative Twitter stats & OPRs
 

Your post had no context, and it was linked to Joseph1825's post #20.

Could not tell to whom the pronoun "you" was referring.

Re: Week3 cumulative Twitter stats & OPRs
 

"The usual Twitter data caveats apply"

Twitter data is known to have omissions and redundancies and possibly a few errors.

Re: Week3 cumulative Twitter stats & OPRs
 

Then this truly isnt the correct OPR calculations.

And your post before said I linked my other post to that Joseph's post, but I did so such thing.

Re: Week3 cumulative Twitter stats & OPRs
 

Perhaps unknowingly, but yes you did.

Re: Week3 cumulative Twitter stats & OPRs
 

You are reading that wrongly. That shows me posting after him. If you look to see that one of your posts comes all the way down from mine, that is one where you respond directly to me.

All I did was click Post Reply at the very bottom of the page.

Re: Week3 cumulative Twitter stats & OPRs
 

Is the twitter data any different from the FRC-Spy data?

The bi-modal nature of this graph is really very surprising to me. I cannot think of why there would be such a discrete break around 0... What am I missing?

Great graph, in any case! Thank you for posting it.

Re: Week3 cumulative Twitter stats & OPRs
 

It looked a bit strange to me at first, so I re-created the graph manually directly from the Twitter data and got the same result. I can post the spreadsheet later this evening if you're interested.

Re: Week3 cumulative Twitter stats & OPRs
 

Same thing.

Re: Week3 cumulative Twitter stats & OPRs
 

No, I am reading it correctly. The threaded view is a hierarchy. The intent is to show which post you are responding to.

That's the problem. When you do that, it links your post in the threaded view to whatever is the most recent post in the thread, even if that isn't the post you are responding to. That might be OK if you include context in your post like quoting a section of the post to which you are responding or use screen names instead of pronouns. Otherwise there is no way to tell who you are addressing.

If you look at the threaded view, you will see this post linked to your post in the hierarchy, even though your post is not the most recent one as I am typing this.

The way to get the links correct is to use the "Reply with quote" or "Quick reply to this message" buttons on the post to which you are responding.

Re: Week3 cumulative Twitter stats & OPRs
 

I can assure you that the algorithm I use to compute the L2 norm of the simple linear combination of team scores (what we're calling OPR) is correct.

But that leaves two questions:

1) What is the "correct" data to use, and

2) Is the L2 norm of a linear combination of team scores the "correct" algorithm to get the most meaningful and useful metric?

Question 2 has been discussed in various threads here on CD in the past. I won't beat that horse here.

Question 1 is especially problematic this year because of the high value and erratic enforcement of fouls (I am not blaming the refs: this is a difficult game to ref and score). To get a truer measure of performance arguably requires that the foul points be removed from the score before computing the OPR. The problem is, you can't do this with the official data. You need to use the Twitter data to remove the foul points.

Ed Law maintains a spreadsheet in which he makes every effort to "repair" the Twitter data whenever possible and deciding when and when not to integrate it with the official FIRST Match Results and Team Standings data. It's apparently a labor of love for Ed and he spends many hours getting it right, for which we are in his debt. If I tried to do that, I'd probably introduce more errors than I corrected.

So if you're looking for the "official" OPR, rather than the "correct" OPR, I would say that Ed Law's spreadsheet is the de facto standard.

Re: Week3 cumulative Twitter stats & OPRs
 

Mark,

I suspect the break around zero occurs due to penalties. All the matches with negative win margins were decided by penalties points and the number of matches where this occurs is a small subset of all the data. All the data with positive win margins include matches with no penalties, offsetting penalties or matches where the penalties did not affect the final outcome.

Mike

Re: Week3 cumulative Twitter stats & OPRs
 

Correct.

Correct.

Some would say "Not small enough."

Correct.

I think Mark's question was why the curve changes so abruptly at the Y axis. I've been trying to come up with a good intuitive way to explain it. Any takers?

Re: Week3 cumulative Twitter stats & OPRs
 

I'll try. What spreadsheet are you referring to?

Re: Week3 cumulative Twitter stats & OPRs
 

I believe the spreadsheet you are referring to is OPR based on Twitter data 3-14-2014 203053 ET revB.XLS.

The numbers are calculated correctly. However: 1) that spreadsheet was prepared from Twitter data from Friday night before you had finished quals (on Saturday), and 2) Twitter contained only 5 of the 8 matches you played on Friday. For such a small data sample size, the OPR may be misleading.

To answer your questions:

Column B is the "standard" OPR, based on final score (which includes awarded foul points).

Column E is unpenalized CCWM... "unpenalized" in this context meaning the awarded foul points are removed from the final score, and CCWM meaning the opposing alliance's score (with awarded foul points removed) is subtracted from your alliance's score for each match. Many folks think removing the foul points gives a better metric for this year's game because of the high value and erratic enforcement* of penalties.

Note: AIUI, Ed does not use the Twitter data to remove foul points if it is not complete for that event. So you won't see unpenalized CCWM for MOKC in his spreadsheet.

update:

* I am not faulting the refs. This is a very difficult game to ref and score.

Re: Week3 cumulative Twitter stats & OPRs
 

I thought Mike Bortfeldt explained that well enough.

We're not seeing a break so much as a graph representing two populations of different sizes.
Everything to the right of the Y-axis shows the fairly expected distribution of winning margins. This represents just under 90% of all matches.

The data to the left of the Y-axis shows a similar albeit reflected pattern. It is scaled down in frequency since it comes from the ~11% of matches that would have a different winner without the penalties.

There are some other effects due to penalties being larger and more quantized than the point value of scoring objectives, but the main cause is due to sub-population size differences.

Re: Week3 cumulative Twitter stats & OPRs
 

I would be interested, yes, thanks.

In the meantime, can you overplot this graph with the penalized version? And (separately) the histogram of penalties?

Re: Week3 cumulative Twitter stats & OPRs
 

The plot is showing frequency, not percentage. In a "normal" world, the winning margin would be normally distributed about some mean, and so would the penalties. When the penalties are applied, the winning margin would "shift" to the right - but there would not be a large, discrete jump at 0.

In our "less normal" world, we probably have a large skew to the left - the winning margin is far more likely to be small than large. But, I would expect it to still be more or less continuous (as you point out, there are quantization effects because of the scoring objectives... just as scores of say 4 or 5 in football are unlikely). I would expect the penalty point distribution to be continuous, as well, but with even larger gaps between likely values.

When the penalties are subtracted from the penalized score, I expect the resulting distribution to be continuous. The jump right at zero is not expected.

I'll withhold my tin foil hat theories as to why this is until I can take a look at Ester's raw data.

Re: Week3 cumulative Twitter stats & OPRs
 

Mike,

Thank you for a better explanation. Perhaps a different way to look this graph is to plot the data as two separate sets. The first being all matches where there were no foul points. The second with the matches that had foul points.

Mike

Re: Week3 cumulative Twitter stats & OPRs
 

Twitter winning margin histograms

As requested, attached is an Excel XLS spreadsheet using non-tied qual matches from Twitter data 3/16 16:44:46. Included are histograms of frequency (counts) versus:

wm: winning margin (with awarded foul points included)

wmu: unpenalized winning margin (with foul points removed)

wfm: winning foul margin (winner's awarded foul points minus loser's awarded foul points)

All the necessary raw data is in the spreadsheet, as well as the derived data and the formulas used to compute it. You can play around with it to see if there's a better way to present it.

Other than removing ties, I made no further effort to modify the Twitter data.

The usual Twitter data caveats apply.

Re: Week3 cumulative Twitter stats & OPRs
 

Ether, maybe I'm just blind as a bat (likely) but have you posted raw data for twitter? I'm trying to avoid going to the effort of scraping it if I can.

Re: Week3 cumulative Twitter stats & OPRs
 

http://www.chiefdelphi.com/forums/sh...7&postcount=11

Thank you, Brandon :)


Here's an XLS version with practice matches removed

Re: Week3 cumulative Twitter stats & OPRs
 

List of 430 Qual & Elim Matches whose outcome was affected by fouls

From Twitter data 3/16/2014 16:44:46

The usual Twitter data caveats apply.

Re: Week3 cumulative Twitter stats & OPRs
 

Attached is a cumulative percent histogram using updated Twitter data 16Mar2014 16:44:46

81.5 - 18.7 = ~63% of Alliance unpenalized scores were within +/-20 points of the "OPR predicted" (L2 norm) value.

Re: Week3 cumulative Twitter stats & OPRs
 

Key:

Re: Week3 cumulative Twitter stats & OPRs
 

Team-by-team analysis of foul points on Win/Loss/Tie

based on Qual & Elim matches from Twitter data Sun 16 Mar 2014 16:44:46

Top "beneficiaries":
Top "victims":
Complete list attached.

The usual Twitter data caveats apply.

Re: Week3 cumulative Twitter stats & OPRs
 

Has anyone located a lookup table with district event codes? and which region each district event is in?

Thanks!

Dan Hudnut, mentor
Team 885, Randolph Ctr., VT

Re: Week3 cumulative Twitter stats & OPRs
 

http://frclinks.frclinks.com/

Re: Week3 cumulative Twitter stats & OPRs
 

Besides FRCLinks, you can download the Alliance Selection Spreadsheet and do a lookup there. It might be more convenient then parsing a webpage. For district events, the begining of the name contains the area, ie MI, NE, PNW, or MAR.

Re: Week3 cumulative Twitter stats & OPRs
 

Week4 Twitter data analyses available here. More to come later this evening.

Re: Week3 cumulative Twitter stats & OPRs
 

Thank you both!

Re: Week3 cumulative Twitter stats & OPRs
 

If you need a historical one, I have one posted in CD-Media: Event Codes for FRC Events 2002-2014