2014 OPR After Week One Events
 

The OPR/CCWM numbers have been posted, please see

http://www.chiefdelphi.com/media/papers/2174

There are a number of points I would like to make:

1) Even though week one events are not completed, I am publishing this because there was a minor problem with the macro. It is now fixed. I also fixed some minor bugs.

2) Team Standings for Centerline is not posted on FIRST website. I posted what the FTA gave me but I am not certain the subscores are in the right columns. There were only two columns in the FTA report and I guess they are for Autonomous points and Teleop points. I will update them once the Team Standings are posted on the FIRST website.

3) We already broke the 100 mark for OPR. Congratulations Jim.

If you find any error or have any questions, please let me know.

Re: 2014 OPR After Week One Events
 

Thanks Ed, although I do not believe that the OPR numbers truly reflect teams' ability properly because of the penalty results being included in the Teleop numbers. The 50 pt technical fouls are big enough to skew the OPR results. There were matches at Southfield with up to 150 points in penalties per side. We will go through today using the twitter feed data to remove the penalties and recompute. I suspect our actual OPR is more like 90.

Someone should attempt to build this feature into one of the various OPR calculators. Without this, the 2014 OPR data will lack integrity (unless, of course, if the GDC fixes the broken penalty rules).

Re: 2014 OPR After Week One Events
 

Would anyone like to speculate on how OPR fits true robot scoring abilities or alliance contributions this year compared to previous years?

One thing I see that is different this year is that any particular alliance can do much better (or much worse) than the sum of the individual teams since robots can't do their own thing for an entire match.

Re: 2014 OPR After Week One Events
 

I have a script that can do that quickly. I'll run the numbers and post here.

Re: 2014 OPR After Week One Events
 

I will update my macro to remove the foul points and recompute the OPR. Ether, can you send me the twitter data that you use or post it here so our numbers will match?

Re: 2014 OPR After Week One Events
 

I think it will be decent this year if penalties are removed. Of course, this would remove the "power" of a design that is good at drawing penalties (think taking a drive in basketball). After watching how some events were officiating, Robowranglers 2008 bot could have been the highest scoring bot this weekend...

Defense is also has an incredible amount of influence. For first timer events (most regionals and 1st round of districts ie weeks 1-3), defensive play usually does not develop well until rounds 3-5. This causes a shift in scoring ability which OPR does not handle well (it assumes even play throughout). This will be more accurate in later season events where Defensive play starts earlier.

The District/State Championships will likely see OPR growth in qualifying similar to 2010. In 2010, 3 bots that were 2s would get together and put up 8-9 pts. 2011 was the opposite, and thus 3 60 pt. bots would only get 135.

I say likely as the defense at District Champs and State champs will also likely increase. 2010 defense was very difficult and not terribly effective where as 2014 defense is much more obvious and effective. Teams at these events will also likely have the ability to muti-task. IE, Inbound, and then play strong defense. So it is really a question of "Can in the improved coordination among good teams outpace drivers defensive abilities"... We ahve several weeks to watch this develop.

My $0.02 is scoring OPR will not be as accurrate as 2008, 2010, or 2012 (adj.), or 2013 but likely on par with 2011 and better than 2009.

************************************************** *******
Now calculating Penalty OPR will likely be a very valuable tool for 2nd round district events and Championship. HINT, if your teammate is average 1 50 pt. penalty a match, you might want to have a talk before the match starts....

I suspect some adjustment by GDC to the big penalty contributors which will likely put an * next to week 1 OPRs.

Re: 2014 OPR After Week One Events
 

Attached is my OPR calculation with and without foul points, based on Qual matches only, from Twitter data as of Saturday March 1 at 20:22:36

659 matches

360 teams

Re: 2014 OPR After Week One Events
 

For some reason, when I type my team (3175)'s number into the query, it doesn't show the numbers. Can someone point me in the right direction?

Re: 2014 OPR After Week One Events
 

Data hasn't been aggregated yet. You can view data on individual event tabs at the bottom.

Re: 2014 OPR After Week One Events
 

Thank you for this.

Can someone make a quick plot of Ether's qualifying match OPR (without foul points) vs. seed rank?

Re: 2014 OPR After Week One Events
 

OPR Distribution for unpenalized score data set Week 1 from Ether.
Distribution follows typical FRC trends as expected.
This appears to a game independent phenomenon, it is essentially the same in every FRC game I have ever studied. The game design affects the scale and the zero crossing point, but the curve shape is always pretty much the same. The GDC does not seem to realize this when they set the penalty scaling.

Count = 360 Teams,
Average Score = 19.0
Median score = 13.1
Max OPR = 91.6 (Go Bees! :) )
Min OPR = -13.0
STDev = 19.6
SNR = -0.3

The Average OPR is less than one standard deviation above zero.
A single technical foul is worth more than 2 matches worth of average team contribution. This is completely out of scale with what is appropriate for penalty scaling.

Re: 2014 OPR After Week One Events
 

Jim beat me to it, but here is a comparison of the distribution with and without fouls.

Re: 2014 OPR After Week One Events
 

does it matter that a bunch of data from Southfield elims didn't get into the twitter feed?

Re: 2014 OPR After Week One Events
 

No, Elim data is not used in these analyses. Only Qual data. We are missing data from a few of the week one events. 469 teams played but we only have twitter data on 360 of them, but this is a big enough sample set for now.

Re: 2014 OPR After Week One Events
 

Could you possibly run the calculation again with Sunday data for us Hatboro-Horsham teams?

Re: 2014 OPR After Week One Events
 

There is no Twitter data available for Hatboro-Horsham.

Re: 2014 OPR After Week One Events
 

You know, I'm actually quite surprised at how accurate OPR is for Palmetto. What we're seeing in the data matches our team's scouting data for the most part. I was under the impression that OPR would not be a good indicator, but I suppose that is dependent on the strength of every team at an event. I suppose the stronger the event is, the less accurate OPR becomes as a statistic.

Re: 2014 OPR After Week One Events
 

The OPR/CCWM week 1 numbers have been updated. Please go to

http://www.chiefdelphi.com/media/papers/2174

There are a number of points I would like to make:

1) Team Standings for Centerline is not posted on FIRST website. I posted what the FTA gave me but I am not certain the subscores are in the right columns. There were only two columns in the FTA report and I guess they are for Autonomous points and Teleop points. I will update them once the Team Standings are posted on the FIRST website.

2) Team Standings for Greater Toronto West has incomplete data. I did not include these data in Query.

3) I added the Adjusted OPR in column AE for each event, which removes the Foul Points from the total score before computing OPR. The foul points came from Twitter data. I did not use the match score from Twitter data because they are not 100% accurate. For events without twitter data, no Adjusted OPR is computed.

4) World ranking is computed using OPR because some events do not have twitter data to calculate Adjusted OPR.

5) I added a tab called Adjusted OPR Results. Events without twitter will show 0 for Adjusted OPR.



If you find any error or have any questions, please let me know.

Re: 2014 OPR After Week One Events
 

It should be pretty excellent at giving a rating for a team's contributions this year, and in fact should be fantastic at finding teams who contribute to their alliance without doing the prestige tasks. A team that adds two assists on every ball but doesn't shoot it through the goal will get a good OPR.

However: even moreso than in past years, you won't want to scout with it. A team might get a 60-70pt OPR without ever firing a ball. If you also don't shoot a ball, you probably want to find a shooter or a low-goal specializer. If you already have a fantastic shooter, you may not want to pick a team that earned their OPR with a shooter. In past years you could maybe get away with picking the top 3 in OPR for an alliance, but with such different functionalities and roles, you'll want to make sure that everyone is compatible.

Also, keep in mind that you can't always compare across events with OPR - an event where everyone tends to play offense and keep to themselves will elevate all their robots OPRs, compared to an event with heavier defense.

Another thought about the penalties: It would be interesting to not only remove the penalties from scores (as has been done), but to actually subtract them from the team that caused them when putting inputs into an OPR-solver. This would help find teams that are racking up a lot of penalties, since their OPRs would go much lower.

tl,dr: OPR will be great for entertainment purposes to do comparisons across robots who do different tasks, but poor for scouting.

Re: 2014 OPR After Week One Events
 

Any particular reason that GSDE isn't included in this data?

Re: 2014 OPR After Week One Events
 

Or you could just do an "OPR" on foul points.

(Note: This uses Twitter data. The usual Twitter data caveats apply).

Re: 2014 OPR After Week One Events
 

Just so I understand the "OPR" on foul points, is lower OPR a good thing?

(Thanks to all contributors for this data. Very interesting stuff.)

Re: 2014 OPR After Week One Events
 

Since the point of the computation was to try to estimate the fouls committed by each team, lower is better.

See cell G1 in the spreadsheet.

Re: 2014 OPR After Week One Events
 

Sigh... I can no longer complain about my customers who do not read the instructions.

Thank you.

Re: 2014 OPR After Week One Events
 

At Inland Empire, we had a robot that only had a drive train playing only defense--no shooter or pick up mechanism with a middling OPR (and ended up an alliance captain), and another that was having quite an offensive day (our 3rd alliance member) with a low OPR despite showing high in our scouting data. The OPR this year seems to be greatly influenced by the schedule and win-loss record. And with 13 matches apiece, this was the best possible statistical mix of any event.

Re: 2014 OPR After Week One Events
 

Thanks Ed, and Jim, and everyone that is working on this. Ed, is the Center Line data updated yet? Thanks for taking out the penalty points. They were ridiculous! (and that's all I will say on that subject here) Can you please let me know when it is?
Thanks!!!

Re: 2014 OPR After Week One Events
 

Im not that good at using excel, but I can't find the adjusted OPR for Hatboro even though the regular OPR data is there. Is this due to the lack of twitter data as Ether said?

Re: 2014 OPR After Week One Events
 

Yes. The Twitter data has foul points. The Team Standings data lumps foul points together with TeleOp Goal points.

Re: 2014 OPR After Week One Events
 

As regards teams with negative "Foul OPR:"

Does that essentially mean that not only do they rarely (if ever) commit fouls, but also that their presence on an alliance reduces the chance/severity of their allies committing fouls as well?

If so, we should really be asking teams 2493 and 4981 what they're doing to keep the game clean. =]

Re: 2014 OPR After Week One Events
 

Statistically, yes. But all the usual OPR caveats apply.

Re: 2014 OPR After Week One Events
 

As a team with one of the most negative foul OPRs: yes, we strategize with fouls in mind. We noticed teams that tend to foul and negotiated with them to not do what they've done in the past to get fouls. For example, we noticed that one of our future partners was getting fouls by deploying their ball grabber while playing defense. In our match with them, we asked them to not deploy their grabbers under any circumstance even if they were near a ball (their role that match was purely defense).

Other times we've had teams that tend to foul play an assist role in order to keep them away from playing defense.

If you scout well and have some good data (and persuasive skills) you can really help keep your foul score down. This is nothing new this year - we've been doing it for years.

Re: 2014 OPR After Week One Events
 

But, please don't forget that not all fouls were called the same. At Center Line, if you had your arm down roping a ball and someone rammed into you, you were still given the foul, despite their G14. Once word of this got around, defensive robots used it to get 50 points on the offensive robots. That is quite evident in the fact that the blue alliances dominated our elims. When 3 robots are destroyed in qf's and the only penalty points are against those 3 robots, something is very wrong!

Re: 2014 OPR After Week One Events
 

Does anybody have the Team Standings results of Greater Toronto West after all 60 Qualification matches. The FIRST website only shows the Team Standings after 51 matches. We are not going to have their data in this year if they don't post it or I will just put unknown for team rankings.

Re: 2014 OPR After Week One Events
 

The Twitter data includes ONTO2 data for Qual matches 49,50,51 only.