View Full Version : Teams scoring vast majority of points
I was joking with some students that Recycle Rush is a good model of the US Economic system, where a small percentage of participants control a vast majority of the wealth... but maybe its not a joke.
Has anyone looked at what % of teams score 80% of the points? Is there a way to easily looks at how many total points team XXXX scored at an event compared to the total points scored at that event?
At the LA Regional, Team 330, Team 1197, and Team 973 seemed to overshadow the scoring of the other 63 teams. Maybe they didn't score 80% of all points scored, but it was quite a bit.
Challenge: calculate, for each regional, what percent of ALL points scored were scored by the top 3 SCORING teams. (this is not just top three ranked by average score, but top three individual scorers.)
Jacob Bendicksen
20-03-2015, 15:24
I would absolutely love to see this (and I have a hunch that you're right), but this requires data beyond what's available through FIRST. We'd need to find at least one team per event (preferably more) that has team-level data for every match, then go from there.
Actually, now that I think about it, maybe the good people behind frcscout.com (http://frcscout.com) could make this happen - they've got a lot of data in a standardized format.
You could try it with OPR. Not perfect, but might be interesting anyway. Insert usual caveats about OPR.
JamesCH95
20-03-2015, 16:07
I would suggest OPR as well. It's the best way we have so far to tease out each teams' individual contributions without actually watching every match.
Rachel Lim
20-03-2015, 16:28
I decided to do some quick analysis with QA data from weeks 1 - 3 since I had it (and didn't have OPR):
The top 10 teams have a higher total QA as the bottom 62 teams
The top 0.5% of teams have a higher total QA as the bottom 3% of teams
The top 20 teams have a higher total QA as the bottom 104 teams
The top 1% of teams have a higher total QA as the bottom 5% of teams
The top 241 teams have 80% of the total QA
The top 11.81% of teams have 80% of the total QA
1114's QA at Waterloo now (197.87) is the higher than the sum of the 13 lowest QAs from weeks 1 - 3.
It's actually not as unevenly distributed as I expected. I'm guessing the fact that QAs are averaged, and therefore top teams can get pulled down by lower scoring partners and vice versa plays a role. It'll definitely be interesting to see how OPR works out.
Notes:
"top" and "bottom" just refer to the highest and lowest QAs, not to any other aspects of those teams.
Percentages for the first two statistics are rounded to the nearest 0.5%
Yeah, looking at QA or OPR is pretty much all that can be done, unless someone is taking perfect scouting data at an event.
Looking up FRCScout.com there are some errors (330 has zero points listed for some matches that definitely weren't zero). Maybe I'll try to get this hooked up for the new Ventura Regional next week. Should be some heavy hitters outscoring a majority of teams.
The other Gabe
20-03-2015, 17:15
it's kinda this way every year...
It'll definitely be interesting to see how OPR works out.
% of OPR sum vs % of Teams for weeks 1 thru 3 Match Results data
% of OPR sum vs % of Teams for weeks 1 thru 3 Match Results data
Maybe I'm not "getting" this graph. It seems to show that as you include all teams in the OPR sum, you get the total OPR sum...
% of OPR sum vs % of Teams for weeks 1 thru 3 Match Results data
OPR Histogram
% of OPR sum vs % of Teams for weeks 1 thru 3 Match Results data
Maybe I'm not "getting" this graph. It seems to show that as you include all teams in the OPR sum, you get the total OPR sum...
You asked for:
what % of teams score 80% of the points?
... and that's what the plot shows (based on OPR, as suggested by previous posters)
On the Y axis look for 80%. Trace horizontally from there until you hit the red line, then go straight down. The answer is 80% of the total OPR is due to 47% of the teams.
It's interesting to see a cumulative distribution go above 100% then decline!
I understand why that's happening (due to negative OPR) but it does underpin that OPR can be an odd beast.
You asked for:
... and that's what the plot shows (based on OPR, as suggested by previous posters)
On the Y axis look for 80%. Trace horizontally from there until you hit the red line, then go straight down. The answer is 80% of the total OPR is due to 47% of the teams.
Is the reason % of OPS sum crosses greater than 100% at ~80% of teams due to interpolation error? Is it true that 80% of teams account for 100% of OPR (in other words, 1 in 5 teams never score anything)
Not sure OPR is necessarily the best metric to use here given that all teams from an alliance benefit from an individual's good performance. Alliances make the individual harder to judge (a team only capable of "capping" stacks with bins but not producing their own will only score well when teamed with a robot that can make large stacks, for instance). OPR is negative for some teams, while points scored cannot be negative (except for penalties).
In other words, I feel the actual scoring potential of the best teams (the 1%) do produce more disparity than the 99% compared to Ether's graph. But without more specific data this is not too knowable.
Is the reason % of OPS sum crosses greater than 100% at ~80% of teams due to interpolation error?
No. It's because some teams have negative OPR.
Is it true that 80% of teams account for 100% of OPR
The sum of the OPRs of the Top (highest OPR) ~78% of the teams equals 100% of the sum of the OPRs of all the teams.
(in other words, 1 in 5 teams never score anything)
You can't draw that conclusion from the graph. That's not how OPR works.
Not sure OPR is necessarily the best metric to use here
This was discussed in earlier posts in this thread. Can you suggest something better, for which the data is available?
Rachel Lim
20-03-2015, 18:40
% of OPR sum vs % of Teams for weeks 1 thru 3 Match Results data
That's really cool, thank you! The negative OPR threw me off for a while; it's strange how the top 90% has a higher total OPR than all teams.
Just some statistics I got from OPR data (I used column G on the "OPR results" tab from Ed Law's spreadsheet):
The top 10 teams have a higher total OPR than the bottom 516 teams
The top 0.5% of teams have a higher total OPR than the bottom 28.5% of teams
The top 20 teams have a higher total OPR than the bottom 603 teams
The top 1% of teams have a higher total OPR than the bottom 33% of teams
The top 845 teams (47%) have 80% of the total OPR
The top 388 teams (21%) have 50% of the total OPR
The top 143 teams (8%) have 25% of the total OPR
What's really interesting is that following the stats above compared to the ones I had about QA, OPR seems to peak off more quickly, but have a larger "middle" section. I feel like part of the issue is that the OPR data I had only took the higher value for teams that have competed twice, as compared to the QA which had everything.
Graphing OPR vs QA gave me this:
18691
Which seems to have a steeper curve in OPR, but not by as much as the stats imply. If I have some time I'll redo the OPR calculations with full data and check what I did for the QA ones.
This was discussed in earlier posts in this thread. Can you suggest something better, for which the data is available?
Could try this if you can point me to the raw data (I only found the Team 2834 OPR generation scouting database):
Calculate an "effective QA" for each team by:
- For each match, sum up the final QA result of all teams in the alliance
- For each team in the alliance, their personal contribution is estimated as a percentage of their alliance's score proportional to the sum of the alliance team's original QA
- Calculate effective individual QA by averaging all matches in their competition (to normalize and account for different # of matches played at different events)
For example:
Team 1 QA = 95
Team 2 QA = 38
Team 3 QA = 56
Sum is 189
Match 1 Score = 87
Match 1, Team 1 "effective individual QA" = 95/189 * 87 = 43.7
Match 1, Team 2 "effective individual QA" = 38/189 * 87 = 17.5
Match 1, Team 3 "effective individual QA" = 56/189 * 87 = 25.8
In this case, teams with higher scores get rewarded with more credit for points in rounds when they played with normally underperforming robots. Also, the final sum of all teams represents the actual (normalized per regional) number of points scored at regionals, which more directly answers OP's question
depth_Finder
20-03-2015, 19:35
I would absolutely love to see this (and I have a hunch that you're right), but this requires data beyond what's available through FIRST. We'd need to find at least one team per event (preferably more) that has team-level data for every match, then go from there.
Actually, now that I think about it, maybe the good people behind frcscout.com (http://frcscout.com) could make this happen - they've got a lot of data in a standardized format.
Did you call?
It was a bit harder than I thought it would be to make this visualization because I wanted to make it automatic and customizable.
Here is an interactive visualization (drag the slider to see the contributions from top nth teams)
https://public.tableau.com/profile/enzoman34#!/vizhome/TopnTeamContributions/TopTeamContributions
And here is a picture for those who have slower internet connections or just want to see a pretty graph.
http://imgur.com/gallery/GcfEz80/
Note: I filtered out any event that had less than 30 matches scouted in it. I could put them back in, but I trust the data for larger events more.
This was actually super fun to make. PLEASE tell your friends to use this app. If we can get more regionals in the database, frcscout.com could be a census of FRC. If anyone else is as big of a data nerd as I am, that would be a VERY exciting new opportunity for some awesome stats.
Could try this if you can point me to the raw data (I only found the Team 2834 OPR generation scouting database):
Calculate an "effective QA" for each team by:
- For each match, sum up the final QA result of all teams in the alliance
- For each team in the alliance, their personal contribution is estimated as a percentage of their alliance's score proportional to the sum of the alliance team's original QA
- Calculate effective individual QA by averaging all matches in their competition (to normalize and account for different # of matches played at different events)
For example:
Team 1 QA = 95
Team 2 QA = 38
Team 3 QA = 56
Sum is 189
Match 1 Score = 87
Match 1, Team 1 "effective individual QA" = 95/189 * 87 = 43.7
Match 1, Team 2 "effective individual QA" = 38/189 * 87 = 17.5
Match 1, Team 3 "effective individual QA" = 56/189 * 87 = 25.8
In this case, teams with higher scores get rewarded with more credit for points in rounds when they played with normally underperforming robots. Also, the final sum of all teams represents the actual (normalized per regional) number of points scored at regionals, which more directly answers OP's question
Your "effective QA" is essentially a simplified Version of OPR.
Has anyone looked at what % of teams score 80% of the points? Is there a way to easily looks at how many total points team XXXX scored at an event compared to the total points scored at that event?
My team has an online OPR calculator (http://frc-team-955.github.io/955-OPR-Calculator/) that does something very similar to this already. We show what percentage a team contributed to their qual average (an interesting thing we noticed is at most regionals 2/3rds of teams contribute <33% to their totals). Should be easy to add exactly what you're asking for.
The Lucas
20-03-2015, 22:42
Auto points are probably even more concentrated at the top than total points.
After SCH District I took a quick look at the 144 qual auto points (sum of Ranking page auto points / 3) scored there. If you take out matches involving 3 robots, 225 (stacker scored the majority of the points), 486 (consistent tote & can shove), and 365 (occasionally got 2 step cans in the auto zone), there are only 28 points left. That's the top ~9% involved in ~80% of auto points. Of course that is just one small event.
We show what percentage a team contributed to their qual average (an interesting thing we noticed is at most regionals 2/3rds of teams contribute <33% to their totals).
The Qual Average of a team is the average of the alliance scores of the alliances that team played with. So it's nominally 3 times the team's OPR.
Here's the MITVC event:
Team OPR Avg/3 OPR-Avg/3
1 245 46.373 27.778 18.595
2 3767 35.807 22.778 13.029
3 51 36.071 23.250 12.821
4 862 35.921 23.361 12.560
5 5534 30.451 21.028 9.423
6 5562 30.224 21.028 9.197
7 4391 27.632 20.361 7.271
8 904 24.555 17.750 6.805
9 5213 26.819 20.028 6.792
10 3688 25.013 19.222 5.790
11 1711 27.487 21.806 5.681
12 5505 24.823 19.750 5.073
13 4398 26.865 21.833 5.032
14 1596 23.370 20.056 3.315
15 5110 17.161 14.417 2.744
16 4983 18.566 17.833 0.733
17 3618 18.898 18.333 0.564
18 94 17.812 17.250 0.562
19 5230 16.063 15.750 0.313
20 3886 15.192 15.528 -0.335
21 5223 13.503 16.056 -2.553
22 5560 12.731 15.361 -2.630
23 2474 14.244 16.889 -2.645
24 2246 11.941 15.000 -3.059
25 5575 12.085 15.306 -3.221
26 5086 12.599 16.472 -3.873
27 4392 12.488 16.417 -3.929
28 3537 10.193 14.278 -4.085
29 4988 12.881 17.000 -4.119
30 5314 9.978 15.000 -5.022
31 5692 12.095 17.333 -5.238
32 1896 8.540 14.722 -6.183
33 3175 7.210 14.028 -6.817
34 5247 4.846 12.111 -7.265
35 5183 4.787 13.611 -8.824
36 3603 1.507 11.139 -9.632
37 5709 4.220 14.056 -9.836
38 4376 2.471 13.917 -11.446
39 5072 4.227 15.722 -11.495
40 5175 -1.733 12.361 -14.094
Notice that about half the teams have an OPR greater than 1/3 of the sum of their alliance final scores, and half less than.
Then the game has 6~7 different things you can build for from Auto to Teleop (very singular specialty, to very overall alone high scorer). Base that on differences between Q Matches, and Playoffs (tossing co-op, add round robin, toss out the win-loss-draw, switch to QPA), then figure other itterations for champs...oy vey.
Yes it would be nice to know the true points scored for all teams over each & all events as singular robots...OPR is as close as you'll get.
But, what you can possibly do, isn't necessarily what you will do...Whatever works for you personally as a team, to get the points up in Q matches, then what you can and will actually do for your Alliance Partners in the Playoffs rising to the occasion when 3 all can actually work together smoothly! (And stay the heck away from those already hard built stacks). LOL
Much worse when you knock your own down too. That has to hurt.
Thank you so much for all your efforts to get a solution to this question.
So I did a little number crunching myself... as best I could with available data (courtesy of Team 995).
I did the following:
QA*(#matches)*OPR/100 to get an idea of total points scored. I then summed up all 66 teams to get a total for the regional.
Can't really get the data to cut/paste properly, but I got the TOP 8, the initial alliance captains, were responsible for 51% of total points scored at the regional.
Pretty interesting. And Los Angeles wasn't a crazy scoring regional. Might run same numbers for Waterloo!
QA*(#matches)*OPR/100 to get an idea of total points scored.
Would you please explain the above calculation? Perhaps by giving a numerical example for one team.
Sure. I changed the calculation a bit. I used Team 955's %Contribution value. That changed the Top 8 score percentage to 36.51%
Rank: 1
Team #: 330
Qual Avg: 93.55
Contribution %: 68.83
ADJ OPR: 63.51
QA*9: 841.95
% contr.: 579.51
% total: 6.7
Scr Top 8: 36.51
I multiplied team's QA by the number of matches (9), and that is QA*9.
I then multiplied that by their "Contribution %"/100, and that is "% contr", the number of total alliance points their scored.
I then totaled up all the "% contr", and divided each team's "% contr" by the total.
That gave me "% total Scr", the percent of the regional points scored.
I changed the calculation a bit... That changed the Top 8 score percentage to 36.51%
That sounds better.
rich2202
22-03-2015, 06:49
In watching the Wisconsin Regional, I think something close to the economy is similar, but not as you posed (80% outscored the rest combined).
I would guess that 10% of the robots could outscore the bottom 30% combined. But, this is not that different from prior years.
What is different is how much ahead the top 10% is from the next 10%. One top 10% bot can beat an entire alliance from the next 10%.
OPR seems this year to be statistically "input flawed" as a reliable scouting metric (Was much better previous games) . Too many every match/event variables at play for a single equation to define accurately individual offensive ranking..as in past years. Where individual bots were tracked more accurately in past games.
There are many bots with High QA 50+ that score <6 solo every match...by pure chance of other two partners being stronger masking their deficiency. Static scouting is only way to see this in action...this year.
When QA is a major variable you need many more data points than 10-20 to infer a reliable OPR in a game like this where only average is a major input variable (as well as tote, noodle, RC all averages of avg alliance)...to easy to skew QA (and other inputs) making using it troubling from a statistical perspective.
You simply need more "based on random alliance averages" data points for OPR to be more accurate at prediction this year. 100-300 matches would be better, in a game like this. Which is impossible even if all teams went to all matches within 1000 miles.
My advice this year as a scout..."eyes on bots." Take any OPR with a grain of salt.
We have all but only 10 bots personally scouted on their play and tendencies in RR for Ventura this weekend and the same in SD after. After all its really solo contribution added to your alliance score...what they do is what they do. They are mostly very predictable. Because many were very specifically designed to do their task repetitively. Not a lot of versatile bots out there. They are either good or bad predictably at the task they do.
There is a limited set off bots each team competes against in events (30-60):
Watch 2-3 matches on each that you face...compare to posted results. Easy to do...over a few weeks unless you play early.
In Worlds perhaps the fact you cannot possibly know out of 3000 who you will team up with and face ..OPR becomes more valuable. But again there are only 75 in a division and could possibly be done with archived video...once you find out who is in your division.
OPR seems flawed as a metric this year.
OPR is flawed every year :-)
Too many every variables at play for a single equation to define offensive ranking
I think you meant "value" not "equation". OPR computation involves scores (for a single event) or thousands (for all events combined) of equations.
Static scouting is only way.
If by "static" you meant manual scouting (i.e. using humans), it has always been the case that such scouting is superior to what can be teased from the data that FIRST provides.
But yes, arguably more so this year.
...But yes, arguably more so this year.This is an interesting argument. I know we don't have the data to address it directly, but is there are way to examine it by proxy, at least ordinally? For instance, we don't have enough live scouting data, but we do have draft order. If we posit that teams draft based on the real scouting data that OPR attempts to replicate*, are these data available in a form that allows for easy comparison? For instance, I just compared the top 15 OPRs to their draft order at 3 random 2015 events. ("Random" is used here non-technically to mean "the first three I clicked on in TBA".) I found that the average absolute value differences were 2.3, 1.2, and 1.3. The medians were even lower. This seems pretty good to me, but I haven't taken the time to do it more comprehensively or with other years.
Of course, this also only works for the top 24 teams at an event. On the other hand, that's the main reason most teams scout in the first place.
*This is an assumption whose falsity varies year-over-year. And also between events and teams, but I'll assume those variations have negligible effects on the YoY rankings for now.
This is an interesting argument. I know we don't have the data to address it directly, but is there are way to examine it by proxy, at least ordinally? For instance, we don't have enough live scouting data, but we do have draft order. If we posit that teams draft based on the real scouting data that OPR attempts to replicate*, are these data available in a form that allows for easy comparison? For instance, I just compared the top 15 OPRs to their draft order at 3 random 2015 events. ("Random" is used here non-technically to mean "the first three I clicked on in TBA".) I found that the average absolute value differences were 2.3, 1.2, and 1.3. The medians were even lower. This seems pretty good to me, but I haven't taken the time to do it more comprehensively or with other years.
Of course, this also only works for the top 24 teams at an event. On the other hand, that's the main reason most teams scout in the first place.
*This is an assumption whose falsity varies year-over-year. And also between events and teams, but I'll assume those variations have negligible effects on the YoY rankings for now.
Draft order is flawed too (this year) ..with lack of accurate OPR auto scouting Alliance captains (who rely on OPR) pick VERY flawed bots (usually along with name or uniforms ) and leave good ones un-picked and undrafted..again not a reliable input variable.
There are many past POWERHOUSE teams struggling badly in RR..My favorite local team who was amazing from last year is very limited this year...and rookies have made the finals in eliminations this year..its a leveling game because it requires...engineering...and live scouting more than in past years.
Draft order is flawed too (this year) ..with lack of accurate OPR auto scouting Alliance captains (who rely on OPR) pick VERY flawed bots (usually along with name or uniforms ) and leave good ones un-picked and undrafted..again not a reliable input variable.
There are many past POWERHOUSE teams struggling badly in RR..My favorite local team who was amazing from last year is very limited this year...and rookies have made the finals this year..its a leveling game because it requires...engineering...and live scouting.Draft order, like OPR, is flawed every year. And live scouting, like all human endeavors, is flawed ever year. Your stated reasons and observations concerning the uniqueness of this game also apply to every year. While the likelihood of captains picking on OPR has probably risen with its popularity, I see no reason to expect that it has skyrocketed this year.
Draft order, like OPR, is flawed every year. And live scouting, like all human endeavors, is flawed ever year. Your stated reasons and observations concerning the uniqueness of this game also apply to every year. While the likelihood of captains picking on OPR has probably risen with its popularity, I see no reason to expect that it has skyrocketed this year.
Fair point , I do think past performance in previous years is not as reliable as an indicator this year as this game is very leveling...and probably a more likely culprit than OPR for missing good bots...this year.
There are always at least one or two bots that should have been picked that I was watching (a subset of teams playing that we will face) and scratch my head at picks I had at or near bottom..on name only or uniform...and that alliance lost big surprise.
Look at ..the NFL draft predictions are highly flawed and has many more data points and much better statistical programs... nothing beats finding the diamond in the rough by witnessing performance first hand...especially this year's game.
I have to disagree that live bot scouting is flawed..this game play is very predictable...without defense being played apart from noodles.
In watching the Wisconsin Regional, I think something close to the economy is similar, but not as you posed (80% outscored the rest combined).
I would guess that 10% of the robots could outscore the bottom 30% combined. But, this is not that different from prior years.
What is different is how much ahead the top 10% is from the next 10%. One top 10% bot can beat an entire alliance from the next 10%.
I did an average of the teams at Los Angeles by groups of 6 (11 groups of 6). The top 6 had an average QA roughly 50% higher than the second tier of 6, and 100% higher than the third tier.
I did an average of the teams at Los Angeles by groups of 6 (11 groups of 6). The top 6 had an average QA roughly 50% higher than the second tier of 6, and 100% higher than the third tier.
That's because there are usually < 10 good-great bots at every regional. More like the 6 that you chose to group in tiers. Then 6-9 OK or near OK, then majority of the rest not-ok...in this years game due to a complete lack of contribution on defense...you simply have to score to be effective.
LA has three of the top 100 world bots... that was a tough regional. Top in SoCal
Caleb Sykes
22-03-2015, 12:59
There are many past POWERHOUSE teams struggling badly in RR.
If a supposed "powerhouse" team is struggling with RR, then they do not meet my definition of a powerhouse team.
In my mind, a powerhouse team is a team that consistently goes far into eliminations at every competition they attend every year. I don't believe that the game matters to true powerhouse teams.
If a supposed "powerhouse" team is struggling with RR, then they do not meet my definition of a powerhouse team.
In my mind, a powerhouse team is a team that consistently goes far into eliminations at every competition they attend every year. I don't believe that the game matters to true powerhouse teams.
Yup.. this year separates the chaff. Better than most.
I did an average of the teams at Los Angeles by groups of 6 (11 groups of 6). The top 6 had an average QA roughly 50% higher than the second tier of 6, and 100% higher than the third tier.
I did the same thing, using the official Qual Match Results from FRC.
For each team, I totaled their alliances' Final scores, divided that total by 3, then divided by the number of matches.
Then I sorted in descending order and did sums for groups of six as you did.
The results are quite a bit different from what you reported. Attached are 3 different views of the same data.
vBulletin® v3.6.4, Copyright ©2000-2017, Jelsoft Enterprises Ltd.