Teams scoring vast majority of points

[seg9585]

Originally Posted by Ether

This was discussed in earlier posts in this thread. Can you suggest something better, for which the data is available?

Could try this if you can point me to the raw data (I only found the Team 2834 OPR generation scouting database):

Calculate an "effective QA" for each team by:
- For each match, sum up the final QA result of all teams in the alliance
- For each team in the alliance, their personal contribution is estimated as a percentage of their alliance's score proportional to the sum of the alliance team's original QA
- Calculate effective individual QA by averaging all matches in their competition (to normalize and account for different # of matches played at different events)

For example:

Team 1 QA = 95
Team 2 QA = 38
Team 3 QA = 56

Sum is 189
Match 1 Score = 87

Match 1, Team 1 "effective individual QA" = 95/189 * 87 = 43.7
Match 1, Team 2 "effective individual QA" = 38/189 * 87 = 17.5
Match 1, Team 3 "effective individual QA" = 56/189 * 87 = 25.8

In this case, teams with higher scores get rewarded with more credit for points in rounds when they played with normally underperforming robots. Also, the final sum of all teams represents the actual (normalized per regional) number of points scored at regionals, which more directly answers OP's question

[depth_Finder]

Originally Posted by Jacob Bendicksen

I would absolutely love to see this (and I have a hunch that you're right), but this requires data beyond what's available through FIRST. We'd need to find at least one team per event (preferably more) that has team-level data for every match, then go from there.

Actually, now that I think about it, maybe the good people behind frcscout.com could make this happen - they've got a lot of data in a standardized format.

Did you call?

It was a bit harder than I thought it would be to make this visualization because I wanted to make it automatic and customizable.

Here is an interactive visualization (drag the slider to see the contributions from top nth teams)

https://public.tableau.com/profile/e...mContributions

And here is a picture for those who have slower internet connections or just want to see a pretty graph.

http://imgur.com/gallery/GcfEz80/

Note: I filtered out any event that had less than 30 matches scouted in it. I could put them back in, but I trust the data for larger events more.

This was actually super fun to make. PLEASE tell your friends to use this app. If we can get more regionals in the database, frcscout.com could be a census of FRC. If anyone else is as big of a data nerd as I am, that would be a VERY exciting new opportunity for some awesome stats.

[lemiant]

Originally Posted by seg9585

Could try this if you can point me to the raw data (I only found the Team 2834 OPR generation scouting database):

Calculate an "effective QA" for each team by:
- For each match, sum up the final QA result of all teams in the alliance
- For each team in the alliance, their personal contribution is estimated as a percentage of their alliance's score proportional to the sum of the alliance team's original QA
- Calculate effective individual QA by averaging all matches in their competition (to normalize and account for different # of matches played at different events)

For example:

Team 1 QA = 95
Team 2 QA = 38
Team 3 QA = 56

Sum is 189
Match 1 Score = 87

Match 1, Team 1 "effective individual QA" = 95/189 * 87 = 43.7
Match 1, Team 2 "effective individual QA" = 38/189 * 87 = 17.5
Match 1, Team 3 "effective individual QA" = 56/189 * 87 = 25.8

In this case, teams with higher scores get rewarded with more credit for points in rounds when they played with normally underperforming robots. Also, the final sum of all teams represents the actual (normalized per regional) number of points scored at regionals, which more directly answers OP's question

Your "effective QA" is essentially a simplified Version of OPR.

[Spoam]

Originally Posted by SenorZ

Has anyone looked at what % of teams score 80% of the points? Is there a way to easily looks at how many total points team XXXX scored at an event compared to the total points scored at that event?

My team has an online OPR calculator that does something very similar to this already. We show what percentage a team contributed to their qual average (an interesting thing we noticed is at most regionals 2/3rds of teams contribute <33% to their totals). Should be easy to add exactly what you're asking for.

[The Lucas]

Auto points are probably even more concentrated at the top than total points.

After SCH District I took a quick look at the 144 qual auto points (sum of Ranking page auto points / 3) scored there. If you take out matches involving 3 robots, 225 (stacker scored the majority of the points), 486 (consistent tote & can shove), and 365 (occasionally got 2 step cans in the auto zone), there are only 28 points left. That's the top ~9% involved in ~80% of auto points. Of course that is just one small event.

[Ether]

Originally Posted by Spoam

We show what percentage a team contributed to their qual average (an interesting thing we noticed is at most regionals 2/3rds of teams contribute <33% to their totals).

The Qual Average of a team is the average of the alliance scores of the alliances that team played with. So it's nominally 3 times the team's OPR.

Here's the MITVC event:

	Team	OPR	Avg/3	OPR-Avg/3
1	245	46.373	27.778	18.595
2	3767	35.807	22.778	13.029
3	51	36.071	23.250	12.821
4	862	35.921	23.361	12.560
5	5534	30.451	21.028	9.423
6	5562	30.224	21.028	9.197
7	4391	27.632	20.361	7.271
8	904	24.555	17.750	6.805
9	5213	26.819	20.028	6.792
10	3688	25.013	19.222	5.790
11	1711	27.487	21.806	5.681
12	5505	24.823	19.750	5.073
13	4398	26.865	21.833	5.032
14	1596	23.370	20.056	3.315
15	5110	17.161	14.417	2.744
16	4983	18.566	17.833	0.733
17	3618	18.898	18.333	0.564
18	94	17.812	17.250	0.562
19	5230	16.063	15.750	0.313
20	3886	15.192	15.528	-0.335
21	5223	13.503	16.056	-2.553
22	5560	12.731	15.361	-2.630
23	2474	14.244	16.889	-2.645
24	2246	11.941	15.000	-3.059
25	5575	12.085	15.306	-3.221
26	5086	12.599	16.472	-3.873
27	4392	12.488	16.417	-3.929
28	3537	10.193	14.278	-4.085
29	4988	12.881	17.000	-4.119
30	5314	9.978	15.000	-5.022
31	5692	12.095	17.333	-5.238
32	1896	8.540	14.722	-6.183
33	3175	7.210	14.028	-6.817
34	5247	4.846	12.111	-7.265
35	5183	4.787	13.611	-8.824
36	3603	1.507	11.139	-9.632
37	5709	4.220	14.056	-9.836
38	4376	2.471	13.917	-11.446
39	5072	4.227	15.722	-11.495
40	5175	-1.733	12.361	-14.094

Notice that about half the teams have an OPR greater than 1/3 of the sum of their alliance final scores, and half less than.

[cglrcng]

Then the game has 6~7 different things you can build for from Auto to Teleop (very singular specialty, to very overall alone high scorer). Base that on differences between Q Matches, and Playoffs (tossing co-op, add round robin, toss out the win-loss-draw, switch to QPA), then figure other itterations for champs...oy vey.

Yes it would be nice to know the true points scored for all teams over each & all events as singular robots...OPR is as close as you'll get.

But, what you can possibly do, isn't necessarily what you will do...Whatever works for you personally as a team, to get the points up in Q matches, then what you can and will actually do for your Alliance Partners in the Playoffs rising to the occasion when 3 all can actually work together smoothly! (And stay the heck away from those already hard built stacks). LOL

Much worse when you knock your own down too. That has to hurt.

[SenorZ]

Thank you so much for all your efforts to get a solution to this question.

So I did a little number crunching myself... as best I could with available data (courtesy of Team 995).
I did the following:

QA*(#matches)*OPR/100 to get an idea of total points scored. I then summed up all 66 teams to get a total for the regional.

Can't really get the data to cut/paste properly, but I got the TOP 8, the initial alliance captains, were responsible for 51% of total points scored at the regional.

Pretty interesting. And Los Angeles wasn't a crazy scoring regional. Might run same numbers for Waterloo!

[Ether]

Originally Posted by SenorZ

QA*(#matches)*OPR/100 to get an idea of total points scored.

Would you please explain the above calculation? Perhaps by giving a numerical example for one team.

[SenorZ]

Sure. I changed the calculation a bit. I used Team 955's %Contribution value. That changed the Top 8 score percentage to 36.51%

Rank: 1
Team #: 330
Qual Avg: 93.55
Contribution %: 68.83
ADJ OPR: 63.51
QA*9: 841.95
% contr.: 579.51
% total: 6.7
Scr Top 8: 36.51

I multiplied team's QA by the number of matches (9), and that is QA*9.
I then multiplied that by their "Contribution %"/100, and that is "% contr", the number of total alliance points their scored.
I then totaled up all the "% contr", and divided each team's "% contr" by the total.
That gave me "% total Scr", the percent of the regional points scored.

[Ether]

Originally Posted by SenorZ

I changed the calculation a bit... That changed the Top 8 score percentage to 36.51%

That sounds better.

[rich2202]

In watching the Wisconsin Regional, I think something close to the economy is similar, but not as you posed (80% outscored the rest combined).

I would guess that 10% of the robots could outscore the bottom 30% combined. But, this is not that different from prior years.

What is different is how much ahead the top 10% is from the next 10%. One top 10% bot can beat an entire alliance from the next 10%.

[Boltman]

OPR seems this year to be statistically "input flawed" as a reliable scouting metric (Was much better previous games) . Too many every match/event variables at play for a single equation to define accurately individual offensive ranking..as in past years. Where individual bots were tracked more accurately in past games.

There are many bots with High QA 50+ that score <6 solo every match...by pure chance of other two partners being stronger masking their deficiency. Static scouting is only way to see this in action...this year.

When QA is a major variable you need many more data points than 10-20 to infer a reliable OPR in a game like this where only average is a major input variable (as well as tote, noodle, RC all averages of avg alliance)...to easy to skew QA (and other inputs) making using it troubling from a statistical perspective.

You simply need more "based on random alliance averages" data points for OPR to be more accurate at prediction this year. 100-300 matches would be better, in a game like this. Which is impossible even if all teams went to all matches within 1000 miles.

My advice this year as a scout..."eyes on bots." Take any OPR with a grain of salt.

We have all but only 10 bots personally scouted on their play and tendencies in RR for Ventura this weekend and the same in SD after. After all its really solo contribution added to your alliance score...what they do is what they do. They are mostly very predictable. Because many were very specifically designed to do their task repetitively. Not a lot of versatile bots out there. They are either good or bad predictably at the task they do.

There is a limited set off bots each team competes against in events (30-60):

Watch 2-3 matches on each that you face...compare to posted results. Easy to do...over a few weeks unless you play early.

In Worlds perhaps the fact you cannot possibly know out of 3000 who you will team up with and face ..OPR becomes more valuable. But again there are only 75 in a division and could possibly be done with archived video...once you find out who is in your division.

[Ether]

Originally Posted by Boltman

OPR seems flawed as a metric this year.

OPR is flawed every year :-)

Too many every variables at play for a single equation to define offensive ranking

I think you meant "value" not "equation". OPR computation involves scores (for a single event) or thousands (for all events combined) of equations.

Static scouting is only way.

If by "static" you meant manual scouting (i.e. using humans), it has always been the case that such scouting is superior to what can be teased from the data that FIRST provides.

But yes, arguably more so this year.

[Siri]

Originally Posted by Ether

...But yes, arguably more so this year.

This is an interesting argument. I know we don't have the data to address it directly, but is there are way to examine it by proxy, at least ordinally? For instance, we don't have enough live scouting data, but we do have draft order. If we posit that teams draft based on the real scouting data that OPR attempts to replicate*, are these data available in a form that allows for easy comparison? For instance, I just compared the top 15 OPRs to their draft order at 3 random 2015 events. ("Random" is used here non-technically to mean "the first three I clicked on in TBA".) I found that the average absolute value differences were 2.3, 1.2, and 1.3. The medians were even lower. This seems pretty good to me, but I haven't taken the time to do it more comprehensively or with other years.

Of course, this also only works for the top 24 teams at an event. On the other hand, that's the main reason most teams scout in the first place.

*This is an assumption whose falsity varies year-over-year. And also between events and teams, but I'll assume those variations have negligible effects on the YoY rankings for now.