2014 District Point Model Analysis

[Navid Shafa]

Main Reference: 2014 Projected District Point System (http://www.usfirst.org/sites/default/files/uploadedFiles/Robotics_Programs/FRC/Resources/FRC_District_Standard_Points_Ranking_System.pdf)

Below are some VERY interesting graphs and stats from the 2/6/14 PNW District Webinar (http://www.firstwa.org/Portals/0/FRCFiles/FRC2014/FRC%20Season%20Briefing%202-6-14.pdf) (Video (http://www.youtube.com/watch?v=Hj8ZcTbP22A&feature=youtu.be&t=52m46s)):

http://i.imgur.com/Vrv4feq.png
http://i.imgur.com/Xyf1gkp.png
http://i.imgur.com/96OaUiE.png
^This graph was the most interesting. I'm currently working on a further analysis including the other newer district regions: MAR, NE and our friends in Canada :)

The Pacific NW webinar and these graphs inspired me to take my old model (http://www.chiefdelphi.com/forums/showthread.php?t=115855&highlight=northwest+rankings) and apply the new point system and include average OPR:

PNW 2013
Rank Team Points OPR OPR Rank
1 1983 130 49.97 4
2 2471 130 52.50 3
3 948 116 65.44 1
4 360 96 31.79 17
5 1425 92 49.91 6
6 1899 89 49.94 5
7 3663 87 46.89 7
8 2046 84 15.58 63
9 1318 76 38.93 12
10 955 74 46.76 8
11 1540 73 26.77 28
12 2907 73 46.26 9
13 1595 70 32.37 16
14 2557 68 26.03 30
15 4089 68 7.83 97
16 2928 66 32.51 15
17 2733 60 21.57 44
18 4061 60 31.35 18
19 4579 60 42.94 10
20 3789 58 22.85 41
21 2930 56 12.28 79
22 3711 56 19.78 51
23 3812 56 53.77 2
24 4125 53 15.34 67
25 4205 52 24.97 34
26 753 48 3.78 117
27 2374 47 29.40 23
28 3219 45 14.42 73
29 4488 45 25.75 31
30 2605 44 28.22 25
31 4512 44 29.02 24
32 488 43 20.15 48
33 2147 43 23.46 40
34 4127 43 36.16 13
35 2635 42 15.40 65
36 492 41 23.48 39
37 3968 41 20.28 47
38 997 40 30.99 20
39 3131 40 12.54 77
40 2811 39 19.61 52
41 2990 37 29.84 22
42 3826 37 30.41 21
43 2412 36 19.98 49
44 4086 36 18.05 54
45 2522 35 26.22 29
46 4681 34 21.65 43
47 3574 33 17.00 57
48 2922 32 23.98 36
49 4772 31 3.46 118
50 2517 30 7.52 98
51 4030 29 23.72 37
52 1258 28 12.34 78
53 3238 28 34.92 14
54 2910 27 15.88 60
55 3681 27 19.07 53
56 847 26 26.91 27
57 4057 26 15.87 61
58 4077 26 15.92 59
59 1294 24 31.25 19
60 3693 23 17.99 56
61 4662 23 3.01 121
62 956 22 23.51 38
63 1359 22 25.26 32
64 2923 22 5.01 108
65 2929 22 22.79 42
66 4060 22 -1.95 140
67 4461 22 4.07 115
68 4495 22 9.85 89
69 4513 22 16.23 58
70 4652 22 15.33 68
71 4692 22 2.60 123
72 4542 21 8.26 94
73 4682 21 6.56 101
74 2944 20 2.73 122
75 2976 19 10.63 85
76 4726 19 2.58 124
77 2411 18 5.17 107
78 2915 18 5.37 106
79 4450 18 41.36 11
80 4560 18 13.13 76
81 4654 18 14.33 74
82 4683 18 15.35 66
83 4038 17 25.04 33
84 4120 17 13.28 75
85 4469 17 19.93 50
86 4457 16 -7.65 154
87 4548 16 3.09 120
88 4559 16 11.95 80
89 4608 16 -9.09 156
90 1510 15 0.96 131
91 2926 15 5.51 104
92 3995 15 -0.86 138
93 4043 15 10.26 87
94 4082 15 -0.56 137
95 4309 15 9.31 91
96 3221 14 8.15 95
97 3860 14 -0.42 136
98 4132 13 -4.35 150
99 4173 13 -2.55 143
100 4180 13 11.43 82
101 2093 12 10.65 84
102 3220 12 9.98 88
103 3587 12 -0.26 135
104 3588 12 21.24 45
105 3787 12 0.67 133
106 3674 11 1.37 129
107 4110 11 4.27 112
108 4304 11 0.84 132
109 957 10 -3.94 148
110 1432 10 9.28 92
111 1778 10 5.77 103
112 2002 10 5.87 102
113 2555 10 3.98 116
114 3268 10 15.11 70
115 3393 10 26.93 26
116 3684 10 20.36 46
117 3712 10 5.50 105
118 3781 10 14.51 72
119 3801 10 7.44 99
120 2521 9 4.26 113
121 2542 9 15.10 71
122 2660 9 8.08 96
123 4051 9 15.52 64
124 4105 9 -2.56 144
125 4131 9 0.97 130
126 949 8 4.69 111
127 2148 8 4.08 114
128 2149 8 24.46 35
129 2192 8 4.83 109
130 2550 8 15.71 62
131 2903 8 1.40 128
132 2906 8 4.72 110
133 2927 8 18.04 55
134 3223 8 10.41 86
135 3286 8 -5.54 151
136 3586 8 11.19 83
137 3813 8 -0.99 139
138 3576 7 9.71 90
139 3786 7 15.29 69
140 1571 6 3.19 119
141 2898 6 6.57 100
142 3024 6 -2.98 146
143 3070 6 -3.95 149
144 3192 6 11.83 81
145 3636 6 -2.47 142
146 3673 6 -2.84 145
147 3876 6 2.18 126
148 2942 4 1.95 127
149 2980 4 -3.05 147
150 3049 4 -2.40 141
151 3662 4 8.74 93
152 1823 3 -6.93 153
153 3575 3 -8.42 155
154 3218 2 -0.05 134
155 3831 2 2.38 125
156 3237 0 -5.84 152

*Teams underlined were Chairman Winners.


I graphed Teams and sorted by Rank, showing Points vs. OPR:

http://i.imgur.com/6XgKztA.png
*I realize there is a lot of statistical noise. I also didn’t normalize the events.

With all of this in mind:

64 teams would qualify for the District Championship. 10 district Chairman winners, 54 taken from the point system ranking.

Points accrued at the District Championship are worth (3x) as much.
PNW would then send the following (24) teams to the World Championship:
-3 Winning Alliance Members
-3 Chairman’s Winners
-1 Engineering Inspiration
-1 Rookie All Star
-16 Teams based off of the District Point Ranking.

I love the fact that last year, our 156 teams made up slightly more than 6% of the 2509 world team count. This means that we would have filled 6% of the 400 Champ slots.

I really hope that when we are in an entirely unified District System that Regions get a percentage of slots based upon their relative size.

Along with the further analysis of regional representation, I am going to create a 2014 PNW District Point Database here (https://docs.google.com/spreadsheet/ccc?key=0Aub1rtWbtmvEdHdRMTB0Mnd6dW8tZTVwTUUtbU5aS Gc&usp=sharing).

I'm super excited for the new District System to play out in the PNW. Good luck teams!

[Lil' Lavery]

Where was the raw data for the first graph obtained? Is there somewhere where it's already compiled, or was it mined by an individual? How far back does it extend?

[Navid Shafa]

Where was the raw data for the first graph obtained? Is there somewhere where it's already compiled, or was it mined by an individual? How far back does it extend?

I would like to know myself, I just shot Kevin a message on Facebook. If I can't obtain it, I plan on mining what I need myself to expand on graph 3.

[Andrew Schreiber]

Where was the raw data for the first graph obtained? Is there somewhere where it's already compiled, or was it mined by an individual? How far back does it extend?

My theory would be Zondag's spreadsheet would have most of this data.

[Mr V]

64 teams would qualify for the District Championship. 10 district Chairman winners, 54 taken from the point system ranking.

Points accrued at the District Championship are worth (3x) as much.
PNW would then send the following (24) teams to the World Championship:
-3 Winning Alliance Members
-3 Chairman’s Winners
-1 Engineering Inspiration
-1 Rookie All Star
-16 Teams based off of the District Point Ranking.

I love the fact that last year, our 156 teams made up slightly more than 6% of the 2509 world team count. This means that we would have filled 6% of the 400 Champ slots.




Being on the District Championship winning alliance does not earn you an automatic spot at World Championship. It is highly likely that you will earn a spot with the points that you earn from being on the winning alliance, but it could be possible that the 2nd pick doesn't make the cut. This is done to give those teams from the Valleys of Doom a better chance of moving on and tries to minimize the chances of that 16th pick getting a ride on the coat tails of the #1 seed.

[Mr V]

My theory would be Zondag's spreadsheet would have most of this data.

Yes the data for the first graph is from data presented by FiM.

[Racer26]

Its interesting, but these graphs don't really show anything that anyone couldn't automatically tell you. #1 and #2 alliances are the strongest, and overreward 15th and 16th picks compared to their OPR.

The district model does a better job of ensuring the best robots go to CMP, and consequently, Michigan representation in CMP elims is more indicative of their size in the league.

[Steven Donow]

Being on the District Championship winning alliance does not earn you an automatic spot at World Championship. It is highly likely that you will earn a spot with the points that you earn from being on the winning alliance, but it could be possible that the 2nd pick doesn't make the cut. This is done to give those teams from the Valleys of Doom a better chance of moving on and tries to minimize the chances of that 16th pick getting a ride on the coat tails of the #1 seed.

This is incorrect. Winning District Championship does give you a bid to Champs. Those teams are then 'eliminated' in regards to looking at standings for the points-based slots.

[Nathan Streeter]

Very cool graphs! Comparing the first two graphs is definitely very cool... it really does show how much of an improvement the district system should make in sending good robots onward!

[Thad House]

This is incorrect. Winning District Championship does give you a bid to Champs. Those teams are then 'eliminated' in regards to looking at standings for the points-based slots.

Actually they told us in the last district meeting that winning the Championship does NOT get you an automatic bid to worlds. I don't really understand it, but if that's what they decided then that's what they decided.

[Andrew Schreiber]

Actually they told us in the last district meeting that winning the Championship does NOT get you an automatic bid to worlds. I don't really understand it, but if that's what they decided then that's what they decided.

It's not automatic but the points involved typical result in it? /speculation

[alectronic]

That's correct. There is no automatic bid- but if you think about it, a winning team would be (assuming no backup bots) 30 points for winning multiplied by 3 for being a district championship. That's 90 points, which should be plenty to put that team in a good position to advance regardless.

[Nathan Streeter]

At least for New England, the winners at DCMP will move on to CMP automatically, as mentioned at the bottom of this link... http://www.nefirst.org/2014/02/04/ne-first-point-structure-faq/

I'm not sure if this is a global FIRST decision or an NEFIRST decision... I don't think it is mentioned in either this blog post (http://www.usfirst.org/roboticsprograms/frc/blog-District-Award-Allocations-and-Kickoff-Taping) or the official points supplement (http://www3.usfirst.org/sites/default/files/uploadedFiles/Robotics_Programs/FRC/Resources/FRC_District_Standard_Points_Ranking_System.pdf).

[Thad House]

For the lower seeds (5-8), it helps all 3 qualify. If you are the 3rd pick on a 1-4 alliance, that might not get enough points for you to make it, because if you face a 5-8 seed in the finals, all 3 of those bots will have more points then a 13-16th picked bot.

I will say this most likely won't happen, but if a team this year wins the championship but does not make it to worlds on points, I think there will be a huge outrage, and I will fight to make it so the winners make it to worlds, because thats how it should be no matter what.

[Jessica Boucher]

At least for New England, the winners at DCMP will move on to CMP automatically, as mentioned at the bottom of this link... http://www.nefirst.org/2014/02/04/ne-first-point-structure-faq/

I'm not sure if this is a global FIRST decision or an NEFIRST decision... I don't think it is mentioned in either this blog post (http://www.usfirst.org/roboticsprograms/frc/blog-District-Award-Allocations-and-Kickoff-Taping) or the official points supplement (http://www3.usfirst.org/sites/default/files/uploadedFiles/Robotics_Programs/FRC/Resources/FRC_District_Standard_Points_Ranking_System.pdf).

This is the way I understand it - but I'm checking in on this. Thanks for the patience!

[MikeE]

There are some other interesting observations from analysis of the 2013 district events using the Unified District Points Model.

Perhaps the most interesting is the basic histogram of points per event. The 17 district events held during the 2013 season had the following distribution:

http://i.imgur.com/8fWoOva.png

As expected this shows a multimodal distribution which roughly corresponds to one component for teams who don't make the eliminations (tall bump to the left) another broader component for teams receiving seeding points then a long tail for teams reaching the last stages of elimination.
This year there are 39 districts, a large jump from last year's 17 events so there will be more data and we'll get a better model of the underlying distribution..

The second observation is the impact of district size. From a team perspective the majority of points are given for seeding/elimination performance. These are always split between 24 teams that make the eliminations so at smaller events where the likelihood of getting to elims is higher there is a correspondingly higher average point score per team.

In 2013 the largest district had 43 teams and the smallest 31 with an average point score of 24.4 and 29.2 respectively. This year the spread in size across all Districts is even greater with the largest event at 45 teams (Bridgewater-Raritan in MAR) and smallest event at 28 teams (Mt Vernon in PNW). The expected points per team just based on event size would be 23.8 and 31.0 respectively.

The following table shows the expected (average) points for hypothetical teams attending the two smallest or two largest events in each of the Districts:

Dist Smallest pts Largest pts Diff
MAR 33, 34 55.8 40, 40 49.1 6.6
PNW 28, 29 61.3 35, 35 54.4 6.9
FiM 31, 35 56.4 40, 40 50.6 5.8
NE 32, 33 56.7 40, 40 50.6 6.1

So choosing smaller events gives a 6 to 7 point advantage or almost as much as the rookie bonus.

Finally the graph below shows the average total points scored split by team qualification score.

http://i.imgur.com/KLWUDNf.png

It's not a surprise to see the points increase as a team wins more matches, but above 8 wins the average points per event grows far more quickly. Seeding points make a small difference but this is mostly due to the same observation as above that 70% of all district events were won by the top alliance (and every district event was won by a top 3 alliance).

[Navid Shafa]

Nice job!


So choosing smaller events gives a 6 to 7 point advantage or almost as much as the rookie bonus.

This is really interesting, and a much bigger point swing than I would have expected.

[Jessica Boucher]

Following up - this was a misunderstanding on my part. Teams who win the District Championship do not receive auto-bids to the FIRST Championship, but as you smart people already figured out, they're going to slide to the top of the invite list because of winning anyway.

Post is updated. Life is an iterative process.

[Navid Shafa]

Life is an iterative process.
+1

Hope this clears up any confusion I've created:

1. Rookie Advantage Points

Teams in their first season ("Rookies") will receive 10 points
Teams in their second season will receive 5 points
Rookie season used for calculations is based on FIRST records


2. Qualification Round Performance

Teams will receive 2 points for each match they win
Teams will receive 1 point for each match they tie
Teams do not receive points for surrogate rounds or rounds in which they are disqualified^


3. Alliance Selection Results

Alliance Captains (#1 to #8) will receive points equal to 17 minus their captain number
Drafted teams (Picks #1 to #16) will receive points equal to 17 minus their acceptance order number
Points are based on positions at the conclusion of alliance selection, not at the end of qualification rounds


4. Elimination Round Performance

Each member of the alliance that wins a given round (quarterfinals, semifinals or finals) will receive 5 points for each match in which their robot participated
Backup robots will receive points in place of the replaced robot starting at the point of the replacement (even if that falls mid-round)
Points are only awarded to an alliance that advances. An eliminated alliance receives no points in the round in which they were eliminated even if they won a match in that series


5. Awards Received

Teams will receive 5 points for each judged award won
Teams that win the Engineering Inspiration or Rookie All-Star awards will receive an additional 3 points
The team that wins the Chairman's award will receive an additional 5 points
Only judged awards receive points (performance based awards do not receive award points)
Only awards presented to a team accumulate points (i.e. Deans List or awards to a person do not earn district ranking points)


Not that it would affect us in the PNW, but I'm curious how NEF will handle Legacy teams. Legacy and HOF teams get a championship ticket regardless, but does that have an effect on how many teams get sent to Champs? I'm wondering as FIRST reaches critical mass if they'll lump these teams in with the total amount of championship slots a district will get...

[Jessica Boucher]

+1
Not that it would affect us in the PNW, but I'm curious how NEF will handle Legacy teams. Legacy and HOF teams get a championship ticket regardless, but does that have an effect on how many teams get sent to Champs? I'm wondering as FIRST reaches critical mass if they'll lump these teams in with the total amount of championship slots a district will get...

This was obviously a big deal for us. Currently, our six Legacy and HOF teams (FRC 23, 126, 151, 175, 190, 236) do not affect our 24 slots under the Unified Point Model, nor does any auto-bid team in any other district. Consequently, this also means some of them are choosing to skip the District Championship to give other teams an opportunity to play on a bigger stage, which has its good and bad points, but is understandable.

[MechEng83]

5. Awards Received

Teams will receive 5 points for each judged award won
Teams that win the Engineering Inspiration or Rookie All-Star awards will receive an additional 3 points
The team that wins the Chairman's award will receive an additional 5 points
Only judged awards receive points (performance based awards do not receive award points)
Only awards presented to a team accumulate points (i.e. Deans List or awards to a person do not earn district ranking points)


Just to clarify the 2nd and 3rd bullets, because I was confused when I first read this: Engineering Inspiration is worth a total of 8 points (5+3) and Chairman's Award is worth a total of 10 points (5+5). The official document (http://www.usfirst.org/sites/default/files/uploadedFiles/Robotics_Programs/FRC/Resources/FRC_District_Standard_Points_Ranking_System_r1.pdf ) from FIRST lists those as the total number of points for the award, rather than saying it's a judged award (5 pts) + the extra.

[Navid Shafa]

Just to clarify the 2nd and 3rd bullets, because I was confused when I first read this: Engineering Inspiration is worth a total of 8 points (5+3) and Chairman's Award is worth a total of 10 points (5+5). The official document (http://www.usfirst.org/sites/default/files/uploadedFiles/Robotics_Programs/FRC/Resources/FRC_District_Standard_Points_Ranking_System_r1.pdf ) from FIRST lists those as the total number of points for the award, rather than saying it's a judged award (5 pts) + the extra.

Correct

[Navid Shafa]

The point system scoring I mentioned above was taken from the new PNW District Ranking Website:

http://district.firstwa.org/