Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   Regional Competitions (http://www.chiefdelphi.com/forums/forumdisplay.php?f=10)
-   -   2014 District Point Model Analysis (http://www.chiefdelphi.com/forums/showthread.php?t=126977)

Navid Shafa 21-02-2014 08:05
2014 District Point Model Analysis
 
Main Reference: 2014 Projected District Point System

Below are some VERY interesting graphs and stats from the 2/6/14 PNW District Webinar (Video):




^This graph was the most interesting. I'm currently working on a further analysis including the other newer district regions: MAR, NE and our friends in Canada :)

The Pacific NW webinar and these graphs inspired me to take my old model and apply the new point system and include average OPR:

PNW 2013
Code:
Rank        Team        Points        OPR        OPR Rank
1        1983        130        49.97        4
2        2471        130        52.50        3
3        948        116        65.44        1
4        360        96        31.79        17
5        1425        92        49.91        6
6        1899        89        49.94        5
7        3663        87        46.89        7
8        2046        84        15.58        63
9        1318        76        38.93        12
10        955        74        46.76        8
11        1540        73        26.77        28
12        2907        73        46.26        9
13        1595        70        32.37        16
14        2557        68        26.03        30
15        4089        68        7.83        97
16        2928        66        32.51        15
17        2733        60        21.57        44
18        4061        60        31.35        18
19        4579        60        42.94        10
20        3789        58        22.85        41
21        2930        56        12.28        79
22        3711        56        19.78        51
23        3812        56        53.77        2
24        4125        53        15.34        67
25        4205        52        24.97        34
26        753        48        3.78        117
27        2374        47        29.40        23
28        3219        45        14.42        73
29        4488        45        25.75        31
30        2605        44        28.22        25
31        4512        44        29.02        24
32        488        43        20.15        48
33        2147        43        23.46        40
34        4127        43        36.16        13
35        2635        42        15.40        65
36        492        41        23.48        39
37        3968        41        20.28        47
38        997        40        30.99        20
39        3131        40        12.54        77
40        2811        39        19.61        52
41        2990        37        29.84        22
42        3826        37        30.41        21
43        2412        36        19.98        49
44        4086        36        18.05        54
45        2522        35        26.22        29
46        4681        34        21.65        43
47        3574        33        17.00        57
48        2922        32        23.98        36
49        4772        31        3.46        118
50        2517        30        7.52        98
51        4030        29        23.72        37
52        1258        28        12.34        78
53        3238        28        34.92        14
54        2910        27        15.88        60
55        3681        27        19.07        53
56        847        26        26.91        27
57        4057        26        15.87        61
58        4077        26        15.92        59
59        1294        24        31.25        19
60        3693        23        17.99        56
61        4662        23        3.01        121
62        956        22        23.51        38
63        1359        22        25.26        32
64        2923        22        5.01        108
65        2929        22        22.79        42
66        4060        22        -1.95        140
67        4461        22        4.07        115
68        4495        22        9.85        89
69        4513        22        16.23        58
70        4652        22        15.33        68
71        4692        22        2.60        123
72        4542        21        8.26        94
73        4682        21        6.56        101
74        2944        20        2.73        122
75        2976        19        10.63        85
76        4726        19        2.58        124
77        2411        18        5.17        107
78        2915        18        5.37        106
79        4450        18        41.36        11
80        4560        18        13.13        76
81        4654        18        14.33        74
82        4683        18        15.35        66
83        4038        17        25.04        33
84        4120        17        13.28        75
85        4469        17        19.93        50
86        4457        16        -7.65        154
87        4548        16        3.09        120
88        4559        16        11.95        80
89        4608        16        -9.09        156
90        1510        15        0.96        131
91        2926        15        5.51        104
92        3995        15        -0.86        138
93        4043        15        10.26        87
94        4082        15        -0.56        137
95        4309        15        9.31        91
96        3221        14        8.15        95
97        3860        14        -0.42        136
98        4132        13        -4.35        150
99        4173        13        -2.55        143
100        4180        13        11.43        82
101        2093        12        10.65        84
102        3220        12        9.98        88
103        3587        12        -0.26        135
104        3588        12        21.24        45
105        3787        12        0.67        133
106        3674        11        1.37        129
107        4110        11        4.27        112
108        4304        11        0.84        132
109        957        10        -3.94        148
110        1432        10        9.28        92
111        1778        10        5.77        103
112        2002        10        5.87        102
113        2555        10        3.98        116
114        3268        10        15.11        70
115        3393        10        26.93        26
116        3684        10        20.36        46
117        3712        10        5.50        105
118        3781        10        14.51        72
119        3801        10        7.44        99
120        2521        9        4.26        113
121        2542        9        15.10        71
122        2660        9        8.08        96
123        4051        9        15.52        64
124        4105        9        -2.56        144
125        4131        9        0.97        130
126        949        8        4.69        111
127        2148        8        4.08        114
128        2149        8        24.46        35
129        2192        8        4.83        109
130        2550        8        15.71        62
131        2903        8        1.40        128
132        2906        8        4.72        110
133        2927        8        18.04        55
134        3223        8        10.41        86
135        3286        8        -5.54        151
136        3586        8        11.19        83
137        3813        8        -0.99        139
138        3576        7        9.71        90
139        3786        7        15.29        69
140        1571        6        3.19        119
141        2898        6        6.57        100
142        3024        6        -2.98        146
143        3070        6        -3.95        149
144        3192        6        11.83        81
145        3636        6        -2.47        142
146        3673        6        -2.84        145
147        3876        6        2.18        126
148        2942        4        1.95        127
149        2980        4        -3.05        147
150        3049        4        -2.40        141
151        3662        4        8.74        93
152        1823        3        -6.93        153
153        3575        3        -8.42        155
154        3218        2        -0.05        134
155        3831        2        2.38        125
156        3237        0        -5.84        152
*Teams underlined were Chairman Winners.


I graphed Teams and sorted by Rank, showing Points vs. OPR:


*I realize there is a lot of statistical noise. I also didn’t normalize the events.

With all of this in mind:

64 teams would qualify for the District Championship. 10 district Chairman winners, 54 taken from the point system ranking.

Points accrued at the District Championship are worth (3x) as much.
PNW would then send the following (24) teams to the World Championship:
-3 Winning Alliance Members
-3 Chairman’s Winners
-1 Engineering Inspiration
-1 Rookie All Star
-16 Teams based off of the District Point Ranking.

I love the fact that last year, our 156 teams made up slightly more than 6% of the 2509 world team count. This means that we would have filled 6% of the 400 Champ slots.

I really hope that when we are in an entirely unified District System that Regions get a percentage of slots based upon their relative size.

Along with the further analysis of regional representation, I am going to create a 2014 PNW District Point Database here.

I'm super excited for the new District System to play out in the PNW. Good luck teams!

Lil' Lavery 21-02-2014 09:01
Re: 2014 District Point Model Analysis
 
Where was the raw data for the first graph obtained? Is there somewhere where it's already compiled, or was it mined by an individual? How far back does it extend?

Navid Shafa 21-02-2014 09:03
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by Lil' Lavery (Post 1347411)
Where was the raw data for the first graph obtained? Is there somewhere where it's already compiled, or was it mined by an individual? How far back does it extend?
I would like to know myself, I just shot Kevin a message on Facebook. If I can't obtain it, I plan on mining what I need myself to expand on graph 3.

Andrew Schreiber 21-02-2014 09:22
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by Lil' Lavery (Post 1347411)
Where was the raw data for the first graph obtained? Is there somewhere where it's already compiled, or was it mined by an individual? How far back does it extend?
My theory would be Zondag's spreadsheet would have most of this data.

Mr V 21-02-2014 11:51
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by Navid Shafa (Post 1347391)

64 teams would qualify for the District Championship. 10 district Chairman winners, 54 taken from the point system ranking.

Points accrued at the District Championship are worth (3x) as much.
PNW would then send the following (24) teams to the World Championship:
-3 Winning Alliance Members
-3 Chairman’s Winners
-1 Engineering Inspiration
-1 Rookie All Star
-16 Teams based off of the District Point Ranking.

I love the fact that last year, our 156 teams made up slightly more than 6% of the 2509 world team count. This means that we would have filled 6% of the 400 Champ slots.

Being on the District Championship winning alliance does not earn you an automatic spot at World Championship. It is highly likely that you will earn a spot with the points that you earn from being on the winning alliance, but it could be possible that the 2nd pick doesn't make the cut. This is done to give those teams from the Valleys of Doom a better chance of moving on and tries to minimize the chances of that 16th pick getting a ride on the coat tails of the #1 seed.

Mr V 21-02-2014 11:55
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by Andrew Schreiber (Post 1347423)
My theory would be Zondag's spreadsheet would have most of this data.
Yes the data for the first graph is from data presented by FiM.

Racer26 21-02-2014 11:57
Re: 2014 District Point Model Analysis
 
Its interesting, but these graphs don't really show anything that anyone couldn't automatically tell you. #1 and #2 alliances are the strongest, and overreward 15th and 16th picks compared to their OPR.

The district model does a better job of ensuring the best robots go to CMP, and consequently, Michigan representation in CMP elims is more indicative of their size in the league.

Steven Donow 21-02-2014 12:00
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by Mr V (Post 1347518)
Being on the District Championship winning alliance does not earn you an automatic spot at World Championship. It is highly likely that you will earn a spot with the points that you earn from being on the winning alliance, but it could be possible that the 2nd pick doesn't make the cut. This is done to give those teams from the Valleys of Doom a better chance of moving on and tries to minimize the chances of that 16th pick getting a ride on the coat tails of the #1 seed.
This is incorrect. Winning District Championship does give you a bid to Champs. Those teams are then 'eliminated' in regards to looking at standings for the points-based slots.

Nathan Streeter 21-02-2014 13:00
Re: 2014 District Point Model Analysis
 
Very cool graphs! Comparing the first two graphs is definitely very cool... it really does show how much of an improvement the district system should make in sending good robots onward!

Thad House 21-02-2014 13:50
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by DevenStonow (Post 1347525)
This is incorrect. Winning District Championship does give you a bid to Champs. Those teams are then 'eliminated' in regards to looking at standings for the points-based slots.
Actually they told us in the last district meeting that winning the Championship does NOT get you an automatic bid to worlds. I don't really understand it, but if that's what they decided then that's what they decided.

Andrew Schreiber 21-02-2014 13:58
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by sst.thad (Post 1347601)
Actually they told us in the last district meeting that winning the Championship does NOT get you an automatic bid to worlds. I don't really understand it, but if that's what they decided then that's what they decided.
It's not automatic but the points involved typical result in it? /speculation

alectronic 21-02-2014 14:06
Re: 2014 District Point Model Analysis
 
That's correct. There is no automatic bid- but if you think about it, a winning team would be (assuming no backup bots) 30 points for winning multiplied by 3 for being a district championship. That's 90 points, which should be plenty to put that team in a good position to advance regardless.

Nathan Streeter 21-02-2014 14:07
Re: 2014 District Point Model Analysis
 
At least for New England, the winners at DCMP will move on to CMP automatically, as mentioned at the bottom of this link... http://www.nefirst.org/2014/02/04/ne...structure-faq/

I'm not sure if this is a global FIRST decision or an NEFIRST decision... I don't think it is mentioned in either this blog post (http://www.usfirst.org/roboticsprogr...Kickoff-Taping) or the official points supplement (http://www3.usfirst.org/sites/defaul...ng_System.pdf).

Thad House 21-02-2014 14:08
Re: 2014 District Point Model Analysis
 
For the lower seeds (5-8), it helps all 3 qualify. If you are the 3rd pick on a 1-4 alliance, that might not get enough points for you to make it, because if you face a 5-8 seed in the finals, all 3 of those bots will have more points then a 13-16th picked bot.

I will say this most likely won't happen, but if a team this year wins the championship but does not make it to worlds on points, I think there will be a huge outrage, and I will fight to make it so the winners make it to worlds, because thats how it should be no matter what.

Jessica Boucher 21-02-2014 21:38
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by Nathan Streeter (Post 1347612)
At least for New England, the winners at DCMP will move on to CMP automatically, as mentioned at the bottom of this link... http://www.nefirst.org/2014/02/04/ne...structure-faq/

I'm not sure if this is a global FIRST decision or an NEFIRST decision... I don't think it is mentioned in either this blog post (http://www.usfirst.org/roboticsprogr...Kickoff-Taping) or the official points supplement (http://www3.usfirst.org/sites/defaul...ng_System.pdf).
This is the way I understand it - but I'm checking in on this. Thanks for the patience!

MikeE 22-02-2014 17:55
Re: 2014 District Point Model Analysis
 
There are some other interesting observations from analysis of the 2013 district events using the Unified District Points Model.

Perhaps the most interesting is the basic histogram of points per event. The 17 district events held during the 2013 season had the following distribution:



As expected this shows a multimodal distribution which roughly corresponds to one component for teams who don't make the eliminations (tall bump to the left) another broader component for teams receiving seeding points then a long tail for teams reaching the last stages of elimination.
This year there are 39 districts, a large jump from last year's 17 events so there will be more data and we'll get a better model of the underlying distribution..

The second observation is the impact of district size. From a team perspective the majority of points are given for seeding/elimination performance. These are always split between 24 teams that make the eliminations so at smaller events where the likelihood of getting to elims is higher there is a correspondingly higher average point score per team.

In 2013 the largest district had 43 teams and the smallest 31 with an average point score of 24.4 and 29.2 respectively. This year the spread in size across all Districts is even greater with the largest event at 45 teams (Bridgewater-Raritan in MAR) and smallest event at 28 teams (Mt Vernon in PNW). The expected points per team just based on event size would be 23.8 and 31.0 respectively.

The following table shows the expected (average) points for hypothetical teams attending the two smallest or two largest events in each of the Districts:
Code:
Dist  Smallest    pts  Largest    pts  Diff
MAR    33, 34  55.8    40, 40  49.1  6.6
PNW    28, 29  61.3    35, 35  54.4  6.9
FiM    31, 35  56.4    40, 40  50.6  5.8
NE      32, 33  56.7    40, 40  50.6  6.1
So choosing smaller events gives a 6 to 7 point advantage or almost as much as the rookie bonus.

Finally the graph below shows the average total points scored split by team qualification score.



It's not a surprise to see the points increase as a team wins more matches, but above 8 wins the average points per event grows far more quickly. Seeding points make a small difference but this is mostly due to the same observation as above that 70% of all district events were won by the top alliance (and every district event was won by a top 3 alliance).

Navid Shafa 22-02-2014 18:17
Re: 2014 District Point Model Analysis
 
Nice job!

Quote:
Originally Posted by MikeE (Post 1348091)
So choosing smaller events gives a 6 to 7 point advantage or almost as much as the rookie bonus.
This is really interesting, and a much bigger point swing than I would have expected.

Jessica Boucher 24-02-2014 15:05
Re: 2014 District Point Model Analysis
 
Following up - this was a misunderstanding on my part. Teams who win the District Championship do not receive auto-bids to the FIRST Championship, but as you smart people already figured out, they're going to slide to the top of the invite list because of winning anyway.

Post is updated. Life is an iterative process.

Navid Shafa 25-02-2014 08:22
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by Jessica Boucher (Post 1349045)
Life is an iterative process.
+1

Hope this clears up any confusion I've created:

1. Rookie Advantage Points
  • Teams in their first season ("Rookies") will receive 10 points
  • Teams in their second season will receive 5 points
  • Rookie season used for calculations is based on FIRST records

2. Qualification Round Performance
  • Teams will receive 2 points for each match they win
  • Teams will receive 1 point for each match they tie
  • Teams do not receive points for surrogate rounds or rounds in which they are disqualified^

3. Alliance Selection Results
  • Alliance Captains (#1 to #8) will receive points equal to 17 minus their captain number
  • Drafted teams (Picks #1 to #16) will receive points equal to 17 minus their acceptance order number
  • Points are based on positions at the conclusion of alliance selection, not at the end of qualification rounds

4. Elimination Round Performance
  • Each member of the alliance that wins a given round (quarterfinals, semifinals or finals) will receive 5 points for each match in which their robot participated
  • Backup robots will receive points in place of the replaced robot starting at the point of the replacement (even if that falls mid-round)
  • Points are only awarded to an alliance that advances. An eliminated alliance receives no points in the round in which they were eliminated even if they won a match in that series

5. Awards Received
  • Teams will receive 5 points for each judged award won
  • Teams that win the Engineering Inspiration or Rookie All-Star awards will receive an additional 3 points
  • The team that wins the Chairman's award will receive an additional 5 points
  • Only judged awards receive points (performance based awards do not receive award points)
  • Only awards presented to a team accumulate points (i.e. Deans List or awards to a person do not earn district ranking points)

Not that it would affect us in the PNW, but I'm curious how NEF will handle Legacy teams. Legacy and HOF teams get a championship ticket regardless, but does that have an effect on how many teams get sent to Champs? I'm wondering as FIRST reaches critical mass if they'll lump these teams in with the total amount of championship slots a district will get...

Jessica Boucher 25-02-2014 10:06
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by Navid Shafa (Post 1349380)
+1
Not that it would affect us in the PNW, but I'm curious how NEF will handle Legacy teams. Legacy and HOF teams get a championship ticket regardless, but does that have an effect on how many teams get sent to Champs? I'm wondering as FIRST reaches critical mass if they'll lump these teams in with the total amount of championship slots a district will get...
This was obviously a big deal for us. Currently, our six Legacy and HOF teams (FRC 23, 126, 151, 175, 190, 236) do not affect our 24 slots under the Unified Point Model, nor does any auto-bid team in any other district. Consequently, this also means some of them are choosing to skip the District Championship to give other teams an opportunity to play on a bigger stage, which has its good and bad points, but is understandable.

MechEng83 25-02-2014 10:37
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by Navid Shafa (Post 1349380)
5. Awards Received
  • Teams will receive 5 points for each judged award won
  • Teams that win the Engineering Inspiration or Rookie All-Star awards will receive an additional 3 points
  • The team that wins the Chairman's award will receive an additional 5 points
  • Only judged awards receive points (performance based awards do not receive award points)
  • Only awards presented to a team accumulate points (i.e. Deans List or awards to a person do not earn district ranking points)
Just to clarify the 2nd and 3rd bullets, because I was confused when I first read this: Engineering Inspiration is worth a total of 8 points (5+3) and Chairman's Award is worth a total of 10 points (5+5). The official document from FIRST lists those as the total number of points for the award, rather than saying it's a judged award (5 pts) + the extra.

Navid Shafa 25-02-2014 10:54
Re: 2014 District Point Model Analysis
 
Quote:
Originally Posted by MechEng83 (Post 1349438)
Just to clarify the 2nd and 3rd bullets, because I was confused when I first read this: Engineering Inspiration is worth a total of 8 points (5+3) and Chairman's Award is worth a total of 10 points (5+5). The official document from FIRST lists those as the total number of points for the award, rather than saying it's a judged award (5 pts) + the extra.
Correct

Navid Shafa 28-02-2014 13:23
Re: 2014 District Point Model Analysis
 
The point system scoring I mentioned above was taken from the new PNW District Ranking Website:

http://district.firstwa.org/


All times are GMT -5. The time now is 07:31.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi