View Full Version : 2013 NE FIRST District Rankings
Special thanks to Brian Smist from 229 for doing half the work. As always, this is done by hand, please bring up any corrections.
Points:
This uses the most recent NE proposal from their site.
Qual Win: 2 pts
Qual Tie: 1 pt
QF: 5 pts
SF: 10 pts
F: 20 pts
W: 30 pts
Awards: 5 (Except RCA, EI, RAS)
Also, despite NE saying they will count the top two events, I did the first
two events for everyone.
Rankings with NY teams from Capital District should Conference Model be adopted:
1 20 108
2 125 89
3 2648 85
4 1519 75
5 3467 72
6 2791 69
7 126 67
8 2168 67
9 1100 66
10 3609 66
11 230 63
12 155 59
13 172 58
14 195 55
15 176 54
16 175 53
17 69 51
18 2067 51
19 228 50
20 1991 49
21 177 47
22 3958 46
23 558 46
24 3280 44
25 1153 44
26 3930 44
27 133 44
28 95 44
29 3044 43
30 4055 39
31 78 38
32 3525 38
33 58 36
34 2877 35
35 1071 35
36 4134 35
37 61 34
38 88 34
39 1073 33
40 839 32
41 157 32
42 1831 32
43 1517 32
44 885 31
45 571 29
46 1735 28
47 4473 28
48 4546 28
49 3205 26
50 236 26
51 509 25
52 181 25
53 3780 24
54 190 24
55 4564 24
56 1922 24
57 250 24
58 3182 24
59 138 23
60 4761 22
61 1058 22
62 121 21
63 1027 21
64 319 21
65 131 21
66 151 21
67 1512 21
68 1277 20
69 3566 20
70 173 20
71 3146 20
72 2370 20
73 3466 19
74 2423 18
75 1350 18
76 2349 18
77 4508 18
78 999 18
79 246 17
80 3236 17
81 1687 17
82 1784 17
83 4812 17
84 3499 17
85 3654 17
86 1493 17
87 1973 16
88 1699 16
89 2876 15
90 97 15
91 1761 15
92 1965 15
93 3585 15
94 4557 15
95 4097 14
96 4609 14
97 3464 14
98 237 14
99 2170 13
100 178 13
101 1768 12
102 2871 12
103 3479 12
104 238 12
105 3323 12
106 3104 12
107 716 12
108 663 12
109 3718 12
110 4176 11
111 2064 11
112 2836 11
113 23 10
114 2079 10
115 2084 10
116 3927 10
117 1289 10
118 4042 10
119 348 10
120 213 10
121 1665 10
122 3687 10
123 4311 9
124 4410 9
125 3597 9
126 4793 9
127 2342 9
128 1474 8
129 1757 8
130 2713 8
131 4041 8
132 1247 8
133 3634 8
134 501 8
135 2523 8
136 4048 7
137 1099 7
138 2262 6
139 4151 6
140 4474 6
141 2104 6
142 2621 6
143 467 6
144 3451 6
145 4555 6
146 166 6
147 1307 6
148 4034 6
149 4254 6
150 1740 6
151 3461 6
152 4628 6
153 1754 4
154 4796 4
155 529 4
156 811 4
157 1729 4
158 1124 4
159 2785 4
160 4572 4
161 1721 4
162 3555 3
163 3623 2
164 3719 0
And rankings without NY teams:
1 125 89
2 2648 85
3 1519 75
4 3467 72
5 126 67
6 2168 67
7 1100 66
8 3609 66
9 230 63
10 155 59
11 172 58
12 195 55
13 176 54
14 175 53
15 69 51
16 2067 51
17 228 50
18 1991 49
19 177 47
20 3958 46
21 558 46
22 3280 44
23 1153 44
24 3930 44
25 133 44
26 95 44
27 4055 39
28 78 38
29 3525 38
30 58 36
31 2877 35
32 1071 35
33 61 34
34 88 34
35 1073 33
36 839 32
37 157 32
38 1831 32
39 1517 32
40 885 31
41 571 29
42 1735 28
43 4473 28
44 4546 28
45 3205 26
46 236 26
47 181 25
48 509 25
49 3780 24
50 190 24
51 4564 24
52 3182 24
53 1922 24
54 138 23
55 4761 22
56 1058 22
57 121 21
58 1027 21
59 319 21
60 131 21
61 151 21
62 1512 21
63 1277 20
64 3566 20
65 173 20
66 3146 20
67 2370 20
68 3466 19
69 2423 18
70 1350 18
71 2349 18
72 999 18
73 246 17
74 3236 17
75 1687 17
76 1784 17
77 4812 17
78 3654 17
79 3499 17
80 1973 16
81 1699 16
82 2876 15
83 97 15
84 1761 15
85 1965 15
86 4557 15
87 3585 15
88 4097 14
89 4609 14
90 3464 14
91 237 14
92 2170 13
93 178 13
94 1768 12
95 2871 12
96 3479 12
97 3104 12
98 716 12
99 238 12
100 3323 12
101 663 12
102 3718 12
103 4176 11
104 2064 11
105 2836 11
106 23 10
107 2079 10
108 2084 10
109 3927 10
110 1289 10
111 4042 10
112 348 10
113 213 10
114 4311 9
115 4410 9
116 3597 9
117 4793 9
118 2342 9
119 1474 8
120 1757 8
121 2713 8
122 4041 8
123 3634 8
124 1247 8
125 501 8
126 2523 8
127 4048 7
128 1099 7
129 2262 6
130 4151 6
131 4474 6
132 2104 6
133 2621 6
134 467 6
135 3451 6
136 4555 6
137 1740 6
138 3461 6
139 4628 6
140 166 6
141 1307 6
142 4034 6
143 1754 4
144 4796 4
145 529 4
146 1124 4
147 2785 4
148 4572 4
149 811 4
150 1729 4
151 1721 4
152 3555 3
153 3623 2
154 3719 0
Jack (and Brian), nice work putting this together. I was wondering where we would stand in a district points structure.
Jessica Boucher
09-04-2013, 09:40
I was wondering where we would stand in a district points structure.
You know, there's supposed to be a post by one of the NE folks who created a district points simulator so that you could play with all the values ::coughKylecough::....just haven't seen it yet :)
Rosiebotboss
09-04-2013, 09:59
You know, there's supposed to be a post by one of the NE folks who created a district points simulator so that you could play with all the values ::coughKylecough::....just haven't seen it yet :)
I'll remind him....:cool:
Just so I can tell if I'm understanding this. The points were calculated as if the regionals in N.E. were district events. They would then determine invitation to the proposed district championship, viz. the top 80, right? NE District championship to be played "this" weekend from which 30 or so teams being invited to CMP in St. Louis. Have I got it right?
Now can we discuss why this points summary may, or may not, be slightly misleading? First, there is the number of events attended by many of the teams. That is, not enough opportunity for the one-event teams to get points for their Q-wins in this summary. A supposed advantage to the district model is that teams will get more matches played as a result. Perhaps the points for single regional event teams could be amplified a bit to reflect this. E.g., the scores from 9 Q-matches at CT regional might get a multiplier of 14/9ths to predict performance in two district events with 7 Q-matches each.
Second, the relative size of regionals to district events would change the dynamics of those competitions somewhat. Not sure about what the effect on points would be, but I'm sure it would be there.
Then there's the even more nebulous effect of the possibility of 8 hours "out of bag" preceding 2-day district events. Is this a factor? At present, teams attending 2-day events get to schedule a sort of "virtual Thursday" by logging time with their robot out of its bag before the competition. Will we allow this in the NE District?
My only questions is does this factor in teams that only attended one even? If so was anything done to factor the fact that they attended one event?
While obviously this is a very rough sketch and many of the metrics in Districts won't apply, I think it's a great way to see how things would play out.
So NE sends 30 teams, and according to the model, 11 teams will be from the champions, chairmans, rookie all-star, etc.
So would that mean the top 22 in this list would be guaranteed to qualify for championships, regardless of wins/awards?
What would be the disappearing regionals?
Brandon Holley
09-04-2013, 11:28
So NE sends 30 teams, and according to the model, 11 teams will be from the champions, chairmans, rookie all-star, etc.
So would that mean the top 22 in this list would be guaranteed to qualify for championships, regardless of wins/awards?
I believe points are still accrued through to the end of the DCMP. I think in MAR the points are tripled, so its still very much anyones ball game...
-Brando
My only questions is does this factor in teams that only attended one even? If so was anything done to factor the fact that they attended one event?
No, this is just raw data. If anyone wants to normalize for different factors PM me and I can send you the excel sheet.
Just so I can tell if I'm understanding this. The points were calculated as if the regionals in N.E. were district events. They would then determine invitation to the proposed district championship, viz. the top 80, right? NE District championship to be played "this" weekend from which 30 or so teams being invited to CMP in St. Louis. Have I got it right?
Now can we discuss why this points summary may, or may not, be slightly misleading? First, there is the number of events attended by many of the teams. That is, not enough opportunity for the one-event teams to get points for their Q-wins in this summary. A supposed advantage to the district model is that teams will get more matches played as a result. Perhaps the points for single regional event teams could be amplified a bit to reflect this. E.g., the scores from 9 Q-matches at CT regional might get a multiplier of 14/9ths to predict performance in two district events with 7 Q-matches each.
Second, the relative size of regionals to district events would change the dynamics of those competitions somewhat. Not sure about what the effect on points would be, but I'm sure it would be there.
Then there's the even more nebulous effect of the possibility of 8 hours "out of bag" preceding 2-day district events. Is this a factor? At present, teams attending 2-day events get to schedule a sort of "virtual Thursday" by logging time with their robot out of its bag before the competition. Will we allow this in the NE District?
Right - the results are informative but not a good predictive model of the district.
The rankings above from this year's Regionals primarily reflect whether a team attended 1 or 2+ Regionals, but under the district model every team would be guaranteed 2 events.
The other impacts come from having smaller events, which increase the expected points awarded in several ways:
increasing the number of qualification matches per team,
increasing the probability of being selected for eliminations (e.g. 24/34 rather than 24/65 for a big regional),
increasing the probability of winning an award, and
arguably diluting the strength of elimination alliances, hence increasing the variance of expected elimination points
Assuming district events average 34 teams each (155 teams attending 2 of 9 events), and a relatively gentle schedule of 12 qualification matches, then the expected point total under the proposed model would be 39.9 for competition performance only. Award points would be additional.
The average from the table above is only 23.2 including awards.
Rosiebotboss
09-04-2013, 12:23
Its great that this conversation is happening. Please continue it, BUT also realize that this is NOT the final point structure. You are giving us good feedback on your thoughts and expectations of what the advancement criteria should look like. Encourage others to take part on this thread. Several of us are monitoring it daily.
The final rounds of meetings are taking place now among the movers and shakers of FIRST in New England to finalize a LOT of things that need finalizing before we go into negotiations with FIRST HQ. And a LOT of the things we are talking about are exctly the things you all brought up in theTown Hall meetings of last fall. So you are having an impact. Keep it up.
Encourage others to take part on this thread. Several of us are monitoring it daily.
If this is true I'd like to add my two cents to the discussion.
First, I'd have preferred to see the 5/2 awards system in place in FiM and MAR. Although the point difference is kind of splitting hairs, I think it matters when a bid to CMP is on the line.
Secondly, I'd like to see the DCMP count more than a district. Frankly, a 60-80 team DCMP is way harder than a 30 team district and teams should be rewarded as such. I think the best points system for CMP qualification would be your best (or two best) district events and 2 x DCMP.
And lastly, I really like the points system for eliminations, however I have two things I'd like to see:
I think the points should be bumped to 10/15/25/35 because NE does not award points for selection the same way that FiM and MAR do. This rewards teams for making eliminations more than it did before, but not quite as much
I think that the points should also be adjusted to be lower for the third robot on an alliance similarly to how selection points are distributed in FiM. Maybe the first two robots on an alliance get 10/15/25/35 while the third robot gets 5/10/20/30.
Jay O'Donnell
09-04-2013, 15:20
I was actually planning on doing this during this week. Thanks for saving me a few days of work!
It seems that only the teams attending two regionals have a legitimate chance of doing really well. There are teams ranked below my own who I know are better than us, but we've attended two regionals, giving us a major boost.
tkell274
09-04-2013, 16:32
I think the points should be bumped to 10/15/25/35 because NE does not award points for selection the same way that FiM and MAR do. This rewards teams for making eliminations more than it did before, but not quite as much
I think that the points should also be adjusted to be lower for the third robot on an alliance similarly to how selection points are distributed in FiM. Maybe the first two robots on an alliance get 10/15/25/35 while the third robot gets 5/10/20/30.
I agree with your idea that the points should be bumped up for eliminations to really help teams that earned their spot in the eliminations. But I disagree with your point on giving the third robot less points. The randomness of qualifications and the possibility of robots breaking sometimes leads to teams qualification score not matching the true power of their robot and therefore they should not be penalized for how they are picked.
Kevin Leonard
09-04-2013, 16:57
I agree with your idea that the points should be bumped up for eliminations to really help teams that earned their spot in the eliminations. But I disagree with your point on giving the third robot less points. The randomness of qualifications and the possibility of robots breaking sometimes leads to teams qualification score not matching the true power of their robot and therefore they should not be penalized for how they are picked.
I agree with tkell. A team that builds a support-based or defense based robot that is still phenomenal in what it is built to do (i.e. 4334 last year and 2789 this year) shouldn't be penalized for pursuing a different strategy than the powerhouse offensive teams. Third picks often make or break alliances, especially at deeper events.
Chris is me
09-04-2013, 17:24
I agree with the logic on making the third pick worth just as many points. The serpentine in particular makes this problematic - I would hate to be the 8th seed and have to pull up on my phone which team "needs" the 1st round pick points more and which team doesn't.
2789_B_Garcia
09-04-2013, 17:30
I agree with tkell. A team that builds a support-based or defense based robot that is still phenomenal in what it is built to do (i.e. 4334 last year and 2789 this year) shouldn't be penalized for pursuing a different strategy than the powerhouse offensive teams. Third picks often make or break alliances, especially at deeper events.
Thanks! It's so hard to get love for defensive play...because of our limitations (personnel, funding, etc.) and sponsor issues, it's hard for us to crank out the robots we design and want. We're forced to compensate with scouting and strategy. I've been concerned about district discussions down here in Texas because of how formulas rate defensive teams, in particular because of how the dynamics for eliminations matches are very different than dynamics for quals matches, and our strategy definitely makes a bigger impact in elims. It's my hope that at the end of the month we'll be able to make the point that a creative and smart team can still make a strong impact on the outcome of matches even if you don't have the fanciest robot on the field...but needless to say, we are working on a few surprises for champs that will help us put points on the board :)
Nathan Streeter
09-04-2013, 21:01
I think that the points should also be adjusted to be lower for the third robot on an alliance similarly to how selection points are distributed in FiM. Maybe the first two robots on an alliance get 10/15/25/35 while the third robot gets 5/10/20/30.
I definitely do agree that the points for the 3rd robot should be adjusted... Primarily because that means that the 3rd robot on the winning alliance gets more points than any other alliance's robots (including the first two robots on all of the other alliances). Additionally, this method of assigning points gives very few points to the quarterfinalist alliances.
Although I suspect these points were removed in NE's proposal because it makes things a bit more complicated, I think assigning points based on alliance selection order as FiM and MAR do (16 to first 2 bots on Alliance 1; 15 to first 2 bots on Alliance 2; 14 to first 2 bots on Alliance 3... and 8 points to 3rd robot on Alliance 8, 7 points to 3rd robot on Alliance 7, 6 points to 3rd robot on Alliance 6) is the best way to assign points for eliminations (in addition to points based on finish).
Using 2013 GSR and 2013 Pine Tree as case studies (I chose these two because they're Week 1 vs Week 6, vary significantly in size, and Pine Tree is interesting because the red alliance won each matchup):
GSR Pick Order & Results:
610-4124-3609... W
138-131-58... QF
230-1991-1153... SF
885-1519-133... SF
151-229-1277... QF
1512-1922-1517... QF
61-175-172... F
2791-3467-78... QF
GSR Points (based on current NE proposal)
30-30-30... W
5-5-5... QF
10-10-10... SF
10-10-10... SF
5-5-5... QF
5-5-5... QF
20-20-20... F
5-5-5... QF
GSR Points (NE proposal + alliance selection points)
46-46-31... W
20-20-7... QF
24-24-13... SF
23-23-14... SF
17-17-10... QF
16-16-11... QF
30-30-27... F
14-14-13... QF
Pine Tree Pick Order & Results:
2648-3467-2386... W
176-125-63... F
1153-172-1831... SF
69-133-4564... SF
4473-58-1058... QF
78-1073-1922... QF
3930-4055-157... QF
3609-1071-181... QF
Pine Tree Points (based on current NE proposal)
30-30-30... W
20-20-20... F
10-10-10... SF
10-10-10... SF
5-5-5... QF
5-5-5... QF
5-5-5... QF
5-5-5... QF
Pine Tree Points (NE proposal + alliance selection points)
46-46-31... W
35-35-22... F
24-24-13... SF
23-23-14... SF
17-17-10... QF
16-16-11... QF
15-15-12... QF
14-14-13... QF
Seems like the current NE Proposal has several weaknesses:
- 1st and 2nd robots of each alliance get same reward as 3rd robot.
- Winners get 6x the points that the quarterfinalists get (3x the semifinalists).
- 1st and 2nd robots of finalist alliance (theoretically 3rd and 4th best teams) get 66% the points of the 3rd robot of the winning alliance (theoretically lower than 20th in ranking of teams).
These particular issues are improved with the inclusion of the alliance selection points. It'd be interesting to also add in the win-loss information... but I don't really have the time for that right now.
Although I suspect these points were removed in NE's proposal because it makes things a bit more complicated, I think assigning points based on alliance selection order as FiM and MAR do (16 to first 2 bots on Alliance 1; 15 to first 2 bots on Alliance 2; 14 to first 2 bots on Alliance 3... and 8 points to 3rd robot on Alliance 8, 7 points to 3rd robot on Alliance 7, 6 points to 3rd robot on Alliance 6) is the best way to assign points for eliminations (in addition to points based on finish).
The one thing I dislike about the FiM system assigning points based on Alliance Selection is that it gives points for essentially the same thing QF gives points for. The thing I do like is it is a good way to breakdown credit on an alliance.
A proposal I would support would be 8 ranking points for AC1 and First pick, and decreasing from there. This would give 3rd robots 0 extra points compared to the pack, but the difference between the last robot and the first is only 8 rather than 6. Then the NE Eliminations points could be bumped to 10/15/25/35 to make up for the point loss and emphasize results more.
Alliance Points:
8-8-0
7-7-0
6-6-0
5-5-0
4-4-0
3-3-0
2-2-0
1-1-0
Using BAE as an example: (10/15/25/35 + Alliance Points)
43-43-35... W
17-17-10... QF
21-21-15... SF
20-20-15... SF
14-14-10... QF
13-13-10... QF
27-27-25... F
11-11-10... QF
The one thing I dislike about the FiM system assigning points based on Alliance Selection is that it gives points for essentially the same thing QF gives points for.
While this is sometimes the case, awarding points based on Alliance Selection can also serve to balance out the variation in teams' qualification schedules. If a team has a great robot, but happens to lose a few qualification matches to tough opponents, being picked high can be a chance for them to regain some of those points.
One thing I keep coming back to with the proposed system for elimination points is the situation where the two strongest alliances at an event happen to face off before the finals. The bracket-style tournament is very effective at determining the best alliance, but it doesn't work as well when it comes to ranking the remaining seven alliances.
For example, let's say the higher seeded alliances all win their quarterfinals. Alliance #3 squeaks out a semifinal win over Alliance #2 in three close matches, but then goes on to win two very one-sided finals matches against Alliance #1. Based on this, it would be reasonable to say Alliance #2 is stronger than Alliance #1. Under the proposed system, Alliance #1 would receive twice as many points as Alliance #2 (who all evidence would suggest is the stronger alliance) based purely on the structure of the bracket.
I can't think of a good way to deal with this situation, short of a complex system that takes "strength of schedule" into account, or a different structure for elimination rounds (neither of which I think would be the right answer). But I do think that it's an important thing to keep in mind when creating a system like this.
Kims Robot
10-04-2013, 09:02
Has anyone run the numbers with the FiM or MAR points system so that its easy to see the difference between the current NE proposal and the FiM or MAR models?
Kims Robot
10-04-2013, 12:15
To add to the dicussion I did a quick World OPR Rank VLOOKUP for the NE Teams (Thanks to Ed & Ether's data (http://www.chiefdelphi.com/forums/showthread.php?t=115849))
This is by no means enough for ranking into DCMP or WCMP, it just gives a screenshot of offensively which teams should maybe make it into the top NE slots... and is just meant for comparison (ie if there is a team in 10th OPR that doesnt make it into the top 50 of our ranking system, its worth looking at to make sure the NE Ranking structure is a good balance).
I do believe some of this will balance out by normalizing for a single event...
Note: the numbers here are the OPR Ranks for World OPR Ranking, NE OPR Ranking and Jack/Brian's NE Ranks sorted by OPR
EDIT - HUGE Apologies - I forgot Rhode Island!! Added now!
Team/ World OPR/ NE OPR/ NE Rank
177 42 1 19
125 45 2 1
2648 55 3 2
131 71 4 60
126 73 5 5
3467 75 6 4
230 76 7 9
1519 101 8 3
236 115 9 46
195 137 10 12
885 141 11 40
69 149 12 15
176 162 13 13
4564 166 14 51
58 176 15 30
1100 208 16 7
716 226 17 98
3464 229 18 90
1153 236 19 23
175 246 20 14
558 257 21 21
1991 260 22 18
2170 261 23 92
4473 287 24 43
78 303 25 28
133 309 26 25
172 316 27 11
2067 326 28 16
3525 329 29 29
1474 372 30 119
2871 378 31 95
228 408 32 17
157 416 33 37
1699 459 34 81
2168 460 35 6
4097 469 36 88
2349 473 37 71
2064 502 38 104
246 505 39 73
1784 509 40 76
3780 547 41 49
348 565 42 112
178 573 43 93
3718 577 44 102
1965 579 45 85
1277 585 46 63
181 586 47 47
121 596 48 57
213 601 49 113
1350 608 50 70
4555 612 51 136
4176 623 52 103
4812 626 53 77
138 643 54 54
319 649 55 59
151 658 56 61
3323 694 57 100
61 702 58 33
1071 709 59 32
3104 720 60 97
2876 730 61 82
3205 764 62 45
1073 769 63 35
1517 772 64 39
4041 782 65 122
1512 789 66 62
3609 791 67 8
4055 824 68 27
3466 837 69 68
2423 845 70 69
155 867 71 10
4557 880 72 86
4761 886 73 55
1922 897 74 53
1831 903 75 38
2877 921 76 31
97 923 77 83
3958 935 78 20
1761 940 79 84
2084 966 80 108
88 980 81 34
3930 991 82 24
2785 1045 83 147
23 1050 84 106
3146 1070 85 66
3597 1110 86 116
1768 1146 87 94
571 1159 88 41
4048 1171 89 127
238 1180 90 99
190 1190 91 50
839 1200 92 36
3451 1209 93 135
1735 1212 94 42
1058 1218 95 56
3566 1224 96 64
2370 1254 97 67
1757 1256 98 120
3280 1287 99 22
663 1323 100 101
2713 1329 101 121
4311 1358 102 114
173 1375 103 65
3182 1378 104 52
3634 1410 105 123
3236 1414 106 74
4410 1540 107 115
4546 1543 108 44
4042 1547 109 111
3555 1574 110 152
166 1594 111 140
4793 1680 112 117
3499 1695 113 79
1027 1743 114 58
237 1752 115 91
4628 1814 116 139
2523 1833 117 126
509 1840 118 48
3479 1862 119 96
1687 1871 120 75
95 1882 121 26
1740 1887 122 137
3654 1927 123 78
3585 1928 124 87
3461 1997 125 138
4151 2000 126 130
999 2009 127 72
4474 2034 128 131
4034 2052 129 142
2836 2080 130 105
1289 2090 131 110
2104 2124 132 132
1754 2138 133 143
1124 2158 134 146
2342 2166 135 118
4796 2173 136 144
529 2192 137 145
3927 2266 138 109
1729 2270 139 150
2262 2284 140 129
4609 2292 141 89
467 2296 142 134
3719 2335 143 154
2079 2347 144 107
1973 2371 145 80
1721 2374 146 151
1247 2377 147 124
1307 2396 148 141
2621 2398 149 133
4572 2410 150 148
501 2420 151 125
3623 2463 152 153
811 2470 153 149
1099 2483 154 128
KrazyCarl92
10-04-2013, 12:40
I can't think of a good way to deal with this situation, short of a complex system that takes "strength of schedule" into account, or a different structure for elimination rounds (neither of which I think would be the right answer). But I do think that it's an important thing to keep in mind when creating a system like this.
How about awarding teams something like 4 or 5 pts. for eliminations victories in addition to selection points? This would not solve the problem in your example, but it would bring the results closer to reality:
Alliance 2 has 6 wins in elims totaling 6*5 = 30 pts.
Alliance 1 has 4 wins in elims (2 from QF, 2 from SF, 0 from F) totaling 4*5 = 20 pts.
Alliance 3 has 3 wins in elims (2 from QF, 1 from SF) totaling 3*5 = 15 pts.
This solves the issue Jack was referring to as well. If you include picking points with this, then you are not rewarding teams doubly for being picked and being quarter finalists. These alliances would only be granted additional points if they won a match in the quarter finals. Notably, this would also provide a good way to reward back up robots for their contributions to an alliance in elims. They would be rewarded only for the matches where they helped the alliance win.
Rosiebotboss
10-04-2013, 13:08
Not to change the subject, but how about points for awards? How many should be given for Chairmans? how many for EI? Safety? Entrepreneurship? Quality? Engineering Excellence?
Banderoonies
10-04-2013, 13:09
Not to change the subject, but how about points for awards? How many should be given for Chairmans? how many for EI? Safety? Entrepreneurship? Quality? Engineering Excellence?
it would also be nice to see points awarded to highest rookie seed.
PVCpirate
10-04-2013, 14:18
Not to change the subject, but how about points for awards? How many should be given for Chairmans? how many for EI? Safety? Entrepreneurship? Quality? Engineering Excellence?
From other threads it seems like FIRST is pretty adamant about Chairman's being an automatic qualifying award, so I won't touch that. I think there needs to be two tiers of awards at least, because I don't think something like the team spirit award should be in any way equivalent to say the Quality Award. I think things like Quality, Xerox Creativity, maybe rookie inspiration etc. should be one amount, with things like Spirit, Gracious professionalism, safety worth some lower amount of points. A third, higher tier would be EI and/or RAS if they end up not fully qualifying a team for the NE championship. I think these two should be worth something like what the finalist teams get.
How about awarding teams something like 4 or 5 pts. for eliminations victories in addition to selection points? This would not solve the problem in your example, but it would bring the results closer to reality:
Alliance 2 has 6 wins in elims totaling 6*5 = 30 pts.
Alliance 1 has 4 wins in elims (2 from QF, 2 from SF, 0 from F) totaling 4*5 = 20 pts.
Alliance 3 has 3 wins in elims (2 from QF, 1 from SF) totaling 3*5 = 15 pts.
This seems fairer in that it takes into account the "closeness" of a round in elims by giving partial credit to an alliance that forces a third match, but without penalizing the winning alliance.
Kim's data suggests that there is a rank correlation of about 0.69 between World OPR and NE proposal points, which is worse than I expected.
But since OPR is far from a perfect ranking measure, and we know there are problems with projecting NE district points from non-normalized Regional results, it's probably not a particularly useful measurement anyway!
Alliance 2 has 6 wins in elims totaling 6*5 = 30 pts.
Alliance 1 has 4 wins in elims (2 from QF, 2 from SF, 0 from F) totaling 4*5 = 20 pts.
Alliance 3 has 3 wins in elims (2 from QF, 1 from SF) totaling 3*5 = 15 pts.
I really like how this system grants points for winning one of three QF/SF/F matches. What I dislike is that alliances that are 0-2 in QF don't receive any extra points for making elims. This also doesn't address the issue of all three robots receiving the same number of points. (Although people seem split 50/50 on that) If every team received a base of 5 points for making elims that would help remedy the first problem. I'm open to suggestions on fixing the second one.
Not to change the subject, but how about points for awards? How many should be given for Chairmans? how many for EI? Safety? Entrepreneurship? Quality? Engineering Excellence?
I think that FiM was very clear in its decision to grant RCA, EI, and RAS autobids to DCMP. Assigning those awards points deflates their value and prestige. Unfortunately for rookies, it becomes difficult to assign points to the other two awards when RAS garners no points. I could see myself supporting autobid and 5 for RAS and 2 for the other two awards (I think MAR does this.)
I do like the 5/2 robot/non robot awards split in FiM.
I like the idea of giving alliances credit for forcing a third match.
I really like how this system grants points for winning one of three QF/SF/F matches. What I dislike is that alliances that are 0-2 in QF don't receive any extra points for making elims.
I think the implication was that points would be awarded for alliance selection, and they would have the points for a QF finish baked into them.
I like the idea of giving alliances credit for forcing a third match.
I think the implication was that points would be awarded for alliance selection, and they would have the points for a QF finish baked into them.
I was mistaken. Good catch. So I like Carl's system a lot actually.
Kim's data suggests that there is a rank correlation of about 0.69 between World OPR and NE proposal points, which is worse than I expected.
But since OPR is far from a perfect ranking measure, and we know there are problems with projecting NE district points from non-normalized Regional results, it's probably not a particularly useful measurement anyway!
I think normalized rankings will help this metric out a lot. I will be getting the spreadsheet out tonight to those who asked for it.
The raw data is available for viewing and download on google docs (https://docs.google.com/file/d/0B_noBCs6LVq3V2FmYmVzLTRGUW8/edit?usp=sharing).
Nathan Streeter
17-04-2013, 13:10
So, I was interested in tinkering around with point values and such, so I created a spreadsheet that has the "raw data" for all the New England teams' performances, including Qualification Match rank/stats, Alliance Selection order, Elim results, and Awards. I did not (yet) do it for any teams interested in joining the NE District as a "conference team." With a list of teams to add, I'd be glad to do so (I saw 20 and 2791 were included in the initially posted spreadsheet from this thread... didn't go through to see who the 10 NY teams were).
Points & Ranking Systems
A ranking system that I recommend, given the results and some time spent tinkering:
Alliance Selection
Cap1 16
Cap2 15
Cap3 14
Cap4 13
Cap5 12
Cap6 11
Cap7 10
Cap8 9
Pic1 16
Pic2 15
Pic3 14
Pic4 13
Pic5 12
Pic6 11
Pic7 10
Pic8 9
Pic9 8
Pic10 7
Pic11 6
Pic12 5
Pic13 4
Pic14 3
Pic15 2
Pic16 1
Elim Results
Winner 0
Finalist 0
SemiFinalist 0
QuarterFinalist 0
Win 4
Tie 2
Loss 0
Qual Results
W 2
T 1
L 0
Awards
Chairmans 25
Creativity 2
Deans List 5
Deans List 5
Eng Excellence 5
Eng Inspiration 15
Entrepreneurship 5
Gracious Prof 2
Imagery 2
Industrial Design 5
Innovation in Control 5
Judges 2
Quality 5
Rookie All Star 15
Rookie Inspiration 5
Rookie Seed 2
Safety 2
Team Spirit 2
Woodie Flowers 5
I assigned points for the alliance selection results because assigning points only for Qualification Results and Elimination Results (uniformly across the alliance) can lead to: teams with a tough/easy qual schedule getting less/more points, teams who were selected onto a weak/strong alliance getting less/more points, and strong teams (229 or 131 at GSR) on Quarterfinalist alliances getting much fewer points than third robots on Winning alliances. I feel very strongly about this being an element to which points are assigned.
I assigned points per number of wins in eliminations (2x qualification values) because it rewards alliances that push a third match (whereas the W=30, F=20, SF=10, QF=5 doesn't). Other than that distinction, either method is pretty reasonable. The pts/number of wins does also have the advantage of being really easy to explain... Even though it may seem I like proposing complexity, I try to only do it when it actually has a noteworthy improvement - otherwise I prefer simple!
For qualifications, I assigned 2pts per win and 1pt per tie. I don't see many people debating this approach... seems like it makes sense. Also has the advantage of just being teams' Qualifying Score!
For awards, I'm suggesting something pretty different from FiM, MAR, or the current NE First proposal. Because of the scalability concern (as you increase districts, more and more spots get filled with auto-bids), I think there should be either no or very few auto-bids. But, you don't need an auto-bid to make it easy for a particular aspect to practically guarantee you a spot at DCMP! Simply assigning a lot of points to various things (i.e. 24 pts for a Regional win (6wins*4pts), 25 pts for Chairmans, 15 for EI, and 10 for Rookie All Star) can practically ensure all those people get to DCMP. Now, with this approach, the Chairmans', EI, and RAS winners from each district really must be allowed to send a judging delegation to the DCMP to compete for the DCMP award (similar to MSC and MAR). Whether you autobid or assign points really only has to do with scalability.
How many points you assign to each element determines how much you want those teams to be at DCMP. You want to guarantee that every DCA (District Chairmans Award) winner goes to DCMP? Assign 100 points. Do you not really care if the DCA winner competes at DCMP any more than the Team Spirit award winner? Assign DCA 2 points. Some might want only the very best robots competing in matches... so they'd assign maybe only 5 pts to DCA, EI, and RAS (they're still eligible for their respective DCMP awards). Others might want these teams to be guaranteed spots at DCMP, so assign more points (25+). Really, I want to have a highly competitive field at DCMP that is every bit as fun, spirited, GP, and inspiring as possible! So, I'd like to see as many of the competitive teams get on the field as possible, amongst DCAs and EIs to keep those teams who model FIRST there, and with the RASs so they can see more of what FIRST is all about.
I think the Chairmans award should be the single biggest point-gathering feature... 25pts isn't unreasonable. With the above points, the DCAs are ranked 2, 3, 4, 5, and 15. If DCA got only 5 points, they would be ranked at 4, 6, 7, 12, and 36. Basically, the RCAs from this year were all very strong teams that only cemented their spots by getting 25pts for their win.
EI should also be awarded "extra" points, given it's role in FIRST. 15pts seems reasonable, and places the 4 New England EI winners (846 from Cali won EI at Boston) at 1, 9, 53, and 70. Keep in mind, the 79th ranked teams have 28pts. These 15pts from EI provided 53.4% of the points needed to qualify for DCMP, if implemented this year (given the 80+ team size currently proposed).
Finally, RAS is awarded 10pts. The 5 NE RAS winners are ranked at 20, 26, 49, 77, and 79. Again, given the 10 pts, RAS winners get 35.7% of the points needed to qualify for DCMP. I do think that it's important for the RASs to get to see DCMP, because qualifying and competing there may really inspire them for their upcoming seasons.
Rankings
Here are the rankings based off the above point system... feel free to double-check your team's points and finishes (Regional Results tab). I'm only putting the top 30 in here, since I don't currently know how to get it to show up as a scroll-able sub-window. Go ahead and download the .xls to see the full spreadsheet.
Rank Team State Pts
1 2648 ME 126.0
2 131 NH 120.0
3 1100 MA 118.0
4 230 CT 116.0
5 126 MA 111.0
6 1519 NH 108.0
7 125 MA 107.3
8 3280 RI 98.0
9 172 ME 88.0
10 3467 NH 86.0
11 155 CT 83.0
12 2168 CT 81.0
13 61 MA 80.0
14 885 VT 78.0
15 175 CT 76.0
15 78 RI 76.0
17 236 CT 74.0
17 3205 MA 74.0
19 177 CT 73.0
20 4564 ME 72.0
20 176 CT 72.0
22 69 MA 70.0
23 228 CT 68.0
23 3780 RI 68.0
25 195 CT 67.0
26 4761 MA 66.0
26 1153 MA 66.0
28 1991 CT 64.0
28 4473 ME 64.0
30 133 ME 62.0
Notes on Spreadsheet & Ranking Calcs
I normalized the rankings for 2 regionals... so teams that competed at 1 got their points doubled, while teams that competed at 3 regionals got their points multiplied by .667.
I assigned points as described above.
I listed most of the accomplishments of the top 64 or so teams on the "Team List" tab, so they can be easily referenced alongside the score. Only 2 teams (one EI, one RAS) that hadn't made elims at all were in these top 64. I believe only 10 of the 154 NE teams won a regional (15 teams won from the 5 NE regionals).
The OPR was taken from Ed Law's spreadsheet after week 7. I just took the single-highest event OPR. For those curious, the excel calculated correlation between my suggested points and that OPR was .784. Assigning all trophies only 2 pts provided .799. Assigning no alliance selection points provided .728. Assigning the 30, 20, 10, 5 for elim finish instead of 4pts/win provided a correlation of .722. Combining these last two provided .667. Seems like this correlation of .784 is pretty reasonable... a correlation of 1.000 would mean we don't value anything but on-field performance and are tied to a flawed a statistic. This approximately .8 correlation indicates a significant - but not identical - relationship.
Feel free to examine the NE rankings (sorting by anything) and the Regional Results (to see who got what at each regional), but make sure that all the tables from Regional Results are sorted by team number so the LOOKUP formulas work as they should. :-)
Thoughts on Qualifying for CMP (not DCMP)
Not sure we should simply have as many slots as 6 per regional (from when regionals were last used). I much prefer the method that FiM just started to implement which sends a number proportional to the number of teams in the district to the number of teams in FIRST. Given the 154 teams in NE, the ~2600 teams in FIRST, and the 400 slots at CMP, that'd mean for 2013, there'd be 24 teams from NE going. 27 are registered for CMP from NE currently.
I also think that DCMP points should be worth more than a single district, given that it is the only time all the top contenders are competing against each other (helps level easy/hard districts) and given that it is the last event. By DCMP, teams are probably going to be pretty close to the level they would be at CMP. I think a weight of between 2 and 3 would make sense.
Anyway, that's all for now... I'm eager to hear what people think of these things, particularly given that a fair number of my suggestions differ from the current NE proposal.
Chris is me
17-04-2013, 13:13
Just a note: FIRST HQ has consistently shot down attempts to give the Chairman's Award, EI, and RAS any point value. The awards are supposed to be "more important than points" or something like that.
Nathan Streeter
17-04-2013, 13:33
Just a note: FIRST HQ has consistently shot down attempts to give the Chairman's Award, EI, and RAS any point value. The awards are supposed to be "more important than points" or something like that.
Yeah, I'm familiar with that... I just wish that weren't the case. Quite frankly, having RAS and EI be 0 points and not auto-qualify a team for competition at the DCMP (other than for their respective award)... which is the case in FiM and MAR... makes less sense than either "pure" approach (imho). It means that RAS and EI aren't valuable enough to give you points to get to DCMP or CMP. Also, winning Chairmans (at Districts) doesn't give you points to help get to CMP.
I see these as both being factors that make the current system for these awards not line up with the prestige FiM, MAR, and FIRST HQ all assign the awards. I'm guessing this is mostly a weird set of compromises. Having them all be auto-qualify really wouldn't work for scalability, so it seems like an improved point method is the best alternative!
* end rant and stabilize heart rate *
Also of note,
Your point system matter little to who is at the top unless you choose to make an award for that (currently just bragging rights). It matters most at your cut points. Specifically where do you cut your invites off at for the event, and to go to the next level of the event. This is the area to pay the most attention to.
I rather like the way you are awarding Elim points. A successful champion will earn an additional 24 points (in theory ties could add more). This makes a good argument that Chairman's at 24 points is more significant than winning.
It does have a wierd issue though that winning an elim matches is worth 2x many of the judged awards.
BrendanB
17-04-2013, 13:35
Nathan, thanks for compiling all of this data as well as your insightful post!
Andrew Bates
17-04-2013, 14:53
Does MAR and FIM awards points to teams for individual awards such as the Dean's List and Woodie Flowers? I agree that those are great awards. However it's not the team winning the award but the individual.
You can only win Chairman's once. Did you correct for this when doubling?
Also, even though the conference model won't be adopted, I think including NY teams in the rankings is important. As far as the 2013 season is concerned, all (except 250) are NE teams and can their performances can still be used as a litmus test for how good/bad a ranking system is. The ten NY teams are: 20, 2791, 3044, 4134, 250, 4508, 1493, 1665, 3687, 4254.
Nathan Streeter
17-04-2013, 15:11
Does MAR and FIM awards points to teams for individual awards such as the Dean's List and Woodie Flowers? I agree that those are great awards. However it's not the team winning the award but the individual.
MAR and FiM point systems are identical to each other, with the exception of RAS getting 5 pts in MAR. Neither give points for DL or WF...
Whether you should or not is a matter of opinion, obviously... I did intentionally include them in my suggestion though. Good teams are - in a significant part - made better by DL-type students and WF-type mentors. Similarly, DL-type students and WF-type mentors are fostered by good teams. Looking down lists of DL and WF award winners (particularly at the CMP level), you rarely see the names associated with a number you don't recognize (as FIRSTers who know a fair bit about "who's who").
So, does that mean the individual award (which was submitted/authored by someone other than the individual him/herself) should get points for the team? Some say yes, others no. I say yes because we're trying to give points "good" teams. I see a DL or WF award winner as typically being a good distinguisher of a good team (similar to Chairmans, Winner, Seed, etc.).
Note, I'm not saying that... 1) all great teams have DL or WF winners/finalists, 2) anyone who has won DL or WF was on a great team, 3) all great teams submit DL and WF nominations, or 4) great teams consistently win DL or WF
Using Nathan's spreadsheet, I quickly adjusted the point values to the most recent NE proposal based off information I got from last weekend's points meeting.
Qual Win: 2
Qual Tie: 1
Elim Win: 5
Elim Tie: 0
Awards: 10
RCA, EI, RAS given auto qual to NE CMP
Rank Team State Pts OPR Regionals
1 20 NY 133.0 61.1 MAWO, CTHA
2 125 MA 90.7 51.8 FLOR, MABO, MELE
3 2648 ME 90.0 56.5 MAWO, MELE
4 3280 RI 88.0 18.1 MAWO
5 1519 NH 82.0 64.1 NHMA, NCRA
6 1100 MA 81.0 37.0 MAWO, MABO
7 2791 NY 79.0 37.3 NHMA, MAWO
8 230 CT 73.0 44.9 NHMA, CTHA
9 3467 NH 72.0 72.8 NHMA, MELE
9 885 VT 72.0 39.6 NHMA
11 61 MA 68.0 26.8 NHMA
12 1153 MA 66.0 48.0 NHMA, MABO, MELE
12 3609 ME 66.0 22.2 NHMA, MELE
14 2168 CT 65.0 43.1 MAWO, CTHA
15 155 CT 64.0 25.5 MAWO, LAKE
16 172 ME 63.0 33.5 NHMA, MELE
17 126 MA 62.0 73.9 MAWO, MABO
18 195 CT 60.0 62.3 VARI, CTHA
19 1991 CT 59.0 47.3 NHMA, CTHA
20 175 CT 58.0 41.4 NHMA, CTHA
21 4473 ME 56.0 28.0 MELE
22 228 CT 55.0 54.3 MAWO, CTHA
23 176 CT 54.0 40.0 NYRO, MELE
23 3930 ME 54.0 15.6 MAWO, MELE
25 236 CT 52.0 52.7 CTHA
25 3205 MA 52.0 24.9 MAWO
27 3958 MA 51.0 41.6 MAWO, MABO
28 1699 CT 50.0 27.7 MDBA, CTHA
29 4564 ME 48.0 33.1 MELE
29 3780 RI 48.0 18.2 MABO
29 3525 CT 48.0 26.8 CTHA, MELE
These points are doubled for those who only attended one regional and multiplied by .667 for those who attended 3. Remember 11 spots of 30 will be given to RCA/EI/RAS, so only the top 19 on points qualify. I included the top 30 and ties because these teams are most likely to compete for points at NE CMP. I do not know the proposed NE points multiplier for the NE CMP.
*Note that 126 and 236 have autobids to CMP and would not take spots from other teams
PVCpirate
17-04-2013, 16:12
So that is 10 points for all awards except for RAS, Chairman's and EI? Interesting.
Ken Streeter
18-04-2013, 01:20
Using Nathan's spreadsheet, I quickly adjusted the point values to the most recent NE proposal based off information I got from last weekend's points meeting.
Qual Win: 2
Qual Tie: 1
Elim Win: 5
Elim Tie: 0
Awards: 10
RCA, EI, RAS given auto qual to NE CMP
Does the above imply that "last weekend's points meeting" resulted in a consensus to assign no points for "alliance selection" to either the seeded captains or their picks?
I hope that's not the case, as the points for alliance selections are actually a key element in having the points earned by teams make sense despite potentially unbalanced qualification match schedules (easy/hard). The alliance selection points are also the only points to distinguish between the robots of 1st picks vs. 2nd picks, and the only way to distinguish between the robots that make the elimination rounds vs. those that do not. Removing these points from the system takes away all of these key pieces of the point assignment system.
For example, if elimination rounds are all won by the higher-seeded alliances (such as at this year's Pine Tree Regional), then the point assignments to robots do not bear much resemblance to which robots perform the best, particularly for the robots near the "cutoff point." In particular, the 3rd robot on the #1 alliance (16th pick of the draft) comes away with 30 elim points (due to 6 elim wins), while the 5th-seeded captain and 1st pick of the 5th-seeded captain come away with 0 elim points. The same is true for the captains and first picks of the #6 alliance, #7 alliance, and #8 alliance -- they will all get no elimination points at all, meaning they have no bonus over all the other robots that didn't play in eliminations. Meanwhile, the 3rd robot on the #2 alliance (15th draft pick) will acquire at least 20 elim points, and the 3rd robots on the #3 and #4 alliances (14th and 13th picks) will get at least 10 elim points.
Having a scoring system that is most likely to earn the 13th, 14th, 15th, and 16th picks of the draft 10-30 elimination points each, while the 5th, 6th, 7th, 8th captains and their 1st picks (5th, 6th, 7th, and 8th picks of the draft) are most likely to get 0 points from elimination rounds isn't going to do a very good job of correlating more points with the better robots!
I'm not trying to slight any teams that were 13th, 14th, 15th, or 16th teams of the draft, but in general, the 5th, 6th, 7th, and 8th captains and the 5th, 6th, 7th, and 8th picks of the draft will be better robots than the 13th, 14th, 15th, and 16th picks and should generally receive more points in a point assignment system, if that point assignment system intends to send the more capable robots to the division championship.
smistthegreat
18-04-2013, 01:23
You can only win Chairman's once. Did you correct for this when doubling?
Also, even though the conference model won't be adopted, I think including NY teams in the rankings is important. As far as the 2013 season is concerned, all (except 250) are NE teams and can their performances can still be used as a litmus test for how good/bad a ranking system is. The ten NY teams are: 20, 2791, 3044, 4134, 250, 4508, 1493, 1665, 3687, 4254.
If you're using the NY teams as a test for this year, you can throw in 229 and 4124 too as they both competed exclusively in NE this season.
Does the above imply that "last weekend's points meeting" resulted in a consensus to assign no points for "alliance selection" to either the seeded captains or their picks?
I hope that's not the case, as the points for alliance selections are actually a key element in having the points earned by teams make sense despite potentially unbalanced qualification match schedules (easy/hard). The alliance selection points are also the only points to distinguish between the robots of 1st picks vs. 2nd picks, and the only way to distinguish between the robots that make the elimination rounds vs. those that do not. Removing these points from the system takes away all of these key pieces of the point assignment system.
For example, if elimination rounds are all won by the higher-seeded alliances (such as at this year's Pine Tree Regional), then the point assignments to robots do not bear much resemblance to which robots perform the best, particularly for the robots near the "cutoff point." In particular, the 3rd robot on the #1 alliance (16th pick of the draft) comes away with 30 elim points (due to 6 elim wins), while the 5th-seeded captain and 1st pick of the 5th-seeded captain come away with 0 elim points. The same is true for the captains and first picks of the #6 alliance, #7 alliance, and #8 alliance -- they will all get no elimination points at all, meaning they have no bonus over all the other robots that didn't play in eliminations. Meanwhile, the 3rd robot on the #2 alliance (15th draft pick) will acquire at least 20 elim points, and the 3rd robots on the #3 and #4 alliances (14th and 13th picks) will get at least 10 elim points.
Having a scoring system that is most likely to earn the 13th, 14th, 15th, and 16th picks of the draft 10-30 elimination points each, while the 5th, 6th, 7th, 8th captains and their 1st picks (5th, 6th, 7th, and 8th picks of the draft) are most likely to get 0 points from elimination rounds isn't going to do a very good job of correlating more points with the better robots!
I'm not trying to slight any teams that were 13th, 14th, 15th, or 16th teams of the draft, but in general, the 5th, 6th, 7th, and 8th captains and the 5th, 6th, 7th, and 8th picks of the draft will be better robots than the 13th, 14th, 15th, and 16th picks and should generally receive more points in a point assignment system, if that point assignment system intends to send the more capable robots to the division championship.
Playing Devil's Advocate: if, as you argue, qualification W-L-T is subject to scheduling randomness, then why would you give an additional bonus via Alliance Captain points? There might be an argument for more points given to a high pick rather than a high captain, but that would obviously lead to it's own set of inconsistencies.
Another way to look at the problem is how to devise a simple to understand system that does not change the calculus of how a team plays the game.
Currently a team simply wants to be part of the strongest alliance for eliminations.
With alliance selection points "strongest alliance" has to be balanced with getting points. Concretely would a strong team seeded #6 change it's thinking about accepting/declining if picked by a moderate team seeded #1?
I would argue that sliding alliance selection points do change the calculus of alliance selection, whereas points per elimination win (or for finishing position) keep the game the same.
If you're using the NY teams as a test for this year, you can throw in 229 and 4124 too as they both competed exclusively in NE this season.
Qual Win: 2
Qual Tie: 1
Elim Win: 5
Elim Tie: 0
Awards: 10
RCA, EI, RAS given auto qual to NE CMP
The following NY teams were in/near top 30 and were included: 20,229,2791,4124.
Rank Team State Pts OPR Regionals
1 20 NY 133.0 61.1 MAWO, CTHA
2 125 MA 90.7 51.8 FLOR, MABO, MELE
3 2648 ME 90.0 56.5 MAWO, MELE
4 3280 RI 88.0 18.1 MAWO
5 1519 NH 82.0 64.1 NHMA, NCRA
6 1100 MA 81.0 37.0 MAWO, MABO
7 4124 NY 80.0 36.0 NHMA
8 2791 NY 79.0 37.3 NHMA, MAWO
9 230 CT 73.0 44.9 NHMA, CTHA
10 3467 NH 72.0 72.8 NHMA, MELE
10 885 VT 72.0 39.6 NHMA
12 61 MA 68.0 26.8 NHMA
13 1153 MA 66.0 48.0 NHMA, MABO, MELE
13 3609 ME 66.0 22.2 NHMA, MELE
15 2168 CT 65.0 43.1 MAWO, CTHA
16 155 CT 64.0 25.5 MAWO, LAKE
17 172 ME 63.0 33.5 NHMA, MELE
18 126 MA 62.0 73.9 MAWO, MABO
19 195 CT 60.0 62.3 VARI, CTHA
20 1991 CT 59.0 47.3 NHMA, CTHA
21 175 CT 58.0 41.4 NHMA, CTHA
22 4473 ME 56.0 28.0 MELE
23 228 CT 55.0 54.3 MAWO, CTHA
24 176 CT 54.0 40.0 NYRO, MELE
24 3930 ME 54.0 15.6 MAWO, MELE
26 236 CT 52.0 52.7 CTHA
26 3205 MA 52.0 24.9 MAWO
28 3958 MA 51.0 41.6 MAWO, MABO
29 1699 CT 50.0 27.7 MDBA, CTHA
30 4564 ME 48.0 33.1 MELE
30 3780 RI 48.0 18.2 MABO
30 3525 CT 48.0 26.8 CTHA, MELE
33 177 CT 47.0 53.3 MAWO, CTHA
34 69 MA 46.0 39.9 MABO, MELE
34 2067 CT 46.0 42.9 MAWO, CTHA
34 558 CT 46.0 40.7 MAWO, CTHA
34 2349 MA 46.0 27.7 MABO
38 4761 MA 44.0 14.6 MABO
38 133 ME 44.0 38.3 NHMA, MELE
38 95 NH 44.0 11.8 NHMA, CTHA
38 1784 CT 44.0 19.7 CTHA
38 3654 CT 44.0 5.2 MAWO
38 4812 CT 44.0 13.6 CTHA
38 229 NY 44.0 41.7 NHMA, MABO
45 131 NH 42.0 50.8 NHMA
46 58 ME 41.0 50.4 NHMA, MELE
47 4557 CT 40.0 17.2 MAWO
47 2877 MA 40.0 19.7 MAWO, MABO
47 1277 MA 40.0 29.9 NHMA
47 2370 VT 40.0 9.3 MAWO
51 571 CT 39.0 18.5 MAWO, CTHA
52 1073 NH 38.0 26.5 NHMA, MELE
53 1517 NH 37.0 25.2 NHMA, QCMO
53 3182 CT 37.0 20.8 MAWO, CTHA
55 2423 MA 36.0 17.3 MABO
55 4609 RI 36.0 -1.3 CTHA
57 138 NH 35.0 26.6 NHMA, DCWA
57 1071 CT 35.0 28.6 CTHA, MELE
59 4055 CT 34.0 20.3 CTHA, MELE
59 88 MA 34.0 17.7 SCMB, MABO
61 78 RI 33.0 38.4 NHMA, MELE
61 1735 MA 33.0 21.1 MAWO, DCWA
63 121 RI 32.0 27.3 MAWO
63 151 NH 32.0 24.8 NHMA
63 1512 NH 32.0 20.0 NHMA
63 839 MA 32.0 24.1 MAWO, CTHA
63 1831 NH 32.0 16.5 NHMA, MELE
63 2836 CT 32.0 1.5 CTHA
63 4474 MA 32.0 -0.7 MABO
63 2064 CT 32.0 28.0 CTHA
71 1922 NH 29.0 20.7 NHMA, MELE
72 4546 NH 28.0 12.0 NHMA, MELE
72 3466 MA 28.0 16.5 MABO
72 3464 CT 28.0 35.7 CTHA
72 4793 ME 28.0 3.2 MELE
72 2342 NH 28.0 6.0 NHMA
77 2170 CT 26.0 37.1 CTHA
77 1350 RI 26.0 24.3 MABO
77 663 MA 26.0 14.0 MAWO
80 246 MA 24.0 23.1 MABO
80 3236 MA 24.0 8.1 MABO
80 3499 NH 24.0 13.0 MAWO
80 716 CT 24.0 30.5 MELE
80 1687 MA 24.0 9.1 MAWO
80 1768 MA 24.0 15.2 MABO
80 2871 MA 24.0 28.0 MABO
80 3104 CT 24.0 13.2 CTHA
80 3323 NH 24.0 26.5 NHMA
80 3479 MA 24.0 3.1 MABO
80 3718 CT 24.0 19.4 CTHA
80 1099 CT 24.0 -14.0 MABO
92 157 MA 22.0 33.3 MAWO, MELE
92 4176 MA 22.0 19.3 MABO
94 1027 MA 21.0 9.0 SCMB, CTHA
95 2876 MA 20.0 22.1 MABO
95 97 MA 20.0 17.2 MABO
95 1761 MA 20.0 15.7 MABO
95 181 CT 20.0 23.3 CTHA, MELE
95 1965 MA 20.0 22.2 MABO
95 23 MA 20.0 14.8 MABO
95 173 CT 20.0 13.3 MAWO, CTHA
95 213 NH 20.0 24.5 NHMA
95 348 MA 20.0 18.4 MELE
95 509 NH 20.0 8.6 NHMA
95 2079 MA 20.0 -4.9 MABO
95 2084 MA 20.0 14.0 MABO
95 3927 MA 20.0 -3.7 MABO
95 4042 ME 20.0 3.5 MELE
109 190 MA 19.0 21.4 MAWO, ONTO2
109 999 CT 19.0 4.6 NYNY, CTHA
111 501 NH 18.0 -5.5 MABO
111 3597 ME 18.0 10.7 MELE
111 4311 MA 18.0 13.5 MABO
111 4410 MA 18.0 15.6 MAWO
115 1058 NH 17.0 18.8 NHMA, MELE
116 238 NH 16.0 13.1 NHMA, MDBA
116 319 NH 16.0 34.8 NHMA, MELE
116 1247 NH 16.0 3.8 NHMA
116 1474 MA 16.0 25.7 MABO
116 1757 MA 16.0 7.4 MABO
116 1973 MA 16.0 -0.5 NHMA, MABO
116 2523 VT 16.0 4.4 MABO
116 2713 MA 16.0 12.3 MABO
116 3634 CT 16.0 0.5 CTHA
116 4041 ME 16.0 17.2 MELE
116 4097 MA 16.0 16.0 CTHA
127 3566 MA 15.0 17.8 NHMA, MELE
127 3585 NH 15.0 2.5 NHMA, MELE
129 166 NH 14.0 12.8 NHMA, DCWA
129 237 CT 14.0 13.7 NYNY, CTHA
129 3146 CT 14.0 21.5 MAWO, CTHA
129 4048 MA 14.0 11.1 MABO
133 178 CT 13.0 23.9 VARI, CTHA
134 467 MA 12.0 1.2 MAWO
134 1307 NH 12.0 2.4 NHMA
134 1740 CT 12.0 7.9 CTHA
134 2104 MA 12.0 5.6 MAWO
134 2262 MA 12.0 -3.8 MABO
134 2621 MA 12.0 1.8 MAWO
134 3451 ME 12.0 11.5 MELE
134 3461 CT 12.0 9.4 CTHA
134 4034 NH 12.0 8.0 NHMA
134 4151 MA 12.0 0.8 MABO
134 4555 ME 12.0 16.0 MELE
134 4628 CT 12.0 -2.4 CTHA
146 1289 MA 10.0 14.6 MAWO, CTHA
147 4572 CT 8.0 -8.1 CTHA
147 529 MA 8.0 1.9 MABO
147 811 NH 8.0 -3.4 NHMA
147 1124 CT 8.0 5.5 CTHA
147 1721 NH 8.0 -0.5 MABO
147 1729 NH 8.0 -0.2 NHMA
147 1754 MA 8.0 1.2 MABO
147 2785 CT 8.0 7.9 CTHA
147 4796 MA 8.0 -0.5 MABO
156 3555 CT 6.0 8.5 CTHA
157 3623 MA 4.0 -3.5 MAWO
158 3719 CT 0.0 -5.9 CTHA
Ken Streeter
20-05-2013, 17:14
Playing Devil's Advocate: if, as you argue, qualification W-L-T is subject to scheduling randomness, then why would you give an additional bonus via Alliance Captain points? There might be an argument for more points given to a high pick rather than a high captain, but that would obviously lead to it's own set of inconsistencies.
I actually think that in district events which have relatively small numbers of teams compared to the number of matches played (e.g. in FiM districts, with no more than 40 teams in an event, and with at least 12 matches played), the qualification W-L-T is less subject to scheduling randomness and the ranking system will do a better job sorting teams. As we see in single-day off-season events, when less qualification matches are played, seed rankings get more "noise" in them and are less well correlated to the best robots.
Have FiM teams generally found that FiM district events result in seeding rankings that are pretty good - ie, there aren't many high-seeding robots that seem out of place at FiM district events?
However, even when there is some scheduling randomness leading to a mis-sorted ranking, I think the benefit of giving bonus points to well-picked robots outweighs the harm of giving bonus points to the "lucky" captain who seeded higher than they deserved to be. There will be both more robots picked than captains (16 vs 8) and the "diamond in the rough" picks will typically gain more points relative to their pre-pick points than the "lucky" captains will.
I agree that having the "lucky" captains get alliance selection points is less than optimal, but I think this is a case where we need to go with the lesser of two evils in order to avoid throwing the baby out with the bathwater.
Another way to look at the problem is how to devise a simple to understand system that does not change the calculus of how a team plays the game. Currently a team simply wants to be part of the strongest alliance for eliminations.
I agree that I don't like the thought that teams would change their decisions of who to pick, or who to accept, based upon the point system. I would prefer that teams simply make picking / acceptance decisions based upon the event, rather than the points system. However, as long as district points are gained in eliminations, potential in-tournament point considerations will always be a factor.
With alliance selection points "strongest alliance" has to be balanced with getting points. Concretely would a strong team seeded #6 change it's thinking about accepting/declining if picked by a moderate team seeded #1? I would argue that sliding alliance selection points do change the calculus of alliance selection, whereas points per elimination win (or for finishing position) keep the game the same.
Even if no "alliance selection" points are issued, but points are assigned for winning matches in eliminations, we have this problem. For example, if there are no alliance selection points but getting eliminated in the finals is worth more points than getting eliminated in the semifinals (i.e. points for elimination win, as you mention), in a tournament with an emerging very strong #1 alliance (think 1114/2056), a #6 captain selected by a #4 or #5 seed has a strong motivation to decline and instead captain their own #6 alliance. The #4/#5 alliance would have very little chance to get to the finals, but the #6 alliance would have a reasonable shot, since they wouldn't face #1 in the semifinals. (By the way, I think the above scenario may have actually played out this year in at least one regional, due to the "wild card" assignment -- getting eliminated in the finals could earn a ticket to CMP at late regionals, while getting eliminated in the semifinals resulted in an ended season.)
Actually, I think with a relatively small (30-40 team) district event and many (12+) matches, the scenario you describe (of a weak #1 seed who might want to be declined by a #6 captain) is not very likely due to better sorting of teams. When it does happen, I think it would be a relatively rare event. However, I think the strong #1 alliance situation I mention above, where a #4 or #5 captain might select a #6 or #7 seed, is more likely to occur (just ask a Toronto-area team.) Then again, here in New England, we benefit from the fact that we don't have any real powerhouse teams that are way above the pack, so it might be less of an issue for us here than in some other regions.
Ken Streeter
20-05-2013, 21:34
Somewhat new theme - Philosophy of NE FIRST Points System.
I'm not entirely sure whether to start a new CD thread for this topic or continue the discussion on the NE FIRST Points System in this existing thread. I've opted to keep it in this thread, presuming that those who are interested in the NE FIRST Points System are already here...
I realize that for the NE FIRST folks, determining the points system is really one of the minor details with regard to switching to the district system. Nearly all of the work is in things like planning district event locations, times and venues; getting plans in place for sponsors and volunteers for events; figuring out a venue and organization for the NE championship, etc.
Figuring out which teams attend the NE CMP is really practically irrelevant from a perspective of switching over to the district model for the NE FIRST organizers -- regardless of which teams attend, there's an incredible amount to work through for the NE FIRST committee! However, for the teams, it is probably one of the top 5 issues of interest.
As has been mentioned previously in this thread by IKE, the real thing to look at in the rankings is the teams near the cutoff. The "obvious" teams will make the cut with nearly any points system. However, the real question is whether the teams that barely make the cut generally make sense as opposed to ones that fall just below the cut? Alas, this is actually kind of hard to evaluate without attending most of the regional events, as familiarity with which teams are more/less deserving is hard to evaluate for teams at this 75th percentile level.
When trying to figure out potential ways to assign teams points for qualification to NE CMP, I realized that it is essential to consider the philosophy of "Who Qualifies for the NE CMP."
In order to evaluate / consider different approaches for tallying up district points, it is necessary to first consider what is desired with respect to "Who Qualifies for the NE CMP?" Without knowing the goal of the "points ranking system" it is hard (impossible) to evaluate any given ranking system against that goal.
Clearly, the tautological objective of the proposed NE FIRST "points ranking system" is to determine which teams qualify for the NE CMP. But, what types of teams does NE FIRST want to qualify? Let's list a few of the potential objectives / types of teams that I have heard mentioned:
* The best robots as determined by demonstrated performance on the field
* The best teams at demonstrating the principles / ideals of FIRST
* The best robots as assessed by knowledgeable FRCers, even if they encountered bad luck in demonstrated performance
* The best robots as determined by judged awards
* The teams that would most benefit from the inspiration derived from attending the district championship
* The teams that haven't been to the district championship recently
From my understanding as an outside observer, the FiM points ranking system is specifically intended to put the best robots onto the playing field at the Michigan State Championship. This is no secret, as it is publicly stated all over the place, and is a regular point made about the FiM Championship - that it has the highest average scores of any event in the world, exceeding even the FIRST World Championship in average caliber of robot capabilities. In accordance with this, the FiM points ranking system provides more points to achievements which are highly correlated with the best robots, and less points to achievements that have less to do with the best robots. (The FIRST World Championship has more excellent robots in attendance, since it draws on robots from around the world, but the average is lowered by including not only the best robots, but also teams that have won the Chairman's Award, Engineering Inspiration Award, or Rookie All-Star award at regional events.)
The FiM leadership has intentionally assigned the point system the way it is to try to put the best robots onto the playing field at the Michigan Championship. As a result, even though FiM district winners of the district chairman's award, district engineering inspiration award, and district rookie-all-star awards get to participate in the Michigan Championship judging for those awards, only the district chairman's award winners automatically qualify to compete with their robot. District engineering inspiration awards and district rookie-all-star awards do not qualify for robot positions at the Michigan Championship, but instead qualify the team to send students to present to the Michigan Championship judges for those awards at the state level. Only district winners of those awards compete for those awards at the Michigan Championship. The Michigan State Championship winners for Chairman's Award (3 selected winners), Engineering Inspiration (1 winner) and Rookie All Star (1 winner) all qualify for the FIRST World Championship. (Such participation includes their robots at the World Championship.)
If NE FIRST's objective for the NE Championship is to have the best robots on the playing field at the NE Championship, then careful consideration should be given to simply adopt the FiM/MAR points system. I can only presume that the FiM folks have each of the different kind of points in their ranking system for a reason. Personally, I think there are good reasons for including the alliance selection points and different amount of elimination points for captains and 1st-picks, vs 2nd-picks. That said, NE FIRST may have a different perspective/objective than that of FiM, and intentionally want to do something other than seek to put the best robots on the playing field at the NE Championship. If so, that would be a good reason to do point assignments differently. Alternatively, even if the goal at NE FIRST is to put the best robots on the field, there may be other ways to determine the best robots that FiM either didn't explore, or couldn't get sufficient backing to implement. Personally, I think NE FIRST should also consider such alternate point scoring systems, but should also realize that the FiM/MAR system has been field tested quite a bit!
Kims Robot
20-05-2013, 21:57
I realize that for the NE FIRST folks, determining the points system is really one of the minor details with regard to switching to the district system. Nearly all of the work is in things like planning district event locations, times and venues; getting plans in place for sponsors and volunteers for events; figuring out a venue and organization for the NE championship, etc.
Figuring out which teams attend the NE CMP is really practically irrelevant from a perspective of switching over to the district model for the NE FIRST organizers -- regardless of which teams attend, there's an incredible amount to work through for the NE FIRST committee! However, for the teams, it is probably one of the top 5 issues of interest.
I just wanted to note that while there are many valid points in your discussion, this section is rather untrue. If you look at the "status" page (http://www.nefirst.org/current-status/) of the NE FIRST website, you will have a glimpse of how much work went into putting together a points model for NE FIRST. Almost immediately they split into operations and organization committees and began work on both parts of the system. They actually spent a lot more time working on and discussing this in the past year and a half than they did on locations. They realized this was incredibly important to teams and not only carefully researched and discussed all the models, they came up with a model that at times has been vastly different from FiM and MAR. As they discussed with teams at town halls, the model would change from month to month as they developed a picture of what the teams wanted. Even more recently, given a lot of these CD threads and opinions they have gathered from teams, they have changed the model to match what they are hearing from teams.
Are there still issues with the current proposal? Well they will never be able to make everyone happy. But they have NOT just blindly adopted the FiM & MAR models as you can see from several of the rankings threads. There are plenty of ways to argue which is better, and to be honest, as you suggest - it is all about what Philosophy NE FIRST & its teams wants to take with regards to its model. Are they in it to send the best robots? Do they want to generate more Chairmans winners? Do they want to give everyone a chance to play? Do they want to promote growth and sustainability? Each of these can lend itself to a slightly different model.
But I wanted to at least set the record straight that the NE FIRST committee has spent quite a lot of time going over and revising the points model, and it is far from an afterthought or a minor detail.
Ken Streeter
20-05-2013, 23:17
I just wanted to note that while there are many valid points in your discussion, this section (qualification points model) is rather untrue. If you look at the "status" page (http://www.nefirst.org/current-status/) of the NE FIRST website, you will have a glimpse of how much work went into putting together a points model for NE FIRST. ...
But I wanted to at least set the record straight that the NE FIRST committee has spent quite a lot of time going over and revising the points model, and it is far from an afterthought or a minor detail.
I'm sorry; I had not intended to offend or imply that the NE FIRST committee thought the points model was an afterthought. I definitely didn't think that the NE FIRST committee thought it was unimportant, but simply thought it had not yet been an issue they have had the bandwidth to address.
With your post, I am clearly mistaken. I have not been aware of any of the ongoing discussion within NE FIRST about the points model. Without seeing any updated status on the NE FIRST website regarding the points model, I had mistakenly presumed that the points system simply hasn't been one of the issues being worked by the committee. I'm glad to hear that it has been given a lot of consideration.
I clearly haven't been proactive enough about trying to become involved, but from my perspective I hadn't been aware of the work going on within NE FIRST regarding the district points structure. I attended the NH town hall meeting (which was not a live meeting, but a web event long after the other town hall meetings, just before Christmas 2012) but I don't recall anything being mentioned about the points model at that meeting, other than that it was coming soon. The first details I saw were Jess Boucher's 3 Jan 2013 blog posting, but that was the first and last time I ever saw anything definitive regarding it. Other than unofficial CD threads such as this one (which was started by NY capital district folks outside of NE FIRST), I haven't heard any news at all regarding the NE FIRST district model since kickoff. I've even been checking the NE FIRST web site regularly looking for new information about how to contribute ideas, but haven't seen anything new. It may just be the case that the NH teams have not had a good way to communicate / interact regularly with each other or the NE FIRST committee, and that there is more of a dialogue between teams and NE FIRST in other states?
In any case, thanks for letting me know that the NE FIRST committee has been working hard on the points model. Is there a more recent draft, or is the one on the web site from 3 Jan 2013 still the latest?
vBulletin® v3.6.4, Copyright ©2000-2017, Jelsoft Enterprises Ltd.