Re: 2013 NE FIRST District Rankings
 

I was mistaken. Good catch. So I like Carl's system a lot actually.

I think normalized rankings will help this metric out a lot. I will be getting the spreadsheet out tonight to those who asked for it.

Re: 2013 NE FIRST District Rankings
 

The raw data is available for viewing and download on google docs.

Re: 2013 NE FIRST District Rankings
 

So, I was interested in tinkering around with point values and such, so I created a spreadsheet that has the "raw data" for all the New England teams' performances, including Qualification Match rank/stats, Alliance Selection order, Elim results, and Awards. I did not (yet) do it for any teams interested in joining the NE District as a "conference team." With a list of teams to add, I'd be glad to do so (I saw 20 and 2791 were included in the initially posted spreadsheet from this thread... didn't go through to see who the 10 NY teams were).

Points & Ranking Systems

A ranking system that I recommend, given the results and some time spent tinkering:

Alliance Selection
Cap1 16
Cap2 15
Cap3 14
Cap4 13
Cap5 12
Cap6 11
Cap7 10
Cap8 9
Pic1 16
Pic2 15
Pic3 14
Pic4 13
Pic5 12
Pic6 11
Pic7 10
Pic8 9
Pic9 8
Pic10 7
Pic11 6
Pic12 5
Pic13 4
Pic14 3
Pic15 2
Pic16 1

Elim Results
Winner 0
Finalist 0
SemiFinalist 0
QuarterFinalist 0
Win 4
Tie 2
Loss 0

Qual Results
W 2
T 1
L 0

Awards
Chairmans 25
Creativity 2
Deans List 5
Deans List 5
Eng Excellence 5
Eng Inspiration 15
Entrepreneurship 5
Gracious Prof 2
Imagery 2
Industrial Design 5
Innovation in Control 5
Judges 2
Quality 5
Rookie All Star 15
Rookie Inspiration 5
Rookie Seed 2
Safety 2
Team Spirit 2
Woodie Flowers 5

I assigned points for the alliance selection results because assigning points only for Qualification Results and Elimination Results (uniformly across the alliance) can lead to: teams with a tough/easy qual schedule getting less/more points, teams who were selected onto a weak/strong alliance getting less/more points, and strong teams (229 or 131 at GSR) on Quarterfinalist alliances getting much fewer points than third robots on Winning alliances. I feel very strongly about this being an element to which points are assigned.

I assigned points per number of wins in eliminations (2x qualification values) because it rewards alliances that push a third match (whereas the W=30, F=20, SF=10, QF=5 doesn't). Other than that distinction, either method is pretty reasonable. The pts/number of wins does also have the advantage of being really easy to explain... Even though it may seem I like proposing complexity, I try to only do it when it actually has a noteworthy improvement - otherwise I prefer simple!

For qualifications, I assigned 2pts per win and 1pt per tie. I don't see many people debating this approach... seems like it makes sense. Also has the advantage of just being teams' Qualifying Score!

For awards, I'm suggesting something pretty different from FiM, MAR, or the current NE First proposal. Because of the scalability concern (as you increase districts, more and more spots get filled with auto-bids), I think there should be either no or very few auto-bids. But, you don't need an auto-bid to make it easy for a particular aspect to practically guarantee you a spot at DCMP! Simply assigning a lot of points to various things (i.e. 24 pts for a Regional win (6wins*4pts), 25 pts for Chairmans, 15 for EI, and 10 for Rookie All Star) can practically ensure all those people get to DCMP. Now, with this approach, the Chairmans', EI, and RAS winners from each district really must be allowed to send a judging delegation to the DCMP to compete for the DCMP award (similar to MSC and MAR). Whether you autobid or assign points really only has to do with scalability.

How many points you assign to each element determines how much you want those teams to be at DCMP. You want to guarantee that every DCA (District Chairmans Award) winner goes to DCMP? Assign 100 points. Do you not really care if the DCA winner competes at DCMP any more than the Team Spirit award winner? Assign DCA 2 points. Some might want only the very best robots competing in matches... so they'd assign maybe only 5 pts to DCA, EI, and RAS (they're still eligible for their respective DCMP awards). Others might want these teams to be guaranteed spots at DCMP, so assign more points (25+). Really, I want to have a highly competitive field at DCMP that is every bit as fun, spirited, GP, and inspiring as possible! So, I'd like to see as many of the competitive teams get on the field as possible, amongst DCAs and EIs to keep those teams who model FIRST there, and with the RASs so they can see more of what FIRST is all about.

I think the Chairmans award should be the single biggest point-gathering feature... 25pts isn't unreasonable. With the above points, the DCAs are ranked 2, 3, 4, 5, and 15. If DCA got only 5 points, they would be ranked at 4, 6, 7, 12, and 36. Basically, the RCAs from this year were all very strong teams that only cemented their spots by getting 25pts for their win.

EI should also be awarded "extra" points, given it's role in FIRST. 15pts seems reasonable, and places the 4 New England EI winners (846 from Cali won EI at Boston) at 1, 9, 53, and 70. Keep in mind, the 79th ranked teams have 28pts. These 15pts from EI provided 53.4% of the points needed to qualify for DCMP, if implemented this year (given the 80+ team size currently proposed).

Finally, RAS is awarded 10pts. The 5 NE RAS winners are ranked at 20, 26, 49, 77, and 79. Again, given the 10 pts, RAS winners get 35.7% of the points needed to qualify for DCMP. I do think that it's important for the RASs to get to see DCMP, because qualifying and competing there may really inspire them for their upcoming seasons.

Rankings

Here are the rankings based off the above point system... feel free to double-check your team's points and finishes (Regional Results tab). I'm only putting the top 30 in here, since I don't currently know how to get it to show up as a scroll-able sub-window. Go ahead and download the .xls to see the full spreadsheet.

Rank Team State Pts
1 2648 ME 126.0
2 131 NH 120.0
3 1100 MA 118.0
4 230 CT 116.0
5 126 MA 111.0
6 1519 NH 108.0
7 125 MA 107.3
8 3280 RI 98.0
9 172 ME 88.0
10 3467 NH 86.0
11 155 CT 83.0
12 2168 CT 81.0
13 61 MA 80.0
14 885 VT 78.0
15 175 CT 76.0
15 78 RI 76.0
17 236 CT 74.0
17 3205 MA 74.0
19 177 CT 73.0
20 4564 ME 72.0
20 176 CT 72.0
22 69 MA 70.0
23 228 CT 68.0
23 3780 RI 68.0
25 195 CT 67.0
26 4761 MA 66.0
26 1153 MA 66.0
28 1991 CT 64.0
28 4473 ME 64.0
30 133 ME 62.0

Notes on Spreadsheet & Ranking Calcs

I normalized the rankings for 2 regionals... so teams that competed at 1 got their points doubled, while teams that competed at 3 regionals got their points multiplied by .667.

I assigned points as described above.

I listed most of the accomplishments of the top 64 or so teams on the "Team List" tab, so they can be easily referenced alongside the score. Only 2 teams (one EI, one RAS) that hadn't made elims at all were in these top 64. I believe only 10 of the 154 NE teams won a regional (15 teams won from the 5 NE regionals).

The OPR was taken from Ed Law's spreadsheet after week 7. I just took the single-highest event OPR. For those curious, the excel calculated correlation between my suggested points and that OPR was .784. Assigning all trophies only 2 pts provided .799. Assigning no alliance selection points provided .728. Assigning the 30, 20, 10, 5 for elim finish instead of 4pts/win provided a correlation of .722. Combining these last two provided .667. Seems like this correlation of .784 is pretty reasonable... a correlation of 1.000 would mean we don't value anything but on-field performance and are tied to a flawed a statistic. This approximately .8 correlation indicates a significant - but not identical - relationship.

Feel free to examine the NE rankings (sorting by anything) and the Regional Results (to see who got what at each regional), but make sure that all the tables from Regional Results are sorted by team number so the LOOKUP formulas work as they should. :-)

Thoughts on Qualifying for CMP (not DCMP)

Not sure we should simply have as many slots as 6 per regional (from when regionals were last used). I much prefer the method that FiM just started to implement which sends a number proportional to the number of teams in the district to the number of teams in FIRST. Given the 154 teams in NE, the ~2600 teams in FIRST, and the 400 slots at CMP, that'd mean for 2013, there'd be 24 teams from NE going. 27 are registered for CMP from NE currently.

I also think that DCMP points should be worth more than a single district, given that it is the only time all the top contenders are competing against each other (helps level easy/hard districts) and given that it is the last event. By DCMP, teams are probably going to be pretty close to the level they would be at CMP. I think a weight of between 2 and 3 would make sense.

Anyway, that's all for now... I'm eager to hear what people think of these things, particularly given that a fair number of my suggestions differ from the current NE proposal.

Re: 2013 NE FIRST District Rankings
 

Just a note: FIRST HQ has consistently shot down attempts to give the Chairman's Award, EI, and RAS any point value. The awards are supposed to be "more important than points" or something like that.

Re: 2013 NE FIRST District Rankings
 

Yeah, I'm familiar with that... I just wish that weren't the case. Quite frankly, having RAS and EI be 0 points and not auto-qualify a team for competition at the DCMP (other than for their respective award)... which is the case in FiM and MAR... makes less sense than either "pure" approach (imho). It means that RAS and EI aren't valuable enough to give you points to get to DCMP or CMP. Also, winning Chairmans (at Districts) doesn't give you points to help get to CMP.

I see these as both being factors that make the current system for these awards not line up with the prestige FiM, MAR, and FIRST HQ all assign the awards. I'm guessing this is mostly a weird set of compromises. Having them all be auto-qualify really wouldn't work for scalability, so it seems like an improved point method is the best alternative!

* end rant and stabilize heart rate *

Re: 2013 NE FIRST District Rankings
 

Also of note,
Your point system matter little to who is at the top unless you choose to make an award for that (currently just bragging rights). It matters most at your cut points. Specifically where do you cut your invites off at for the event, and to go to the next level of the event. This is the area to pay the most attention to.

I rather like the way you are awarding Elim points. A successful champion will earn an additional 24 points (in theory ties could add more). This makes a good argument that Chairman's at 24 points is more significant than winning.

It does have a wierd issue though that winning an elim matches is worth 2x many of the judged awards.

Re: 2013 NE FIRST District Rankings
 

Nathan, thanks for compiling all of this data as well as your insightful post!

Re: 2013 NE FIRST District Rankings
 

Does MAR and FIM awards points to teams for individual awards such as the Dean's List and Woodie Flowers? I agree that those are great awards. However it's not the team winning the award but the individual.

Re: 2013 NE FIRST District Rankings
 

You can only win Chairman's once. Did you correct for this when doubling?

Also, even though the conference model won't be adopted, I think including NY teams in the rankings is important. As far as the 2013 season is concerned, all (except 250) are NE teams and can their performances can still be used as a litmus test for how good/bad a ranking system is. The ten NY teams are: 20, 2791, 3044, 4134, 250, 4508, 1493, 1665, 3687, 4254.

Re: 2013 NE FIRST District Rankings
 

MAR and FiM point systems are identical to each other, with the exception of RAS getting 5 pts in MAR. Neither give points for DL or WF...

Whether you should or not is a matter of opinion, obviously... I did intentionally include them in my suggestion though. Good teams are - in a significant part - made better by DL-type students and WF-type mentors. Similarly, DL-type students and WF-type mentors are fostered by good teams. Looking down lists of DL and WF award winners (particularly at the CMP level), you rarely see the names associated with a number you don't recognize (as FIRSTers who know a fair bit about "who's who").

So, does that mean the individual award (which was submitted/authored by someone other than the individual him/herself) should get points for the team? Some say yes, others no. I say yes because we're trying to give points "good" teams. I see a DL or WF award winner as typically being a good distinguisher of a good team (similar to Chairmans, Winner, Seed, etc.).

Note, I'm not saying that... 1) all great teams have DL or WF winners/finalists, 2) anyone who has won DL or WF was on a great team, 3) all great teams submit DL and WF nominations, or 4) great teams consistently win DL or WF

Re: 2013 NE FIRST District Rankings
 

Using Nathan's spreadsheet, I quickly adjusted the point values to the most recent NE proposal based off information I got from last weekend's points meeting.

These points are doubled for those who only attended one regional and multiplied by .667 for those who attended 3. Remember 11 spots of 30 will be given to RCA/EI/RAS, so only the top 19 on points qualify. I included the top 30 and ties because these teams are most likely to compete for points at NE CMP. I do not know the proposed NE points multiplier for the NE CMP.

*Note that 126 and 236 have autobids to CMP and would not take spots from other teams

Re: 2013 NE FIRST District Rankings
 

So that is 10 points for all awards except for RAS, Chairman's and EI? Interesting.

Re: 2013 NE FIRST District Rankings
 

Does the above imply that "last weekend's points meeting" resulted in a consensus to assign no points for "alliance selection" to either the seeded captains or their picks?

I hope that's not the case, as the points for alliance selections are actually a key element in having the points earned by teams make sense despite potentially unbalanced qualification match schedules (easy/hard). The alliance selection points are also the only points to distinguish between the robots of 1st picks vs. 2nd picks, and the only way to distinguish between the robots that make the elimination rounds vs. those that do not. Removing these points from the system takes away all of these key pieces of the point assignment system.

For example, if elimination rounds are all won by the higher-seeded alliances (such as at this year's Pine Tree Regional), then the point assignments to robots do not bear much resemblance to which robots perform the best, particularly for the robots near the "cutoff point." In particular, the 3rd robot on the #1 alliance (16th pick of the draft) comes away with 30 elim points (due to 6 elim wins), while the 5th-seeded captain and 1st pick of the 5th-seeded captain come away with 0 elim points. The same is true for the captains and first picks of the #6 alliance, #7 alliance, and #8 alliance -- they will all get no elimination points at all, meaning they have no bonus over all the other robots that didn't play in eliminations. Meanwhile, the 3rd robot on the #2 alliance (15th draft pick) will acquire at least 20 elim points, and the 3rd robots on the #3 and #4 alliances (14th and 13th picks) will get at least 10 elim points.

Having a scoring system that is most likely to earn the 13th, 14th, 15th, and 16th picks of the draft 10-30 elimination points each, while the 5th, 6th, 7th, 8th captains and their 1st picks (5th, 6th, 7th, and 8th picks of the draft) are most likely to get 0 points from elimination rounds isn't going to do a very good job of correlating more points with the better robots!

I'm not trying to slight any teams that were 13th, 14th, 15th, or 16th teams of the draft, but in general, the 5th, 6th, 7th, and 8th captains and the 5th, 6th, 7th, and 8th picks of the draft will be better robots than the 13th, 14th, 15th, and 16th picks and should generally receive more points in a point assignment system, if that point assignment system intends to send the more capable robots to the division championship.

Re: 2013 NE FIRST District Rankings
 

If you're using the NY teams as a test for this year, you can throw in 229 and 4124 too as they both competed exclusively in NE this season.

Re: 2013 NE FIRST District Rankings
 

Playing Devil's Advocate: if, as you argue, qualification W-L-T is subject to scheduling randomness, then why would you give an additional bonus via Alliance Captain points? There might be an argument for more points given to a high pick rather than a high captain, but that would obviously lead to it's own set of inconsistencies.

Another way to look at the problem is how to devise a simple to understand system that does not change the calculus of how a team plays the game.
Currently a team simply wants to be part of the strongest alliance for eliminations.
With alliance selection points "strongest alliance" has to be balanced with getting points. Concretely would a strong team seeded #6 change it's thinking about accepting/declining if picked by a moderate team seeded #1?

I would argue that sliding alliance selection points do change the calculus of alliance selection, whereas points per elimination win (or for finishing position) keep the game the same.