2013 NE FIRST District Rankings
 

Special thanks to Brian Smist from 229 for doing half the work. As always, this is done by hand, please bring up any corrections.

Points:

Rankings with NY teams from Capital District should Conference Model be adopted:

And rankings without NY teams:

Re: 2013 NE FIRST District Rankings
 

Jack (and Brian), nice work putting this together. I was wondering where we would stand in a district points structure.

Re: 2013 NE FIRST District Rankings
 

You know, there's supposed to be a post by one of the NE folks who created a district points simulator so that you could play with all the values ::coughKylecough::....just haven't seen it yet :)

Re: 2013 NE FIRST District Rankings
 

I'll remind him....:cool:

Re: 2013 NE FIRST District Rankings
 

Just so I can tell if I'm understanding this. The points were calculated as if the regionals in N.E. were district events. They would then determine invitation to the proposed district championship, viz. the top 80, right? NE District championship to be played "this" weekend from which 30 or so teams being invited to CMP in St. Louis. Have I got it right?

Now can we discuss why this points summary may, or may not, be slightly misleading? First, there is the number of events attended by many of the teams. That is, not enough opportunity for the one-event teams to get points for their Q-wins in this summary. A supposed advantage to the district model is that teams will get more matches played as a result. Perhaps the points for single regional event teams could be amplified a bit to reflect this. E.g., the scores from 9 Q-matches at CT regional might get a multiplier of 14/9ths to predict performance in two district events with 7 Q-matches each.

Second, the relative size of regionals to district events would change the dynamics of those competitions somewhat. Not sure about what the effect on points would be, but I'm sure it would be there.

Then there's the even more nebulous effect of the possibility of 8 hours "out of bag" preceding 2-day district events. Is this a factor? At present, teams attending 2-day events get to schedule a sort of "virtual Thursday" by logging time with their robot out of its bag before the competition. Will we allow this in the NE District?

My only questions is does this factor in teams that only attended one even? If so was anything done to factor the fact that they attended one event?

While obviously this is a very rough sketch and many of the metrics in Districts won't apply, I think it's a great way to see how things would play out.

Re: 2013 NE FIRST District Rankings
 

So NE sends 30 teams, and according to the model, 11 teams will be from the champions, chairmans, rookie all-star, etc.

So would that mean the top 22 in this list would be guaranteed to qualify for championships, regardless of wins/awards?

Re: 2013 NE FIRST District Rankings
 

What would be the disappearing regionals?

Re: 2013 NE FIRST District Rankings
 

I believe points are still accrued through to the end of the DCMP. I think in MAR the points are tripled, so its still very much anyones ball game...

-Brando

Re: 2013 NE FIRST District Rankings
 

No, this is just raw data. If anyone wants to normalize for different factors PM me and I can send you the excel sheet.

Re: 2013 NE FIRST District Rankings
 

Right - the results are informative but not a good predictive model of the district.

The rankings above from this year's Regionals primarily reflect whether a team attended 1 or 2+ Regionals, but under the district model every team would be guaranteed 2 events.

The other impacts come from having smaller events, which increase the expected points awarded in several ways:

• increasing the number of qualification matches per team,
• increasing the probability of being selected for eliminations (e.g. 24/34 rather than 24/65 for a big regional),
• increasing the probability of winning an award, and
• arguably diluting the strength of elimination alliances, hence increasing the variance of expected elimination points

Assuming district events average 34 teams each (155 teams attending 2 of 9 events), and a relatively gentle schedule of 12 qualification matches, then the expected point total under the proposed model would be 39.9 for competition performance only. Award points would be additional.

The average from the table above is only 23.2 including awards.

Re: 2013 NE FIRST District Rankings
 

Its great that this conversation is happening. Please continue it, BUT also realize that this is NOT the final point structure. You are giving us good feedback on your thoughts and expectations of what the advancement criteria should look like. Encourage others to take part on this thread. Several of us are monitoring it daily.

The final rounds of meetings are taking place now among the movers and shakers of FIRST in New England to finalize a LOT of things that need finalizing before we go into negotiations with FIRST HQ. And a LOT of the things we are talking about are exctly the things you all brought up in theTown Hall meetings of last fall. So you are having an impact. Keep it up.

Re: 2013 NE FIRST District Rankings
 

If this is true I'd like to add my two cents to the discussion.

First, I'd have preferred to see the 5/2 awards system in place in FiM and MAR. Although the point difference is kind of splitting hairs, I think it matters when a bid to CMP is on the line.

Secondly, I'd like to see the DCMP count more than a district. Frankly, a 60-80 team DCMP is way harder than a 30 team district and teams should be rewarded as such. I think the best points system for CMP qualification would be your best (or two best) district events and 2 x DCMP.

And lastly, I really like the points system for eliminations, however I have two things I'd like to see:

1. I think the points should be bumped to 10/15/25/35 because NE does not award points for selection the same way that FiM and MAR do. This rewards teams for making eliminations more than it did before, but not quite as much
2. I think that the points should also be adjusted to be lower for the third robot on an alliance similarly to how selection points are distributed in FiM. Maybe the first two robots on an alliance get 10/15/25/35 while the third robot gets 5/10/20/30.

Re: 2013 NE FIRST District Rankings
 

I was actually planning on doing this during this week. Thanks for saving me a few days of work!

It seems that only the teams attending two regionals have a legitimate chance of doing really well. There are teams ranked below my own who I know are better than us, but we've attended two regionals, giving us a major boost.

Re: 2013 NE FIRST District Rankings
 

I agree with your idea that the points should be bumped up for eliminations to really help teams that earned their spot in the eliminations. But I disagree with your point on giving the third robot less points. The randomness of qualifications and the possibility of robots breaking sometimes leads to teams qualification score not matching the true power of their robot and therefore they should not be penalized for how they are picked.

Re: 2013 NE FIRST District Rankings
 

I agree with tkell. A team that builds a support-based or defense based robot that is still phenomenal in what it is built to do (i.e. 4334 last year and 2789 this year) shouldn't be penalized for pursuing a different strategy than the powerhouse offensive teams. Third picks often make or break alliances, especially at deeper events.

Re: 2013 NE FIRST District Rankings
 

I agree with the logic on making the third pick worth just as many points. The serpentine in particular makes this problematic - I would hate to be the 8th seed and have to pull up on my phone which team "needs" the 1st round pick points more and which team doesn't.

Re: 2013 NE FIRST District Rankings
 

Thanks! It's so hard to get love for defensive play...because of our limitations (personnel, funding, etc.) and sponsor issues, it's hard for us to crank out the robots we design and want. We're forced to compensate with scouting and strategy. I've been concerned about district discussions down here in Texas because of how formulas rate defensive teams, in particular because of how the dynamics for eliminations matches are very different than dynamics for quals matches, and our strategy definitely makes a bigger impact in elims. It's my hope that at the end of the month we'll be able to make the point that a creative and smart team can still make a strong impact on the outcome of matches even if you don't have the fanciest robot on the field...but needless to say, we are working on a few surprises for champs that will help us put points on the board :)

Re: 2013 NE FIRST District Rankings
 

I definitely do agree that the points for the 3rd robot should be adjusted... Primarily because that means that the 3rd robot on the winning alliance gets more points than any other alliance's robots (including the first two robots on all of the other alliances). Additionally, this method of assigning points gives very few points to the quarterfinalist alliances.

Although I suspect these points were removed in NE's proposal because it makes things a bit more complicated, I think assigning points based on alliance selection order as FiM and MAR do (16 to first 2 bots on Alliance 1; 15 to first 2 bots on Alliance 2; 14 to first 2 bots on Alliance 3... and 8 points to 3rd robot on Alliance 8, 7 points to 3rd robot on Alliance 7, 6 points to 3rd robot on Alliance 6) is the best way to assign points for eliminations (in addition to points based on finish).

Using 2013 GSR and 2013 Pine Tree as case studies (I chose these two because they're Week 1 vs Week 6, vary significantly in size, and Pine Tree is interesting because the red alliance won each matchup):

GSR Pick Order & Results:

610-4124-3609... W
138-131-58... QF
230-1991-1153... SF
885-1519-133... SF
151-229-1277... QF
1512-1922-1517... QF
61-175-172... F
2791-3467-78... QF

GSR Points (based on current NE proposal)

30-30-30... W
5-5-5... QF
10-10-10... SF
10-10-10... SF
5-5-5... QF
5-5-5... QF
20-20-20... F
5-5-5... QF

GSR Points (NE proposal + alliance selection points)

46-46-31... W
20-20-7... QF
24-24-13... SF
23-23-14... SF
17-17-10... QF
16-16-11... QF
30-30-27... F
14-14-13... QF

Pine Tree Pick Order & Results:

2648-3467-2386... W
176-125-63... F
1153-172-1831... SF
69-133-4564... SF
4473-58-1058... QF
78-1073-1922... QF
3930-4055-157... QF
3609-1071-181... QF

Pine Tree Points (based on current NE proposal)

30-30-30... W
20-20-20... F
10-10-10... SF
10-10-10... SF
5-5-5... QF
5-5-5... QF
5-5-5... QF
5-5-5... QF

Pine Tree Points (NE proposal + alliance selection points)

46-46-31... W
35-35-22... F
24-24-13... SF
23-23-14... SF
17-17-10... QF
16-16-11... QF
15-15-12... QF
14-14-13... QF

Seems like the current NE Proposal has several weaknesses:
- 1st and 2nd robots of each alliance get same reward as 3rd robot.
- Winners get 6x the points that the quarterfinalists get (3x the semifinalists).
- 1st and 2nd robots of finalist alliance (theoretically 3rd and 4th best teams) get 66% the points of the 3rd robot of the winning alliance (theoretically lower than 20th in ranking of teams).

These particular issues are improved with the inclusion of the alliance selection points. It'd be interesting to also add in the win-loss information... but I don't really have the time for that right now.

Re: 2013 NE FIRST District Rankings
 

The one thing I dislike about the FiM system assigning points based on Alliance Selection is that it gives points for essentially the same thing QF gives points for. The thing I do like is it is a good way to breakdown credit on an alliance.

A proposal I would support would be 8 ranking points for AC1 and First pick, and decreasing from there. This would give 3rd robots 0 extra points compared to the pack, but the difference between the last robot and the first is only 8 rather than 6. Then the NE Eliminations points could be bumped to 10/15/25/35 to make up for the point loss and emphasize results more.

Alliance Points:

8-8-0
7-7-0
6-6-0
5-5-0
4-4-0
3-3-0
2-2-0
1-1-0

Using BAE as an example: (10/15/25/35 + Alliance Points)

43-43-35... W
17-17-10... QF
21-21-15... SF
20-20-15... SF
14-14-10... QF
13-13-10... QF
27-27-25... F
11-11-10... QF

Re: 2013 NE FIRST District Rankings
 

While this is sometimes the case, awarding points based on Alliance Selection can also serve to balance out the variation in teams' qualification schedules. If a team has a great robot, but happens to lose a few qualification matches to tough opponents, being picked high can be a chance for them to regain some of those points.

One thing I keep coming back to with the proposed system for elimination points is the situation where the two strongest alliances at an event happen to face off before the finals. The bracket-style tournament is very effective at determining the best alliance, but it doesn't work as well when it comes to ranking the remaining seven alliances.

For example, let's say the higher seeded alliances all win their quarterfinals. Alliance #3 squeaks out a semifinal win over Alliance #2 in three close matches, but then goes on to win two very one-sided finals matches against Alliance #1. Based on this, it would be reasonable to say Alliance #2 is stronger than Alliance #1. Under the proposed system, Alliance #1 would receive twice as many points as Alliance #2 (who all evidence would suggest is the stronger alliance) based purely on the structure of the bracket.

I can't think of a good way to deal with this situation, short of a complex system that takes "strength of schedule" into account, or a different structure for elimination rounds (neither of which I think would be the right answer). But I do think that it's an important thing to keep in mind when creating a system like this.

Re: 2013 NE FIRST District Rankings
 

Has anyone run the numbers with the FiM or MAR points system so that its easy to see the difference between the current NE proposal and the FiM or MAR models?

Re: 2013 NE FIRST District Rankings
 

To add to the dicussion I did a quick World OPR Rank VLOOKUP for the NE Teams (Thanks to Ed & Ether's data)

This is by no means enough for ranking into DCMP or WCMP, it just gives a screenshot of offensively which teams should maybe make it into the top NE slots... and is just meant for comparison (ie if there is a team in 10th OPR that doesnt make it into the top 50 of our ranking system, its worth looking at to make sure the NE Ranking structure is a good balance).

I do believe some of this will balance out by normalizing for a single event...

Note: the numbers here are the OPR Ranks for World OPR Ranking, NE OPR Ranking and Jack/Brian's NE Ranks sorted by OPR

EDIT - HUGE Apologies - I forgot Rhode Island!! Added now!

Re: 2013 NE FIRST District Rankings
 

How about awarding teams something like 4 or 5 pts. for eliminations victories in addition to selection points? This would not solve the problem in your example, but it would bring the results closer to reality:
Alliance 2 has 6 wins in elims totaling 6*5 = 30 pts.
Alliance 1 has 4 wins in elims (2 from QF, 2 from SF, 0 from F) totaling 4*5 = 20 pts.
Alliance 3 has 3 wins in elims (2 from QF, 1 from SF) totaling 3*5 = 15 pts.

This solves the issue Jack was referring to as well. If you include picking points with this, then you are not rewarding teams doubly for being picked and being quarter finalists. These alliances would only be granted additional points if they won a match in the quarter finals. Notably, this would also provide a good way to reward back up robots for their contributions to an alliance in elims. They would be rewarded only for the matches where they helped the alliance win.

Re: 2013 NE FIRST District Rankings
 

Not to change the subject, but how about points for awards? How many should be given for Chairmans? how many for EI? Safety? Entrepreneurship? Quality? Engineering Excellence?

Re: 2013 NE FIRST District Rankings
 

it would also be nice to see points awarded to highest rookie seed.

Re: 2013 NE FIRST District Rankings
 

From other threads it seems like FIRST is pretty adamant about Chairman's being an automatic qualifying award, so I won't touch that. I think there needs to be two tiers of awards at least, because I don't think something like the team spirit award should be in any way equivalent to say the Quality Award. I think things like Quality, Xerox Creativity, maybe rookie inspiration etc. should be one amount, with things like Spirit, Gracious professionalism, safety worth some lower amount of points. A third, higher tier would be EI and/or RAS if they end up not fully qualifying a team for the NE championship. I think these two should be worth something like what the finalist teams get.

Re: 2013 NE FIRST District Rankings
 

This seems fairer in that it takes into account the "closeness" of a round in elims by giving partial credit to an alliance that forces a third match, but without penalizing the winning alliance.

Kim's data suggests that there is a rank correlation of about 0.69 between World OPR and NE proposal points, which is worse than I expected.
But since OPR is far from a perfect ranking measure, and we know there are problems with projecting NE district points from non-normalized Regional results, it's probably not a particularly useful measurement anyway!

Re: 2013 NE FIRST District Rankings
 

I really like how this system grants points for winning one of three QF/SF/F matches. What I dislike is that alliances that are 0-2 in QF don't receive any extra points for making elims. This also doesn't address the issue of all three robots receiving the same number of points. (Although people seem split 50/50 on that) If every team received a base of 5 points for making elims that would help remedy the first problem. I'm open to suggestions on fixing the second one.

I think that FiM was very clear in its decision to grant RCA, EI, and RAS autobids to DCMP. Assigning those awards points deflates their value and prestige. Unfortunately for rookies, it becomes difficult to assign points to the other two awards when RAS garners no points. I could see myself supporting autobid and 5 for RAS and 2 for the other two awards (I think MAR does this.)

I do like the 5/2 robot/non robot awards split in FiM.

Re: 2013 NE FIRST District Rankings
 

I like the idea of giving alliances credit for forcing a third match.

I think the implication was that points would be awarded for alliance selection, and they would have the points for a QF finish baked into them.

Re: 2013 NE FIRST District Rankings
 

I was mistaken. Good catch. So I like Carl's system a lot actually.

I think normalized rankings will help this metric out a lot. I will be getting the spreadsheet out tonight to those who asked for it.

Re: 2013 NE FIRST District Rankings
 

The raw data is available for viewing and download on google docs.

Re: 2013 NE FIRST District Rankings
 

So, I was interested in tinkering around with point values and such, so I created a spreadsheet that has the "raw data" for all the New England teams' performances, including Qualification Match rank/stats, Alliance Selection order, Elim results, and Awards. I did not (yet) do it for any teams interested in joining the NE District as a "conference team." With a list of teams to add, I'd be glad to do so (I saw 20 and 2791 were included in the initially posted spreadsheet from this thread... didn't go through to see who the 10 NY teams were).

Points & Ranking Systems

A ranking system that I recommend, given the results and some time spent tinkering:

Alliance Selection
Cap1 16
Cap2 15
Cap3 14
Cap4 13
Cap5 12
Cap6 11
Cap7 10
Cap8 9
Pic1 16
Pic2 15
Pic3 14
Pic4 13
Pic5 12
Pic6 11
Pic7 10
Pic8 9
Pic9 8
Pic10 7
Pic11 6
Pic12 5
Pic13 4
Pic14 3
Pic15 2
Pic16 1

Elim Results
Winner 0
Finalist 0
SemiFinalist 0
QuarterFinalist 0
Win 4
Tie 2
Loss 0

Qual Results
W 2
T 1
L 0

Awards
Chairmans 25
Creativity 2
Deans List 5
Deans List 5
Eng Excellence 5
Eng Inspiration 15
Entrepreneurship 5
Gracious Prof 2
Imagery 2
Industrial Design 5
Innovation in Control 5
Judges 2
Quality 5
Rookie All Star 15
Rookie Inspiration 5
Rookie Seed 2
Safety 2
Team Spirit 2
Woodie Flowers 5

I assigned points for the alliance selection results because assigning points only for Qualification Results and Elimination Results (uniformly across the alliance) can lead to: teams with a tough/easy qual schedule getting less/more points, teams who were selected onto a weak/strong alliance getting less/more points, and strong teams (229 or 131 at GSR) on Quarterfinalist alliances getting much fewer points than third robots on Winning alliances. I feel very strongly about this being an element to which points are assigned.

I assigned points per number of wins in eliminations (2x qualification values) because it rewards alliances that push a third match (whereas the W=30, F=20, SF=10, QF=5 doesn't). Other than that distinction, either method is pretty reasonable. The pts/number of wins does also have the advantage of being really easy to explain... Even though it may seem I like proposing complexity, I try to only do it when it actually has a noteworthy improvement - otherwise I prefer simple!

For qualifications, I assigned 2pts per win and 1pt per tie. I don't see many people debating this approach... seems like it makes sense. Also has the advantage of just being teams' Qualifying Score!

For awards, I'm suggesting something pretty different from FiM, MAR, or the current NE First proposal. Because of the scalability concern (as you increase districts, more and more spots get filled with auto-bids), I think there should be either no or very few auto-bids. But, you don't need an auto-bid to make it easy for a particular aspect to practically guarantee you a spot at DCMP! Simply assigning a lot of points to various things (i.e. 24 pts for a Regional win (6wins*4pts), 25 pts for Chairmans, 15 for EI, and 10 for Rookie All Star) can practically ensure all those people get to DCMP. Now, with this approach, the Chairmans', EI, and RAS winners from each district really must be allowed to send a judging delegation to the DCMP to compete for the DCMP award (similar to MSC and MAR). Whether you autobid or assign points really only has to do with scalability.

How many points you assign to each element determines how much you want those teams to be at DCMP. You want to guarantee that every DCA (District Chairmans Award) winner goes to DCMP? Assign 100 points. Do you not really care if the DCA winner competes at DCMP any more than the Team Spirit award winner? Assign DCA 2 points. Some might want only the very best robots competing in matches... so they'd assign maybe only 5 pts to DCA, EI, and RAS (they're still eligible for their respective DCMP awards). Others might want these teams to be guaranteed spots at DCMP, so assign more points (25+). Really, I want to have a highly competitive field at DCMP that is every bit as fun, spirited, GP, and inspiring as possible! So, I'd like to see as many of the competitive teams get on the field as possible, amongst DCAs and EIs to keep those teams who model FIRST there, and with the RASs so they can see more of what FIRST is all about.

I think the Chairmans award should be the single biggest point-gathering feature... 25pts isn't unreasonable. With the above points, the DCAs are ranked 2, 3, 4, 5, and 15. If DCA got only 5 points, they would be ranked at 4, 6, 7, 12, and 36. Basically, the RCAs from this year were all very strong teams that only cemented their spots by getting 25pts for their win.

EI should also be awarded "extra" points, given it's role in FIRST. 15pts seems reasonable, and places the 4 New England EI winners (846 from Cali won EI at Boston) at 1, 9, 53, and 70. Keep in mind, the 79th ranked teams have 28pts. These 15pts from EI provided 53.4% of the points needed to qualify for DCMP, if implemented this year (given the 80+ team size currently proposed).

Finally, RAS is awarded 10pts. The 5 NE RAS winners are ranked at 20, 26, 49, 77, and 79. Again, given the 10 pts, RAS winners get 35.7% of the points needed to qualify for DCMP. I do think that it's important for the RASs to get to see DCMP, because qualifying and competing there may really inspire them for their upcoming seasons.

Rankings

Here are the rankings based off the above point system... feel free to double-check your team's points and finishes (Regional Results tab). I'm only putting the top 30 in here, since I don't currently know how to get it to show up as a scroll-able sub-window. Go ahead and download the .xls to see the full spreadsheet.

Rank Team State Pts
1 2648 ME 126.0
2 131 NH 120.0
3 1100 MA 118.0
4 230 CT 116.0
5 126 MA 111.0
6 1519 NH 108.0
7 125 MA 107.3
8 3280 RI 98.0
9 172 ME 88.0
10 3467 NH 86.0
11 155 CT 83.0
12 2168 CT 81.0
13 61 MA 80.0
14 885 VT 78.0
15 175 CT 76.0
15 78 RI 76.0
17 236 CT 74.0
17 3205 MA 74.0
19 177 CT 73.0
20 4564 ME 72.0
20 176 CT 72.0
22 69 MA 70.0
23 228 CT 68.0
23 3780 RI 68.0
25 195 CT 67.0
26 4761 MA 66.0
26 1153 MA 66.0
28 1991 CT 64.0
28 4473 ME 64.0
30 133 ME 62.0

Notes on Spreadsheet & Ranking Calcs

I normalized the rankings for 2 regionals... so teams that competed at 1 got their points doubled, while teams that competed at 3 regionals got their points multiplied by .667.

I assigned points as described above.

I listed most of the accomplishments of the top 64 or so teams on the "Team List" tab, so they can be easily referenced alongside the score. Only 2 teams (one EI, one RAS) that hadn't made elims at all were in these top 64. I believe only 10 of the 154 NE teams won a regional (15 teams won from the 5 NE regionals).

The OPR was taken from Ed Law's spreadsheet after week 7. I just took the single-highest event OPR. For those curious, the excel calculated correlation between my suggested points and that OPR was .784. Assigning all trophies only 2 pts provided .799. Assigning no alliance selection points provided .728. Assigning the 30, 20, 10, 5 for elim finish instead of 4pts/win provided a correlation of .722. Combining these last two provided .667. Seems like this correlation of .784 is pretty reasonable... a correlation of 1.000 would mean we don't value anything but on-field performance and are tied to a flawed a statistic. This approximately .8 correlation indicates a significant - but not identical - relationship.

Feel free to examine the NE rankings (sorting by anything) and the Regional Results (to see who got what at each regional), but make sure that all the tables from Regional Results are sorted by team number so the LOOKUP formulas work as they should. :-)

Thoughts on Qualifying for CMP (not DCMP)

Not sure we should simply have as many slots as 6 per regional (from when regionals were last used). I much prefer the method that FiM just started to implement which sends a number proportional to the number of teams in the district to the number of teams in FIRST. Given the 154 teams in NE, the ~2600 teams in FIRST, and the 400 slots at CMP, that'd mean for 2013, there'd be 24 teams from NE going. 27 are registered for CMP from NE currently.

I also think that DCMP points should be worth more than a single district, given that it is the only time all the top contenders are competing against each other (helps level easy/hard districts) and given that it is the last event. By DCMP, teams are probably going to be pretty close to the level they would be at CMP. I think a weight of between 2 and 3 would make sense.

Anyway, that's all for now... I'm eager to hear what people think of these things, particularly given that a fair number of my suggestions differ from the current NE proposal.

Re: 2013 NE FIRST District Rankings
 

Just a note: FIRST HQ has consistently shot down attempts to give the Chairman's Award, EI, and RAS any point value. The awards are supposed to be "more important than points" or something like that.

Re: 2013 NE FIRST District Rankings
 

Yeah, I'm familiar with that... I just wish that weren't the case. Quite frankly, having RAS and EI be 0 points and not auto-qualify a team for competition at the DCMP (other than for their respective award)... which is the case in FiM and MAR... makes less sense than either "pure" approach (imho). It means that RAS and EI aren't valuable enough to give you points to get to DCMP or CMP. Also, winning Chairmans (at Districts) doesn't give you points to help get to CMP.

I see these as both being factors that make the current system for these awards not line up with the prestige FiM, MAR, and FIRST HQ all assign the awards. I'm guessing this is mostly a weird set of compromises. Having them all be auto-qualify really wouldn't work for scalability, so it seems like an improved point method is the best alternative!

* end rant and stabilize heart rate *

Re: 2013 NE FIRST District Rankings
 

Also of note,
Your point system matter little to who is at the top unless you choose to make an award for that (currently just bragging rights). It matters most at your cut points. Specifically where do you cut your invites off at for the event, and to go to the next level of the event. This is the area to pay the most attention to.

I rather like the way you are awarding Elim points. A successful champion will earn an additional 24 points (in theory ties could add more). This makes a good argument that Chairman's at 24 points is more significant than winning.

It does have a wierd issue though that winning an elim matches is worth 2x many of the judged awards.

Re: 2013 NE FIRST District Rankings
 

Nathan, thanks for compiling all of this data as well as your insightful post!

Re: 2013 NE FIRST District Rankings
 

Does MAR and FIM awards points to teams for individual awards such as the Dean's List and Woodie Flowers? I agree that those are great awards. However it's not the team winning the award but the individual.

Re: 2013 NE FIRST District Rankings
 

You can only win Chairman's once. Did you correct for this when doubling?

Also, even though the conference model won't be adopted, I think including NY teams in the rankings is important. As far as the 2013 season is concerned, all (except 250) are NE teams and can their performances can still be used as a litmus test for how good/bad a ranking system is. The ten NY teams are: 20, 2791, 3044, 4134, 250, 4508, 1493, 1665, 3687, 4254.

Re: 2013 NE FIRST District Rankings
 

MAR and FiM point systems are identical to each other, with the exception of RAS getting 5 pts in MAR. Neither give points for DL or WF...

Whether you should or not is a matter of opinion, obviously... I did intentionally include them in my suggestion though. Good teams are - in a significant part - made better by DL-type students and WF-type mentors. Similarly, DL-type students and WF-type mentors are fostered by good teams. Looking down lists of DL and WF award winners (particularly at the CMP level), you rarely see the names associated with a number you don't recognize (as FIRSTers who know a fair bit about "who's who").

So, does that mean the individual award (which was submitted/authored by someone other than the individual him/herself) should get points for the team? Some say yes, others no. I say yes because we're trying to give points "good" teams. I see a DL or WF award winner as typically being a good distinguisher of a good team (similar to Chairmans, Winner, Seed, etc.).

Note, I'm not saying that... 1) all great teams have DL or WF winners/finalists, 2) anyone who has won DL or WF was on a great team, 3) all great teams submit DL and WF nominations, or 4) great teams consistently win DL or WF

Re: 2013 NE FIRST District Rankings
 

Using Nathan's spreadsheet, I quickly adjusted the point values to the most recent NE proposal based off information I got from last weekend's points meeting.

These points are doubled for those who only attended one regional and multiplied by .667 for those who attended 3. Remember 11 spots of 30 will be given to RCA/EI/RAS, so only the top 19 on points qualify. I included the top 30 and ties because these teams are most likely to compete for points at NE CMP. I do not know the proposed NE points multiplier for the NE CMP.

*Note that 126 and 236 have autobids to CMP and would not take spots from other teams

Re: 2013 NE FIRST District Rankings
 

So that is 10 points for all awards except for RAS, Chairman's and EI? Interesting.

Re: 2013 NE FIRST District Rankings
 

Does the above imply that "last weekend's points meeting" resulted in a consensus to assign no points for "alliance selection" to either the seeded captains or their picks?

I hope that's not the case, as the points for alliance selections are actually a key element in having the points earned by teams make sense despite potentially unbalanced qualification match schedules (easy/hard). The alliance selection points are also the only points to distinguish between the robots of 1st picks vs. 2nd picks, and the only way to distinguish between the robots that make the elimination rounds vs. those that do not. Removing these points from the system takes away all of these key pieces of the point assignment system.

For example, if elimination rounds are all won by the higher-seeded alliances (such as at this year's Pine Tree Regional), then the point assignments to robots do not bear much resemblance to which robots perform the best, particularly for the robots near the "cutoff point." In particular, the 3rd robot on the #1 alliance (16th pick of the draft) comes away with 30 elim points (due to 6 elim wins), while the 5th-seeded captain and 1st pick of the 5th-seeded captain come away with 0 elim points. The same is true for the captains and first picks of the #6 alliance, #7 alliance, and #8 alliance -- they will all get no elimination points at all, meaning they have no bonus over all the other robots that didn't play in eliminations. Meanwhile, the 3rd robot on the #2 alliance (15th draft pick) will acquire at least 20 elim points, and the 3rd robots on the #3 and #4 alliances (14th and 13th picks) will get at least 10 elim points.

Having a scoring system that is most likely to earn the 13th, 14th, 15th, and 16th picks of the draft 10-30 elimination points each, while the 5th, 6th, 7th, 8th captains and their 1st picks (5th, 6th, 7th, and 8th picks of the draft) are most likely to get 0 points from elimination rounds isn't going to do a very good job of correlating more points with the better robots!

I'm not trying to slight any teams that were 13th, 14th, 15th, or 16th teams of the draft, but in general, the 5th, 6th, 7th, and 8th captains and the 5th, 6th, 7th, and 8th picks of the draft will be better robots than the 13th, 14th, 15th, and 16th picks and should generally receive more points in a point assignment system, if that point assignment system intends to send the more capable robots to the division championship.

Re: 2013 NE FIRST District Rankings
 

If you're using the NY teams as a test for this year, you can throw in 229 and 4124 too as they both competed exclusively in NE this season.

Re: 2013 NE FIRST District Rankings
 

Playing Devil's Advocate: if, as you argue, qualification W-L-T is subject to scheduling randomness, then why would you give an additional bonus via Alliance Captain points? There might be an argument for more points given to a high pick rather than a high captain, but that would obviously lead to it's own set of inconsistencies.

Another way to look at the problem is how to devise a simple to understand system that does not change the calculus of how a team plays the game.
Currently a team simply wants to be part of the strongest alliance for eliminations.
With alliance selection points "strongest alliance" has to be balanced with getting points. Concretely would a strong team seeded #6 change it's thinking about accepting/declining if picked by a moderate team seeded #1?

I would argue that sliding alliance selection points do change the calculus of alliance selection, whereas points per elimination win (or for finishing position) keep the game the same.

Re: 2013 NE FIRST District Rankings
 

Re: 2013 NE FIRST District Rankings
 

I actually think that in district events which have relatively small numbers of teams compared to the number of matches played (e.g. in FiM districts, with no more than 40 teams in an event, and with at least 12 matches played), the qualification W-L-T is less subject to scheduling randomness and the ranking system will do a better job sorting teams. As we see in single-day off-season events, when less qualification matches are played, seed rankings get more "noise" in them and are less well correlated to the best robots.

Have FiM teams generally found that FiM district events result in seeding rankings that are pretty good - ie, there aren't many high-seeding robots that seem out of place at FiM district events?

However, even when there is some scheduling randomness leading to a mis-sorted ranking, I think the benefit of giving bonus points to well-picked robots outweighs the harm of giving bonus points to the "lucky" captain who seeded higher than they deserved to be. There will be both more robots picked than captains (16 vs 8) and the "diamond in the rough" picks will typically gain more points relative to their pre-pick points than the "lucky" captains will.

I agree that having the "lucky" captains get alliance selection points is less than optimal, but I think this is a case where we need to go with the lesser of two evils in order to avoid throwing the baby out with the bathwater.

I agree that I don't like the thought that teams would change their decisions of who to pick, or who to accept, based upon the point system. I would prefer that teams simply make picking / acceptance decisions based upon the event, rather than the points system. However, as long as district points are gained in eliminations, potential in-tournament point considerations will always be a factor.

Even if no "alliance selection" points are issued, but points are assigned for winning matches in eliminations, we have this problem. For example, if there are no alliance selection points but getting eliminated in the finals is worth more points than getting eliminated in the semifinals (i.e. points for elimination win, as you mention), in a tournament with an emerging very strong #1 alliance (think 1114/2056), a #6 captain selected by a #4 or #5 seed has a strong motivation to decline and instead captain their own #6 alliance. The #4/#5 alliance would have very little chance to get to the finals, but the #6 alliance would have a reasonable shot, since they wouldn't face #1 in the semifinals. (By the way, I think the above scenario may have actually played out this year in at least one regional, due to the "wild card" assignment -- getting eliminated in the finals could earn a ticket to CMP at late regionals, while getting eliminated in the semifinals resulted in an ended season.)

Actually, I think with a relatively small (30-40 team) district event and many (12+) matches, the scenario you describe (of a weak #1 seed who might want to be declined by a #6 captain) is not very likely due to better sorting of teams. When it does happen, I think it would be a relatively rare event. However, I think the strong #1 alliance situation I mention above, where a #4 or #5 captain might select a #6 or #7 seed, is more likely to occur (just ask a Toronto-area team.) Then again, here in New England, we benefit from the fact that we don't have any real powerhouse teams that are way above the pack, so it might be less of an issue for us here than in some other regions.

Re: 2013 NE FIRST District Rankings
 

Somewhat new theme - Philosophy of NE FIRST Points System.

I'm not entirely sure whether to start a new CD thread for this topic or continue the discussion on the NE FIRST Points System in this existing thread. I've opted to keep it in this thread, presuming that those who are interested in the NE FIRST Points System are already here...

I realize that for the NE FIRST folks, determining the points system is really one of the minor details with regard to switching to the district system. Nearly all of the work is in things like planning district event locations, times and venues; getting plans in place for sponsors and volunteers for events; figuring out a venue and organization for the NE championship, etc.

Figuring out which teams attend the NE CMP is really practically irrelevant from a perspective of switching over to the district model for the NE FIRST organizers -- regardless of which teams attend, there's an incredible amount to work through for the NE FIRST committee! However, for the teams, it is probably one of the top 5 issues of interest.

As has been mentioned previously in this thread by IKE, the real thing to look at in the rankings is the teams near the cutoff. The "obvious" teams will make the cut with nearly any points system. However, the real question is whether the teams that barely make the cut generally make sense as opposed to ones that fall just below the cut? Alas, this is actually kind of hard to evaluate without attending most of the regional events, as familiarity with which teams are more/less deserving is hard to evaluate for teams at this 75th percentile level.

When trying to figure out potential ways to assign teams points for qualification to NE CMP, I realized that it is essential to consider the philosophy of "Who Qualifies for the NE CMP."

In order to evaluate / consider different approaches for tallying up district points, it is necessary to first consider what is desired with respect to "Who Qualifies for the NE CMP?" Without knowing the goal of the "points ranking system" it is hard (impossible) to evaluate any given ranking system against that goal.

Clearly, the tautological objective of the proposed NE FIRST "points ranking system" is to determine which teams qualify for the NE CMP. But, what types of teams does NE FIRST want to qualify? Let's list a few of the potential objectives / types of teams that I have heard mentioned:
* The best robots as determined by demonstrated performance on the field
* The best teams at demonstrating the principles / ideals of FIRST
* The best robots as assessed by knowledgeable FRCers, even if they encountered bad luck in demonstrated performance
* The best robots as determined by judged awards
* The teams that would most benefit from the inspiration derived from attending the district championship
* The teams that haven't been to the district championship recently

From my understanding as an outside observer, the FiM points ranking system is specifically intended to put the best robots onto the playing field at the Michigan State Championship. This is no secret, as it is publicly stated all over the place, and is a regular point made about the FiM Championship - that it has the highest average scores of any event in the world, exceeding even the FIRST World Championship in average caliber of robot capabilities. In accordance with this, the FiM points ranking system provides more points to achievements which are highly correlated with the best robots, and less points to achievements that have less to do with the best robots. (The FIRST World Championship has more excellent robots in attendance, since it draws on robots from around the world, but the average is lowered by including not only the best robots, but also teams that have won the Chairman's Award, Engineering Inspiration Award, or Rookie All-Star award at regional events.)

The FiM leadership has intentionally assigned the point system the way it is to try to put the best robots onto the playing field at the Michigan Championship. As a result, even though FiM district winners of the district chairman's award, district engineering inspiration award, and district rookie-all-star awards get to participate in the Michigan Championship judging for those awards, only the district chairman's award winners automatically qualify to compete with their robot. District engineering inspiration awards and district rookie-all-star awards do not qualify for robot positions at the Michigan Championship, but instead qualify the team to send students to present to the Michigan Championship judges for those awards at the state level. Only district winners of those awards compete for those awards at the Michigan Championship. The Michigan State Championship winners for Chairman's Award (3 selected winners), Engineering Inspiration (1 winner) and Rookie All Star (1 winner) all qualify for the FIRST World Championship. (Such participation includes their robots at the World Championship.)

If NE FIRST's objective for the NE Championship is to have the best robots on the playing field at the NE Championship, then careful consideration should be given to simply adopt the FiM/MAR points system. I can only presume that the FiM folks have each of the different kind of points in their ranking system for a reason. Personally, I think there are good reasons for including the alliance selection points and different amount of elimination points for captains and 1st-picks, vs 2nd-picks. That said, NE FIRST may have a different perspective/objective than that of FiM, and intentionally want to do something other than seek to put the best robots on the playing field at the NE Championship. If so, that would be a good reason to do point assignments differently. Alternatively, even if the goal at NE FIRST is to put the best robots on the field, there may be other ways to determine the best robots that FiM either didn't explore, or couldn't get sufficient backing to implement. Personally, I think NE FIRST should also consider such alternate point scoring systems, but should also realize that the FiM/MAR system has been field tested quite a bit!

Re: 2013 NE FIRST District Rankings
 

I just wanted to note that while there are many valid points in your discussion, this section is rather untrue. If you look at the "status" page of the NE FIRST website, you will have a glimpse of how much work went into putting together a points model for NE FIRST. Almost immediately they split into operations and organization committees and began work on both parts of the system. They actually spent a lot more time working on and discussing this in the past year and a half than they did on locations. They realized this was incredibly important to teams and not only carefully researched and discussed all the models, they came up with a model that at times has been vastly different from FiM and MAR. As they discussed with teams at town halls, the model would change from month to month as they developed a picture of what the teams wanted. Even more recently, given a lot of these CD threads and opinions they have gathered from teams, they have changed the model to match what they are hearing from teams.

Are there still issues with the current proposal? Well they will never be able to make everyone happy. But they have NOT just blindly adopted the FiM & MAR models as you can see from several of the rankings threads. There are plenty of ways to argue which is better, and to be honest, as you suggest - it is all about what Philosophy NE FIRST & its teams wants to take with regards to its model. Are they in it to send the best robots? Do they want to generate more Chairmans winners? Do they want to give everyone a chance to play? Do they want to promote growth and sustainability? Each of these can lend itself to a slightly different model.

But I wanted to at least set the record straight that the NE FIRST committee has spent quite a lot of time going over and revising the points model, and it is far from an afterthought or a minor detail.

Re: 2013 NE FIRST District Rankings
 

I'm sorry; I had not intended to offend or imply that the NE FIRST committee thought the points model was an afterthought. I definitely didn't think that the NE FIRST committee thought it was unimportant, but simply thought it had not yet been an issue they have had the bandwidth to address.

With your post, I am clearly mistaken. I have not been aware of any of the ongoing discussion within NE FIRST about the points model. Without seeing any updated status on the NE FIRST website regarding the points model, I had mistakenly presumed that the points system simply hasn't been one of the issues being worked by the committee. I'm glad to hear that it has been given a lot of consideration.

I clearly haven't been proactive enough about trying to become involved, but from my perspective I hadn't been aware of the work going on within NE FIRST regarding the district points structure. I attended the NH town hall meeting (which was not a live meeting, but a web event long after the other town hall meetings, just before Christmas 2012) but I don't recall anything being mentioned about the points model at that meeting, other than that it was coming soon. The first details I saw were Jess Boucher's 3 Jan 2013 blog posting, but that was the first and last time I ever saw anything definitive regarding it. Other than unofficial CD threads such as this one (which was started by NY capital district folks outside of NE FIRST), I haven't heard any news at all regarding the NE FIRST district model since kickoff. I've even been checking the NE FIRST web site regularly looking for new information about how to contribute ideas, but haven't seen anything new. It may just be the case that the NH teams have not had a good way to communicate / interact regularly with each other or the NE FIRST committee, and that there is more of a dialogue between teams and NE FIRST in other states?

In any case, thanks for letting me know that the NE FIRST committee has been working hard on the points model. Is there a more recent draft, or is the one on the web site from 3 Jan 2013 still the latest?