Re: 2013 NE FIRST District Rankings
 

Re: 2013 NE FIRST District Rankings
 

I actually think that in district events which have relatively small numbers of teams compared to the number of matches played (e.g. in FiM districts, with no more than 40 teams in an event, and with at least 12 matches played), the qualification W-L-T is less subject to scheduling randomness and the ranking system will do a better job sorting teams. As we see in single-day off-season events, when less qualification matches are played, seed rankings get more "noise" in them and are less well correlated to the best robots.

Have FiM teams generally found that FiM district events result in seeding rankings that are pretty good - ie, there aren't many high-seeding robots that seem out of place at FiM district events?

However, even when there is some scheduling randomness leading to a mis-sorted ranking, I think the benefit of giving bonus points to well-picked robots outweighs the harm of giving bonus points to the "lucky" captain who seeded higher than they deserved to be. There will be both more robots picked than captains (16 vs 8) and the "diamond in the rough" picks will typically gain more points relative to their pre-pick points than the "lucky" captains will.

I agree that having the "lucky" captains get alliance selection points is less than optimal, but I think this is a case where we need to go with the lesser of two evils in order to avoid throwing the baby out with the bathwater.

I agree that I don't like the thought that teams would change their decisions of who to pick, or who to accept, based upon the point system. I would prefer that teams simply make picking / acceptance decisions based upon the event, rather than the points system. However, as long as district points are gained in eliminations, potential in-tournament point considerations will always be a factor.

Even if no "alliance selection" points are issued, but points are assigned for winning matches in eliminations, we have this problem. For example, if there are no alliance selection points but getting eliminated in the finals is worth more points than getting eliminated in the semifinals (i.e. points for elimination win, as you mention), in a tournament with an emerging very strong #1 alliance (think 1114/2056), a #6 captain selected by a #4 or #5 seed has a strong motivation to decline and instead captain their own #6 alliance. The #4/#5 alliance would have very little chance to get to the finals, but the #6 alliance would have a reasonable shot, since they wouldn't face #1 in the semifinals. (By the way, I think the above scenario may have actually played out this year in at least one regional, due to the "wild card" assignment -- getting eliminated in the finals could earn a ticket to CMP at late regionals, while getting eliminated in the semifinals resulted in an ended season.)

Actually, I think with a relatively small (30-40 team) district event and many (12+) matches, the scenario you describe (of a weak #1 seed who might want to be declined by a #6 captain) is not very likely due to better sorting of teams. When it does happen, I think it would be a relatively rare event. However, I think the strong #1 alliance situation I mention above, where a #4 or #5 captain might select a #6 or #7 seed, is more likely to occur (just ask a Toronto-area team.) Then again, here in New England, we benefit from the fact that we don't have any real powerhouse teams that are way above the pack, so it might be less of an issue for us here than in some other regions.

Re: 2013 NE FIRST District Rankings
 

Somewhat new theme - Philosophy of NE FIRST Points System.

I'm not entirely sure whether to start a new CD thread for this topic or continue the discussion on the NE FIRST Points System in this existing thread. I've opted to keep it in this thread, presuming that those who are interested in the NE FIRST Points System are already here...

I realize that for the NE FIRST folks, determining the points system is really one of the minor details with regard to switching to the district system. Nearly all of the work is in things like planning district event locations, times and venues; getting plans in place for sponsors and volunteers for events; figuring out a venue and organization for the NE championship, etc.

Figuring out which teams attend the NE CMP is really practically irrelevant from a perspective of switching over to the district model for the NE FIRST organizers -- regardless of which teams attend, there's an incredible amount to work through for the NE FIRST committee! However, for the teams, it is probably one of the top 5 issues of interest.

As has been mentioned previously in this thread by IKE, the real thing to look at in the rankings is the teams near the cutoff. The "obvious" teams will make the cut with nearly any points system. However, the real question is whether the teams that barely make the cut generally make sense as opposed to ones that fall just below the cut? Alas, this is actually kind of hard to evaluate without attending most of the regional events, as familiarity with which teams are more/less deserving is hard to evaluate for teams at this 75th percentile level.

When trying to figure out potential ways to assign teams points for qualification to NE CMP, I realized that it is essential to consider the philosophy of "Who Qualifies for the NE CMP."

In order to evaluate / consider different approaches for tallying up district points, it is necessary to first consider what is desired with respect to "Who Qualifies for the NE CMP?" Without knowing the goal of the "points ranking system" it is hard (impossible) to evaluate any given ranking system against that goal.

Clearly, the tautological objective of the proposed NE FIRST "points ranking system" is to determine which teams qualify for the NE CMP. But, what types of teams does NE FIRST want to qualify? Let's list a few of the potential objectives / types of teams that I have heard mentioned:
* The best robots as determined by demonstrated performance on the field
* The best teams at demonstrating the principles / ideals of FIRST
* The best robots as assessed by knowledgeable FRCers, even if they encountered bad luck in demonstrated performance
* The best robots as determined by judged awards
* The teams that would most benefit from the inspiration derived from attending the district championship
* The teams that haven't been to the district championship recently

From my understanding as an outside observer, the FiM points ranking system is specifically intended to put the best robots onto the playing field at the Michigan State Championship. This is no secret, as it is publicly stated all over the place, and is a regular point made about the FiM Championship - that it has the highest average scores of any event in the world, exceeding even the FIRST World Championship in average caliber of robot capabilities. In accordance with this, the FiM points ranking system provides more points to achievements which are highly correlated with the best robots, and less points to achievements that have less to do with the best robots. (The FIRST World Championship has more excellent robots in attendance, since it draws on robots from around the world, but the average is lowered by including not only the best robots, but also teams that have won the Chairman's Award, Engineering Inspiration Award, or Rookie All-Star award at regional events.)

The FiM leadership has intentionally assigned the point system the way it is to try to put the best robots onto the playing field at the Michigan Championship. As a result, even though FiM district winners of the district chairman's award, district engineering inspiration award, and district rookie-all-star awards get to participate in the Michigan Championship judging for those awards, only the district chairman's award winners automatically qualify to compete with their robot. District engineering inspiration awards and district rookie-all-star awards do not qualify for robot positions at the Michigan Championship, but instead qualify the team to send students to present to the Michigan Championship judges for those awards at the state level. Only district winners of those awards compete for those awards at the Michigan Championship. The Michigan State Championship winners for Chairman's Award (3 selected winners), Engineering Inspiration (1 winner) and Rookie All Star (1 winner) all qualify for the FIRST World Championship. (Such participation includes their robots at the World Championship.)

If NE FIRST's objective for the NE Championship is to have the best robots on the playing field at the NE Championship, then careful consideration should be given to simply adopt the FiM/MAR points system. I can only presume that the FiM folks have each of the different kind of points in their ranking system for a reason. Personally, I think there are good reasons for including the alliance selection points and different amount of elimination points for captains and 1st-picks, vs 2nd-picks. That said, NE FIRST may have a different perspective/objective than that of FiM, and intentionally want to do something other than seek to put the best robots on the playing field at the NE Championship. If so, that would be a good reason to do point assignments differently. Alternatively, even if the goal at NE FIRST is to put the best robots on the field, there may be other ways to determine the best robots that FiM either didn't explore, or couldn't get sufficient backing to implement. Personally, I think NE FIRST should also consider such alternate point scoring systems, but should also realize that the FiM/MAR system has been field tested quite a bit!

Re: 2013 NE FIRST District Rankings
 

I just wanted to note that while there are many valid points in your discussion, this section is rather untrue. If you look at the "status" page of the NE FIRST website, you will have a glimpse of how much work went into putting together a points model for NE FIRST. Almost immediately they split into operations and organization committees and began work on both parts of the system. They actually spent a lot more time working on and discussing this in the past year and a half than they did on locations. They realized this was incredibly important to teams and not only carefully researched and discussed all the models, they came up with a model that at times has been vastly different from FiM and MAR. As they discussed with teams at town halls, the model would change from month to month as they developed a picture of what the teams wanted. Even more recently, given a lot of these CD threads and opinions they have gathered from teams, they have changed the model to match what they are hearing from teams.

Are there still issues with the current proposal? Well they will never be able to make everyone happy. But they have NOT just blindly adopted the FiM & MAR models as you can see from several of the rankings threads. There are plenty of ways to argue which is better, and to be honest, as you suggest - it is all about what Philosophy NE FIRST & its teams wants to take with regards to its model. Are they in it to send the best robots? Do they want to generate more Chairmans winners? Do they want to give everyone a chance to play? Do they want to promote growth and sustainability? Each of these can lend itself to a slightly different model.

But I wanted to at least set the record straight that the NE FIRST committee has spent quite a lot of time going over and revising the points model, and it is far from an afterthought or a minor detail.

Re: 2013 NE FIRST District Rankings
 

I'm sorry; I had not intended to offend or imply that the NE FIRST committee thought the points model was an afterthought. I definitely didn't think that the NE FIRST committee thought it was unimportant, but simply thought it had not yet been an issue they have had the bandwidth to address.

With your post, I am clearly mistaken. I have not been aware of any of the ongoing discussion within NE FIRST about the points model. Without seeing any updated status on the NE FIRST website regarding the points model, I had mistakenly presumed that the points system simply hasn't been one of the issues being worked by the committee. I'm glad to hear that it has been given a lot of consideration.

I clearly haven't been proactive enough about trying to become involved, but from my perspective I hadn't been aware of the work going on within NE FIRST regarding the district points structure. I attended the NH town hall meeting (which was not a live meeting, but a web event long after the other town hall meetings, just before Christmas 2012) but I don't recall anything being mentioned about the points model at that meeting, other than that it was coming soon. The first details I saw were Jess Boucher's 3 Jan 2013 blog posting, but that was the first and last time I ever saw anything definitive regarding it. Other than unofficial CD threads such as this one (which was started by NY capital district folks outside of NE FIRST), I haven't heard any news at all regarding the NE FIRST district model since kickoff. I've even been checking the NE FIRST web site regularly looking for new information about how to contribute ideas, but haven't seen anything new. It may just be the case that the NH teams have not had a good way to communicate / interact regularly with each other or the NE FIRST committee, and that there is more of a dialogue between teams and NE FIRST in other states?

In any case, thanks for letting me know that the NE FIRST committee has been working hard on the points model. Is there a more recent draft, or is the one on the web site from 3 Jan 2013 still the latest?