FRC Blogged-Standard District Point Structure

[Jon Jack]

Sean is 100% correct about the approach. Most people immediately think that ranking system is supposed to determine who the best overall team is. In reality that is irrelevant. The best teams will always float to the top of the ranking system and being ranked #1 vs. #6 means nothing except for bragging rights - they're both advancing. The actual purpose of the point system is to determine the cut off of who advances and who doesn't advance.

In a league of any size the best teams always standout. Think about college football or basketball rankings. How many times do you see people arguing that the #25 team should really be the #1 team? You don't. Usually the conversation is about how the #5 team was "robbed" by not being #1. The problem is that once you get down in the middle of the pack things are harder to sort. This point system is a solution that objectively sorts out those teams in the middle. One of the things we carefully considered was how to not over-reward the top end since that could have negative effects on the middle tier. We felt it was more important to reward consistent play over winning just one event and missing elims at another.

Just to clarify one thing - you get 5 points per elimination win IF you win the round. Rewarding teams for taking an elimination matchup to a third match or penalizing teams for not sweeping is a dangerous proposition. In this situation you now have a system with a variable number of total points. This shifts the cut off around a lot, which makes it harder for teams to predict the likelihood they'll advance. If teams near the cut off have to wait until after week 6 to know for sure that they're advancing that leaves them a few days to secure transportation and hotel rooms for their district championship.

[Citrus Dad]

First don't call this a ranking system if it's a actually a qualificationsystem. And don't post world "rankings" of they aren't intended to be so.
That said the current scoring will not give you enough differentiation. And you haven't thought through the incentives enough. Thinking about incentives is my day job. Again it's the elimination round where the action is. Qualifications is the prelude.
Best as I can tell most of the discussion has been about awards not competition.

[cadandcookies]

Originally Posted by Citrus Dad

First don't call this a ranking system if it's a actually a qualificationsystem. And don't post world "rankings" of they aren't intended to be so.
That said the current scoring will not give you enough differentiation. And you haven't thought through the incentives enough. Thinking about incentives is my day job. Again it's the elimination round where the action is. Qualifications is the prelude.
Best as I can tell most of the discussion has been about awards not competition.

You do realize that a good deal of very smart people who have been in FIRST in some cases since the beginning came up with and probably argued all of you points in the process of inventing the district ranking system, right? There's a thread from this fall that I can't find right now because I'm on mobile that goes into a lot of depth on the changes and adjustments made to ranking teams in Districts.

And it is a ranking system, it's just a ranking system designed for the middle of the bell curve more than the edges. Which is how it needs to be. I'm not sure how much you've read on districts and the logic that has gone into creating and running the involved systems, but it might be a good idea to familiarize yourself with the underlying logic a bit more before you start accusing people of not having thought things through enough.

That being said, I do applaud you for actually coming up with some possible changes instead of just complaining (even if I disagree with your proposals).

[Andrew Schreiber]

Originally Posted by Citrus Dad

First don't call this a ranking system if it's a actually a qualificationsystem. And don't post world "rankings" of they aren't intended to be so.
That said the current scoring will not give you enough differentiation. And you haven't thought through the incentives enough. Thinking about incentives is my day job. Again it's the elimination round where the action is. Qualifications is the prelude.
Best as I can tell most of the discussion has been about awards not competition.

It IS a ranking system. But is a ranking system not geared around determining who is #1. It was primarily optimized about the boundary points with questionable accuracy about the edges (first and last ranks).

[Jim Zondag]

Originally Posted by Jon Jack

The actual purpose of the point system is to determine the cut off of who advances and who doesn't advance.

Jon is exactly right here. The purpose of these systems is to perform the difficult sort at the entry boundary to the next tier. The District system must sort the combined input from hundreds of teams playing 2 events each and determine who will advance. The same system is then used again at the regional championship level to determine who will advance to the world championship. While being on the top is great for bragging rights, it is functionally irrelevant: the only thing that really matters is 'who is in and who is out?'. The system is designed to take in 4 inputs from standard measurable items present at all FRC events: QP W/L/T results, Alliance Selection position, Elimination match tier reached, and Judged Awards received. These items are game independent and are portable from year to year regardless of the game design. Succeed at achievement in these 4 areas, and your team will rank higher than those who achieve less. Simple: do good things, get noticed, move forward.

While this is still new to most of the FRC, we are in our 6th year of deployment of this system in Michigan. The adjustments made in 2014 are fairly minor tweaks to original 2009 design. There is not really any problem with properly incentivising teams here.

If someone wants to post "worldranks" or the like, just relax have fun with the math. No one number can truly tell us how good any team actually is, but it is fun to try to find one which can.

"Measure what is measurable, and make measurable what is not so." -
Galileo

[GaryVoshol]

To add to Jim's points, it's not hard to pick out the top teams who should advance. Even with 15 districts this year, I'm sure there will be general consensus that these 10 or 15 teams absolutely deserve to be at MSC. (Yes, sometimes circumstances could gang up on a "good" team and they don't qualify, but that's rare.)

The hard part is deciding who's in 60th place and gets to go, and who's 61st and is staying home. (Adjust numbers accordingly depending on how many advance, how many auto-advancers, declines, etc.)

Judges in FLL have the same quandary when they are deciding who advances from qualifiers. The top 2-4 teams are obvious. But who is 8, 9, 10? And who is 11 and doesn't advance?

[Nemo]

I agree with leaving it the way it is to put the focus on getting the best separation among the teams on the bubble of qualifying or not.

I did think it was an interesting idea to split the elimination round points a bit for teams that take one of the matches. If the split was 8 / 2 in a 2-1 series instead of 10 / 0 for a 2-0 series, that wouldn't seem unreasonable to me.

[Jessica Boucher]

Jim's also missing the age incentive, but all the points made earlier are still valid.

That being said, I think it is great that the model is being questioned, so keep questioning! That's why we went through the whole process of creating our own model in NE in the first place: to iterate on what we thought was good and improve in places that we thought needed it. Because of this, we were able to get to a model that felt more unified and represented all of FRC.

I think the biggest reason why we must sort is when it comes to invitations to the DCMP and the CMP. Teams may decline to either event, and thus the invitation list can get very deep. As Jim said: Do good things, advance.