FRC Blogged-Standard District Point Structure

[dodar]

Originally Posted by Citrus Dad

Here's my thoughts on how the District scoring should be changed as its implemented more widely:

- Losses in elimination rounds should count against a team, say -1.
- Also, later wins should gain more wait, beyond simply 5 more points. This will give a bigger distinction for teams that make it to the finals, more than just 5 more points. Winning an overall event should be worth much more than winning 100% of the matches (e.g. 24 points for going 12-0 vs 30 points for winning the Regional.) Probably should aim for winning the regional to be worth 48 points instead (double winning the qualifying rounds). Scoring 6 points for quarters, 8 points for semis and 10 points for finals would do that.
- Finally, you should normalize on a non-integer basis as a percentage. It's more difficult to win more matches in a longer tourney because of the abuse of the robots, but the current scaling doesn't give full credit for that.

So you believe a team could make the eliminations and end up lower than a team that didnt?

[AdamHeard]

Originally Posted by dodar

So you believe a team could make the eliminations and end up lower than a team that didnt?

that's what draft points cover.

[PVCpirate]

Originally Posted by Citrus Dad

Here's my thoughts on how the District scoring should be changed as its implemented more widely:

- Losses in elimination rounds should count against a team, say -1.
- Also, later wins should gain more wait, beyond simply 5 more points. This will give a bigger distinction for teams that make it to the finals, more than just 5 more points. Winning an overall event should be worth much more than winning 100% of the matches (e.g. 24 points for going 12-0 vs 30 points for winning the Regional.) Probably should aim for winning the regional to be worth 48 points instead (double winning the qualifying rounds). Scoring 6 points for quarters, 8 points for semis and 10 points for finals would do that.
- Finally, you should normalize on a non-integer basis as a percentage. It's more difficult to win more matches in a longer tourney because of the abuse of the robots, but the current scaling doesn't give full credit for that.

For the elims losses, I think a similar effect could be obtained by allowing alliances that take their opponents to 3 matches but lose to get the 5 points for winning one match. It's still in an alliance's best interest to sweep, but no negative points to worry about.

Draft points mean that the alliance captain and first pick of an alliance score between 39 and 46 points from winning an elimination tournament. Knowing that, I don't agree that there needs to be more points for advancing in the tournament.

For your last point, are you talking about eliminations or qualification? Every team gets 12 qualification matches at a district event, regardless of number of teams.

[Citrus Dad]

Originally Posted by dodar

So you believe a team could make the eliminations and end up lower than a team that didnt?

As Adam pointed out, draft points will cover at least one loss and most likely two. Only the last team drafted might end up with negative points--however, they are drafted by the No. 1 Alliance Captain, so the odds are much less that they are going to lose two matches in the first round (and they'll almost certainly get at least one win worth 5 points.) A No. 1 alliance that goes down to No. 8 in two matches deserves to get negative points...

[Citrus Dad]

The issue I'm addressing is not boosting the rating of the teams that extend the round, but rather penalizing the teams that fail to sweep it quickly. If you look at the "rankings" recently posted, they're crowded at the top. Yet teams that swept through the eliminations (yes, we were one of those) get no bonus for avoiding all losses, while teams that might have gone to 3 matches in every round get the same number of points. That doesn't reflect the relative strengths of the different alliances within their regionals. An alternative solution would be to give the total points for the wins in that round (e.g., 5 points each for a total of 10) but divide it by the number of matches played in the round, so a sweep would be worth 10 points, and 2 out of 3 would be worth only 6.7 points.

I think that winning the elimination rounds should be worth MUCH more than a high qualifying position. The point of the Regionals/Districts is to win the overall competition. The top qualifier does not go to Championships for a reason--those are only preliminary matches. In fact with the current weighting I can see how a team that might have won a District one week might find it advantageous to play through the qualifying round to get sufficient points and then just sandbag the eliminations to save their robot. In addition, but awarding so many points for just being drafted high, this makes the situation even more likely. You need to include the draft position points under "qualifying rounds", not in "elimination rounds." The points need to be set up to encourage competition throughout by given the lions share of points at the end, not midway through. An Alliance captain can get 47 points at the end of the draft. Why should they earn more than a No. 8 seed that goes through and wins the District? Thinking about this, I think the elimination rounds need even higher point scores than what I proposed originally--probably should target an additional 100 points for winning the whole thing. That could be 6 points for quarter wins, 12 points for semis, and 32 points for finals.

I was commenting in the context of regionals. However, I'm not sure that there's a guarantee that all districts will have the same format. That may have to change depending on geography. For example, California may not have enough venues to run such small district competitions.

Originally Posted by PVCpirate

For the elims losses, I think a similar effect could be obtained by allowing alliances that take their opponents to 3 matches but lose to get the 5 points for winning one match. It's still in an alliance's best interest to sweep, but no negative points to worry about.

Draft points mean that the alliance captain and first pick of an alliance score between 39 and 46 points from winning an elimination tournament. Knowing that, I don't agree that there needs to be more points for advancing in the tournament.

For your last point, are you talking about eliminations or qualification? Every team gets 12 qualification matches at a district event, regardless of number of teams.

[Lil' Lavery]

You're evaluating the district ranking system from the completely wrong perspective. The point of the ranking system isn't to assess who's #1. The ranking system is designed to have value at the cut-off for the district championship, and there's where the point allocation matters most. Instead of attempting to further validate your own team's achievements based on what you determine to be important or difficult, how about you review the extensive documentation explaining the logic behind the ranking system?

Penalizing losses in the eliminations, beyond the implicit penalty of not earning +5 points for a win, is silly. It would actually be advantageous for a team about to get blown out to "break" and call in a back-up robot. Likewise, it would remove incentive for teams to remain unbagged to become back-up robots, knowing the odds are they will receive -1 or -2 points.

Rewarding sweeps and increasing bonuses for elimination success will further boost the 2nd round selections of powerhouse alliances, while both explicitly and implicitly penalizing the teams drafted ahead of them to lower seeded alliances. Considering these will be the teams vying for the district championship cutoff, these are the points that matter. You're making it even more difficult for the mid-tier teams to advance, which is the exact opposite of the intended effect of district competitions.

The point about teams "sandbagging" after winning a district is nothing short of ridiculous. If a team has already won a district and drafted high in their elimination tournament at their second event, they're going to be in the hunt for one of the top points spots (and earning a bid to Championship) heading into their district championship. They're going to want as many of the potential 30 elimination advancement points as they can get.

And California has plenty of high schools, no? There will be plenty of venues for 40-team districts.

I'm not saying the current system is perfect, but these suggestions take it in the opposite direction. I doubt you'd find many, if anyone, who's actually played in the district format who'd support them.

Originally Posted by Citrus Dad

A No. 1 alliance that goes down to No. 8 in two matches deserves to get negative points...

As a member of a team who's been on both sides of a #8 sweeping a #1 at district events, hell no they don't.

[Jon Jack]

Sean is 100% correct about the approach. Most people immediately think that ranking system is supposed to determine who the best overall team is. In reality that is irrelevant. The best teams will always float to the top of the ranking system and being ranked #1 vs. #6 means nothing except for bragging rights - they're both advancing. The actual purpose of the point system is to determine the cut off of who advances and who doesn't advance.

In a league of any size the best teams always standout. Think about college football or basketball rankings. How many times do you see people arguing that the #25 team should really be the #1 team? You don't. Usually the conversation is about how the #5 team was "robbed" by not being #1. The problem is that once you get down in the middle of the pack things are harder to sort. This point system is a solution that objectively sorts out those teams in the middle. One of the things we carefully considered was how to not over-reward the top end since that could have negative effects on the middle tier. We felt it was more important to reward consistent play over winning just one event and missing elims at another.

Just to clarify one thing - you get 5 points per elimination win IF you win the round. Rewarding teams for taking an elimination matchup to a third match or penalizing teams for not sweeping is a dangerous proposition. In this situation you now have a system with a variable number of total points. This shifts the cut off around a lot, which makes it harder for teams to predict the likelihood they'll advance. If teams near the cut off have to wait until after week 6 to know for sure that they're advancing that leaves them a few days to secure transportation and hotel rooms for their district championship.

[Citrus Dad]

First don't call this a ranking system if it's a actually a qualificationsystem. And don't post world "rankings" of they aren't intended to be so.
That said the current scoring will not give you enough differentiation. And you haven't thought through the incentives enough. Thinking about incentives is my day job. Again it's the elimination round where the action is. Qualifications is the prelude.
Best as I can tell most of the discussion has been about awards not competition.

[cadandcookies]

Originally Posted by Citrus Dad

First don't call this a ranking system if it's a actually a qualificationsystem. And don't post world "rankings" of they aren't intended to be so.
That said the current scoring will not give you enough differentiation. And you haven't thought through the incentives enough. Thinking about incentives is my day job. Again it's the elimination round where the action is. Qualifications is the prelude.
Best as I can tell most of the discussion has been about awards not competition.

You do realize that a good deal of very smart people who have been in FIRST in some cases since the beginning came up with and probably argued all of you points in the process of inventing the district ranking system, right? There's a thread from this fall that I can't find right now because I'm on mobile that goes into a lot of depth on the changes and adjustments made to ranking teams in Districts.

And it is a ranking system, it's just a ranking system designed for the middle of the bell curve more than the edges. Which is how it needs to be. I'm not sure how much you've read on districts and the logic that has gone into creating and running the involved systems, but it might be a good idea to familiarize yourself with the underlying logic a bit more before you start accusing people of not having thought things through enough.

That being said, I do applaud you for actually coming up with some possible changes instead of just complaining (even if I disagree with your proposals).

[Jim Zondag]

Originally Posted by Jon Jack

The actual purpose of the point system is to determine the cut off of who advances and who doesn't advance.

Jon is exactly right here. The purpose of these systems is to perform the difficult sort at the entry boundary to the next tier. The District system must sort the combined input from hundreds of teams playing 2 events each and determine who will advance. The same system is then used again at the regional championship level to determine who will advance to the world championship. While being on the top is great for bragging rights, it is functionally irrelevant: the only thing that really matters is 'who is in and who is out?'. The system is designed to take in 4 inputs from standard measurable items present at all FRC events: QP W/L/T results, Alliance Selection position, Elimination match tier reached, and Judged Awards received. These items are game independent and are portable from year to year regardless of the game design. Succeed at achievement in these 4 areas, and your team will rank higher than those who achieve less. Simple: do good things, get noticed, move forward.

While this is still new to most of the FRC, we are in our 6th year of deployment of this system in Michigan. The adjustments made in 2014 are fairly minor tweaks to original 2009 design. There is not really any problem with properly incentivising teams here.

If someone wants to post "worldranks" or the like, just relax have fun with the math. No one number can truly tell us how good any team actually is, but it is fun to try to find one which can.

"Measure what is measurable, and make measurable what is not so." -
Galileo

[GaryVoshol]

To add to Jim's points, it's not hard to pick out the top teams who should advance. Even with 15 districts this year, I'm sure there will be general consensus that these 10 or 15 teams absolutely deserve to be at MSC. (Yes, sometimes circumstances could gang up on a "good" team and they don't qualify, but that's rare.)

The hard part is deciding who's in 60th place and gets to go, and who's 61st and is staying home. (Adjust numbers accordingly depending on how many advance, how many auto-advancers, declines, etc.)

Judges in FLL have the same quandary when they are deciding who advances from qualifiers. The top 2-4 teams are obvious. But who is 8, 9, 10? And who is 11 and doesn't advance?

[Nemo]

I agree with leaving it the way it is to put the focus on getting the best separation among the teams on the bubble of qualifying or not.

I did think it was an interesting idea to split the elimination round points a bit for teams that take one of the matches. If the split was 8 / 2 in a 2-1 series instead of 10 / 0 for a 2-0 series, that wouldn't seem unreasonable to me.

[Jessica Boucher]

Jim's also missing the age incentive, but all the points made earlier are still valid.

That being said, I think it is great that the model is being questioned, so keep questioning! That's why we went through the whole process of creating our own model in NE in the first place: to iterate on what we thought was good and improve in places that we thought needed it. Because of this, we were able to get to a model that felt more unified and represented all of FRC.

I think the biggest reason why we must sort is when it comes to invitations to the DCMP and the CMP. Teams may decline to either event, and thus the invitation list can get very deep. As Jim said: Do good things, advance.

[Andrew Schreiber]

Originally Posted by Citrus Dad

First don't call this a ranking system if it's a actually a qualificationsystem. And don't post world "rankings" of they aren't intended to be so.
That said the current scoring will not give you enough differentiation. And you haven't thought through the incentives enough. Thinking about incentives is my day job. Again it's the elimination round where the action is. Qualifications is the prelude.
Best as I can tell most of the discussion has been about awards not competition.

It IS a ranking system. But is a ranking system not geared around determining who is #1. It was primarily optimized about the boundary points with questionable accuracy about the edges (first and last ranks).

[Joe Ross]

Originally Posted by Lil' Lavery

You're evaluating the district ranking system from the completely wrong perspective. The point of the ranking system isn't to assess who's #1. The ranking system is designed to have value at the cut-off for the district championship, and there's where the point allocation matters most. Instead of attempting to further validate your own team's achievements based on what you determine to be important or difficult, how about you review the extensive documentation explaining the logic behind the ranking system?

Moving to a graduated level of points for elimination victories does not affect the top of the top, it affects the middle to lower top. The top in any district will have won their first two districts. At that point, the differentiation comes from qualification wins and awards. What it does change is it provides differentiation between a team that is a finalist at both districts vs a team that wins 1 district and is semifinalist at their other. Which is better? What about a team that is semifinalist at both events, vs Finalist and Quarter Finalist?