Idea to Balance Championship Divisions
 

As everyone who followed this year's FIRST Championship saw, sometimes the mostly-random assignment of teams to Championship Divisions can result in some divisions being considerably more competitive than others. In order to better balance the divisions and improve the Championship experience for everyone, especially with the move to two Championships of more than four divisions each separating top teams even more, I suggest applying the following process to each Championship next year:

Upon qualifying for their respective FIRST Championship, each team is assigned to one of seven pools, or groups of teams, based on how they qualified:

1. Competition Results I
   -DCMP Winner (Captain / First Pick)
   -Regional Winner (Captain / First Pick)
2. Competition Results II
   -DCMP Winner (Second Pick / Backup)
   -Regional Finalist (Captain / First Pick)
   -Veteran District Points Qualifying Team
3. Culture Awards
   -DCMP Chairman’s Award
   -DCMP Engineering Inspiration Award
   -Regional Chairman’s Award
   -Regional Engineering Inspiration Award
4. Rookies
   -DCMP Rookie All-Star Award
   -Regional Rookie All-Star Award
   -Rookie District Points Qualifying Team
5. Competition Results III
   -Regional Winner (Second Pick / Backup)
   -Regional Finalist (Second Pick / Backup)
6. Pre-Qualified
7. Waitlist

The pools are constructed to group together teams of similar competitiveness. If anyone has an alternative set of categories in mind, do feel free to share.
If a team qualifies for the Championship and earns multiple of the listed achievements (including at multiple events, or at a regional and in a district), they are placed in the highest ranked pool they qualify for. The pools are then divided equally and randomly between each Championship Division. This way, each division receives a more balanced mix of the varying levels of competitiveness.

Re: Idea to Balance Championship Divisions
 

I just wanted to interject, do we know for certain this is true? I think it's been a long-time assumption that assignment of a Championship division is pseudo-random, but I don't think we know that for certain. I sort of remember Frank going through how they assign rookies to make sure that there's an equal amount in each division, but I haven't heard anything else other than conjecture about team sorting.

There's plenty of statistical analysis to show how unbalanced the divisions were this past year. However, without knowledge of the algorithm used to generate the divisions, we don't know if it was just random chance or a problem with the algorithm itself (or if competitive balance was even a factor within the algorithm in the first place).

Re: Idea to Balance Championship Divisions
 

http://www.firstinspires.org/robotic...atch-Schedules

Re: Idea to Balance Championship Divisions
 

While it was a bit presumptuous of me to assume they're still using exactly the same process as 2014, I think my proposed system would be an improvement over the 2014 system or whatever variation of it they've been using since.

Re: Idea to Balance Championship Divisions
 

Thanks! Didn't know they did the same thing with veterans as they did with rookies.

I definitely think this proposed system has some merit. It's similar enough to the current system (just increasing the number of subgroups teams are divided in instead of just rookie/veteran), but should even out the fields a bit more.

Re: Idea to Balance Championship Divisions
 

I think this has merit and should lead to more balanced divisions... it's reasonably straightforward to implement and doesn't rely on rank or statistics that are subject to change each year.

The one thing that I think FIRST would likely find as a significant sticking point is having the Chairman's and EI (Culture Awards) as the third sort instead of the first. I agree with you that having them third is probably more likely to result in well-balanced divisions from an on-field performance standpoint, but even if the Culture Awards were moved to the first-sort I think this general method would probably still be better than the status quo.

Re: Idea to Balance Championship Divisions
 

I think they should do a draft... with previous championship WFA winners each picking divisions.

Could make a draft night show out of it. :p


EDIT:
And with the 10th round, 86th pick of the draft, Andy Baker selects 3940 for Newton.

Re: Idea to Balance Championship Divisions
 

I vaguely remember, perhaps a decade ago, teams were accused of gunning for particular awards and registering for particular regionals because they believed it would get them in a "favorable" CMP division. Does anybody else recall this?
The reason I bring this up is setting the precedent for teams being accused of colluding before the season to end up in the same CMP division, if they knew and could outsmart the placement algorithm.

Re: Idea to Balance Championship Divisions
 

This is an amazing idea.

I think FIRST hasn't TRIED to balance championship divisions, other than making sure rookies are relatively evenly distributed. If they wanted to, they could do some sort of structure like you're suggesting, Brian, or literally just distribute teams evenly by OPR. It's not too difficult to create even divisions, they just haven't tried.

Re: Idea to Balance Championship Divisions
 

I guess I'll weigh in a bit.

Personally, I think the current way divisions are organised is just fine. Sure, this year we got extremely competitive divisions and some less competitive divisions, but that's just the nature of how these things work out.

From what I understand the current (or at least, 2014, judging by the earlier link to FIRST's own website) system for sorting for championships is done in a sort of round-robin style, first with rookies, then with veterans. In this way, it's very hard to introduce bias into the division split. The same would be true for generating division placements by random number. In both ways, there's no 'filter' applied to each division, making it seem like a fair and balanced way to distribute teams.

In the suggestion given by the OP, it seems like they're suggesting much the same system, but breaking it up to more categories than just "rookie/veteran", i.e. round-robin on pools of qualification level (winner, finalist, award, pre-qual, waitlist etc). This is all well and fine, in fact, from face value, it's better than the current system. However, don't stop reading here.

The main problem with ALL methods of distributing teams is that teams aren't individual, but play in alliances. The performance of the whole alliance depends on how each team within that alliance performs, and how they perform together. This is the reason I don't think splitting teams in this way will make a huge difference when looking at the bigger picture, because rankings all come down to alliance allocations in matches. For the sake of argument, let's say it does make a difference, and the (objectively) best teams end up ranked after each other in descending order. During alliance selections is really where things start to kick off. At the end of the day, no matter the distribution, Alliance selections will often follow the same kind of recipe: "Top Team / Top Team #2 / Middle Team / Last Team". When it comes to playoffs within the divisions, only one alliance can be victorious. This gets rid of that team-stacking distribution we saw this year on some divisions. I believe that this is the reason playoffs and alliance selections are the way they are. Now would be a good time to mention that all of this comes down to interpretation. As a final point, playoff matches are where the real competition starts, and qualification matches depend almost entirely on how the alliances are made up on the match schedule.

Through either distribution of teams, I theorize that the same teams would end up on Einstein either way, which is where the competition really amps up to another level.

Again, this is all just my interpretation of each distribution and how I *THINK* it would work out. Don't take it as gospel

Re: Idea to Balance Championship Divisions
 

Here is a question. Do stacked divisions actually matter?

Most people would agree that the threshold of "stacked divisions mattering" would be that they usually win champs because they are "stacked".

Lets look at the data. (Taken from this post on average OPR)

So for the 4 division era, the average OPR placement of the world champs is 2.91. 8 division avg is 4.

Scaled avg for 8 divisions is an average pre champs OPR rank of 5.54. So... No correlation.

What about how well "Stacked" divisions preformed at champs? (Based on avg opr)

Assigning 1,2,3,4 for each stage they make it through elims, you get an average round KO of 1.62 meaning on average, the most "stacked" division ended their day in quarters for Einstein.

If someone wants to do avg of the top 24 for each division so we can redo our analysis, but right now your odds don't look particularly good for this whole "division" balancing having any impact. Looks like divisions don't really need to balanced from everything I can see. luck in match schedule and creating a cohesive and competent alliance matters way more.

EDIT

In fact the top 5 most "stacked" divisions 4/5 got KOed in the first round on Einstein. That makes sense. 50% of teams get KOed first round, hence it's at best a random correlation, and too small of a sample size to actually say that the majority of stacked divisions actually perform poorly compared to average divisions.

So yeah. No correlation.

Re: Idea to Balance Championship Divisions
 

I don't think "most people" would say that, and the OP definitely didn't. He said "improve the Championship experience for everyone", not just the few teams that make Einstein. Many teams want to play with the best in the world, not watch them compete in another division.

Re: Idea to Balance Championship Divisions
 

The implication of your post, Brennan, is that the primary downside of a stacked division is an unfair advantage on Einstein. I don't think that's the main reason at all that unbalanced divisions are a bad thing. It's mainly the worse playing experience playing in an overly strong or weak division, good teams being cut from the elims because they lucked into the strongest division, etc.

Anyways, we have all of this information - anyone invested in this idea could do some programming and spit out 8 hypothetical divisions this year sorted by this algorithm, and then analyze these divisions using different metrics (OPR, etc) to see how unbalanced they are. I'd love to see this sort of analysis.

Re: Idea to Balance Championship Divisions
 

Fair enough, how does balancing divisions help improve the championship experience? Is there a viable way to have elite teams play with elite teams? Doesn't that happen already?

Chris brings up a decent point. I can definitely seeing that have an impact on some teams. I wonder how much of that is a hard or easy division and more just people making strange picks during alliance selection? And even if we balance it. Who is to say we will do much better at creating a more balanced division? Perhaps slightly. Robots change. Things happen.

Re: Idea to Balance Championship Divisions
 

As Jaci mentioned, since (bar bad scouting) you only have two "top" robots per alliance, stacked divisions don't end up with a stacked alliance on Einstein. Stacked Einstein alliances are almost entirely due to bad scouting by the majority of the division they come from. What they do result in is many stacked alliances in divisional playoffs. Some of those alliances that lose in the semis or finals of a stacked division could be objectively better than alliances that win weaker divisions, and it sucks for those teams to know that they missed out on Einstein just by being unlucky in their assigned division.

What's the opportunity cost for making this change? I've seen some people debating that it would make a positive change, and some arguing that there wouldn't be a significant effect at all. Does anybody thing this would cause a negative change?

Re: Idea to Balance Championship Divisions
 

I propose that they just go back to what they used previously.

Let the divisions be letters A, B, C, D, E, and F.
Let the teams that register be 1, 2, 3, ..., n, sorting by numerical order.

1 goes in A, 2 goes in B, 3 goes in C...6 goes in F, 7 goes in A, 8 goes in B...n goes in the next-in-sequence of A-F.

Assign A-F to random division names.

The problem (or not) with this method is that anybody can generate a division list with a few minutes and some programming knowledge, and knowing which teams are signed up.

Re: Idea to Balance Championship Divisions
 

I agree. The problem with division stacking is not who wins the title, but rather who gets the opportunity to play for the title. The best example is this year comparing Curie to Newton. There were teams in Newton left off playoff alliances that would have played on Saturday in Curie if divisions were more balanced. And the Curie division winning alliance clearly wasn't up to the level of the other division alliances on Einstein because it didn't have a deeper pool of teams to draw from.

I too like the segmenting approach to assigning. It's really only a variation on what FIRST is already doing.

Re: Idea to Balance Championship Divisions
 

You can fix that by grabbing the next 6 teams, randomizing them, then assigning them A-F.

Re: Idea to Balance Championship Divisions
 

This is the first I've heard of it. To the best of my knowledge at no point in the last 17 years has what regional you qualified at or what qualifying award you won had anything to do with your division assignment.

Re: Idea to Balance Championship Divisions
 

The only correlation I am aware of is that by using the registration timing mechanism, and then assigning them to each division, and the deadlines after winning an event, you would likely get a scenario where the teams that won an early event together would not be in the same division.

Re: Idea to Balance Championship Divisions
 

I generated some division lists based on what was outlined in the OP. The pools were slightly different though (mostly because I got the qualification types from a different project I've been working on).

The pools are:
Note that I used first qualification method, rather than the "highest", and that DCMP winners are in "District Points" pool because FIRST doesn't give DCMP winners automatic qualification.
I've attached an OPR chart for the different divisions, along with the actual distribution last year. The balanced distribution looks slightly better, but there is still a division that's clearly above everyone else.

Re: Idea to Balance Championship Divisions
 

Just wanted to say I find it a little strange that the OP placed DCMP Winner Second Pick/Backup in Category II while he put Regional Winner Second Pick/Backup in Category V. I understand that obviously District Champs will have significantly deeper fields than Regionals, but is it really this big of a difference? Seems kinda extreme...

Re: Idea to Balance Championship Divisions
 

Problem fixed.

Re: Idea to Balance Championship Divisions
 

There are three reasons behind this decision:

1) The fact that number of district points earned by these robots almost always puts them above the cutoff to qualify as a District Points team

2) Average OPR from this year's Championship shows that the DCMP Winner 2nd Pick / Backups are considerably stronger than those of regionals:

• DCMP Winner Pick 2 / Backup: 36.6
• Regional Winner Pick 2 / Backup: 28.3

Note: 2nd picks and backups were lumped together because I felt the sample sizes (1 for DCMP winners and 1 for Regional winners) for backups were too small to evaluate individually.

3) Having attended or watched footage from a number of regionals and District Championships, anecdotally, I have seen considerably higher performance from the winning second picks at DCMPs than at regionals. This makes sense when the reasoning for why this may be is considered.
The barrier for entry to a DCMP is way higher than that of a regional: strong performance at District events (or District Chairman's) versus nothing at all.
In addition, the winning alliance at a regional represents the 30-65 teams attending that regional. The winning alliance at a District Championship represents the 50-400 teams in that district.

While this is generally a non-issue due to the impact of the 90 playoff round performance points most DCMP winners receive, that is technically false.

Re: Idea to Balance Championship Divisions
 

Hmm, I was looking at this document which contains a table that does not state that DCMP winners automatically qualify. Although looking at it now, it also seems to leave out Regional Winners and Wildcards! I'm going to assume that this table is in error and that the Admin Manual is correct.