Idea to Balance Championship Divisions

[Brian Maher]

As everyone who followed this year's FIRST Championship saw, sometimes the mostly-random assignment of teams to Championship Divisions can result in some divisions being considerably more competitive than others. In order to better balance the divisions and improve the Championship experience for everyone, especially with the move to two Championships of more than four divisions each separating top teams even more, I suggest applying the following process to each Championship next year:

Upon qualifying for their respective FIRST Championship, each team is assigned to one of seven pools, or groups of teams, based on how they qualified:

1. Competition Results I
   -DCMP Winner (Captain / First Pick)
   -Regional Winner (Captain / First Pick)
2. Competition Results II
   -DCMP Winner (Second Pick / Backup)
   -Regional Finalist (Captain / First Pick)
   -Veteran District Points Qualifying Team
3. Culture Awards
   -DCMP Chairman’s Award
   -DCMP Engineering Inspiration Award
   -Regional Chairman’s Award
   -Regional Engineering Inspiration Award
4. Rookies
   -DCMP Rookie All-Star Award
   -Regional Rookie All-Star Award
   -Rookie District Points Qualifying Team
5. Competition Results III
   -Regional Winner (Second Pick / Backup)
   -Regional Finalist (Second Pick / Backup)
6. Pre-Qualified
7. Waitlist

The pools are constructed to group together teams of similar competitiveness. If anyone has an alternative set of categories in mind, do feel free to share.
If a team qualifies for the Championship and earns multiple of the listed achievements (including at multiple events, or at a regional and in a district), they are placed in the highest ranked pool they qualify for. The pools are then divided equally and randomly between each Championship Division. This way, each division receives a more balanced mix of the varying levels of competitiveness.

[Knufire]

Originally Posted by BMSOTM

sometimes the mostly-random assignment of teams to Championship Divisions

I just wanted to interject, do we know for certain this is true? I think it's been a long-time assumption that assignment of a Championship division is pseudo-random, but I don't think we know that for certain. I sort of remember Frank going through how they assign rookies to make sure that there's an equal amount in each division, but I haven't heard anything else other than conjecture about team sorting.

There's plenty of statistical analysis to show how unbalanced the divisions were this past year. However, without knowledge of the algorithm used to generate the divisions, we don't know if it was just random chance or a problem with the algorithm itself (or if competitive balance was even a factor within the algorithm in the first place).

[Ty Tremblay]

Originally Posted by Knufire

I just wanted to interject, do we know for certain this is true? I think it's been a long-time assumption that assignment of a Championship division is pseudo-random, but I don't think we know that for certain. I sort of remember Frank going through how they assign rookies to make sure that there's an equal amount in each division, but I haven't heard anything else other than conjecture about team sorting.

There's plenty of statistical analysis to show how unbalanced the divisions were this past year. However, without knowledge of the algorithm used to generate the divisions, we don't know if it was just random chance or a problem with the algorithm itself (or if competitive balance was even a factor within the algorithm in the first place).

http://www.firstinspires.org/robotic...atch-Schedules

Wondering how we assigned teams to FIRST Championship Divisions this year? First, we assigned Rookies, taking all Rookies signed up for Championship and putting them team by team sequentially in Divisions - one team in Division 1, one team in Division 2, one team in Division 3, one team in Division 4, then back to Division 1 again - in the order in which they registered for CMP, until we ran out of Rookies. This made sure no Division was over- or under-weighted with these least experienced of all FRC teams. Then, we took the Veterans and did the same thing. That’s it.

[Brian Maher]

Originally Posted by Ty Tremblay

http://www.firstinspires.org/robotic...atch-Schedules

Wondering how we assigned teams to FIRST Championship Divisions this year? First, we assigned Rookies, taking all Rookies signed up for Championship and putting them team by team sequentially in Divisions - one team in Division 1, one team in Division 2, one team in Division 3, one team in Division 4, then back to Division 1 again - in the order in which they registered for CMP, until we ran out of Rookies. This made sure no Division was over- or under-weighted with these least experienced of all FRC teams. Then, we took the Veterans and did the same thing. That’s it.

While it was a bit presumptuous of me to assume they're still using exactly the same process as 2014, I think my proposed system would be an improvement over the 2014 system or whatever variation of it they've been using since.

[Knufire]

Originally Posted by Ty Tremblay

http://www.firstinspires.org/robotic...atch-Schedules

Thanks! Didn't know they did the same thing with veterans as they did with rookies.

I definitely think this proposed system has some merit. It's similar enough to the current system (just increasing the number of subgroups teams are divided in instead of just rookie/veteran), but should even out the fields a bit more.

[Nathan Streeter]

I think this has merit and should lead to more balanced divisions... it's reasonably straightforward to implement and doesn't rely on rank or statistics that are subject to change each year.

The one thing that I think FIRST would likely find as a significant sticking point is having the Chairman's and EI (Culture Awards) as the third sort instead of the first. I agree with you that having them third is probably more likely to result in well-balanced divisions from an on-field performance standpoint, but even if the Culture Awards were moved to the first-sort I think this general method would probably still be better than the status quo.

[efoote868]

I think they should do a draft... with previous championship WFA winners each picking divisions.

Could make a draft night show out of it.


EDIT:
And with the 10th round, 86th pick of the draft, Andy Baker selects 3940 for Newton.

[Taylor]

I vaguely remember, perhaps a decade ago, teams were accused of gunning for particular awards and registering for particular regionals because they believed it would get them in a "favorable" CMP division. Does anybody else recall this?
The reason I bring this up is setting the precedent for teams being accused of colluding before the season to end up in the same CMP division, if they knew and could outsmart the placement algorithm.

[Kevin Leonard]

Originally Posted by efoote868

I think they should do a draft... with previous championship WFA winners each picking divisions.

Could make a draft night show out of it.


EDIT:
And with the 10th round, 86th pick of the draft, Andy Baker selects 3940 for Newton.

This is an amazing idea.

I think FIRST hasn't TRIED to balance championship divisions, other than making sure rookies are relatively evenly distributed. If they wanted to, they could do some sort of structure like you're suggesting, Brian, or literally just distribute teams evenly by OPR. It's not too difficult to create even divisions, they just haven't tried.

[Jaci]

I guess I'll weigh in a bit.

Personally, I think the current way divisions are organised is just fine. Sure, this year we got extremely competitive divisions and some less competitive divisions, but that's just the nature of how these things work out.

From what I understand the current (or at least, 2014, judging by the earlier link to FIRST's own website) system for sorting for championships is done in a sort of round-robin style, first with rookies, then with veterans. In this way, it's very hard to introduce bias into the division split. The same would be true for generating division placements by random number. In both ways, there's no 'filter' applied to each division, making it seem like a fair and balanced way to distribute teams.

In the suggestion given by the OP, it seems like they're suggesting much the same system, but breaking it up to more categories than just "rookie/veteran", i.e. round-robin on pools of qualification level (winner, finalist, award, pre-qual, waitlist etc). This is all well and fine, in fact, from face value, it's better than the current system. However, don't stop reading here.

The main problem with ALL methods of distributing teams is that teams aren't individual, but play in alliances. The performance of the whole alliance depends on how each team within that alliance performs, and how they perform together. This is the reason I don't think splitting teams in this way will make a huge difference when looking at the bigger picture, because rankings all come down to alliance allocations in matches. For the sake of argument, let's say it does make a difference, and the (objectively) best teams end up ranked after each other in descending order. During alliance selections is really where things start to kick off. At the end of the day, no matter the distribution, Alliance selections will often follow the same kind of recipe: "Top Team / Top Team #2 / Middle Team / Last Team". When it comes to playoffs within the divisions, only one alliance can be victorious. This gets rid of that team-stacking distribution we saw this year on some divisions. I believe that this is the reason playoffs and alliance selections are the way they are. Now would be a good time to mention that all of this comes down to interpretation. As a final point, playoff matches are where the real competition starts, and qualification matches depend almost entirely on how the alliances are made up on the match schedule.

Through either distribution of teams, I theorize that the same teams would end up on Einstein either way, which is where the competition really amps up to another level.

Again, this is all just my interpretation of each distribution and how I *THINK* it would work out. Don't take it as gospel

[BrennanB]

Here is a question. Do stacked divisions actually matter?

Most people would agree that the threshold of "stacked divisions mattering" would be that they usually win champs because they are "stacked".

Lets look at the data. (Taken from this post on average OPR)

Year	Winner	Prechamps OPR rank
2016	Carver	        4 out of 8
2015	Newton	        4 out of 8
2014	Curie	        4 out of 4
2013	Galileo	        1 out of 4
2012	Galileo	        4 out of 4
2011	Galileo	        2 out of 4
2010	Newton	        4 out of 4
2009	Galileo	        3 out of 4
2008	Galileo	        4 out of 4
2007	Newton	        1 out of 4
2006	Archimedies	4 out of 4
2005	Newton	        4 out of 4
2004	Archimedies	1 out of 4

So for the 4 division era, the average OPR placement of the world champs is 2.91. 8 division avg is 4.

Scaled avg for 8 divisions is an average pre champs OPR rank of 5.54. So... No correlation.

What about how well "Stacked" divisions preformed at champs? (Based on avg opr)

Year	Division	Result
2016	Newton	        QF
2015	Carson	        QF
2014	Archimedes	QF
2013	Galileo	        W
2012	Archimedes	QF
2011	Curie	        F
2010	Archimedes	QF
2009	Newton	        QF
2008	Archimedes	QF
2007	Newton	        W
2006	Newton	        F
2005	Archimedes	QF
2004	Archimedes	W

Assigning 1,2,3,4 for each stage they make it through elims, you get an average round KO of 1.62 meaning on average, the most "stacked" division ended their day in quarters for Einstein.

If someone wants to do avg of the top 24 for each division so we can redo our analysis, but right now your odds don't look particularly good for this whole "division" balancing having any impact. Looks like divisions don't really need to balanced from everything I can see. luck in match schedule and creating a cohesive and competent alliance matters way more.

EDIT

In fact the top 5 most "stacked" divisions 4/5 got KOed in the first round on Einstein. That makes sense. 50% of teams get KOed first round, hence it's at best a random correlation, and too small of a sample size to actually say that the majority of stacked divisions actually perform poorly compared to average divisions.

1	2016	new	QF
2	2015	cars	QF
3	2005	arc	QF
4	2008	arc	QF
5	2013	gal	W

So yeah. No correlation.

[AGPapa]

Originally Posted by BrennanB

Here is a question. Do stacked divisions actually matter?

Most people would agree that the threshold of "stacked divisions mattering" would be that they usually win champs because they are "stacked".

I don't think "most people" would say that, and the OP definitely didn't. He said "improve the Championship experience for everyone", not just the few teams that make Einstein. Many teams want to play with the best in the world, not watch them compete in another division.

[Chris is me]

The implication of your post, Brennan, is that the primary downside of a stacked division is an unfair advantage on Einstein. I don't think that's the main reason at all that unbalanced divisions are a bad thing. It's mainly the worse playing experience playing in an overly strong or weak division, good teams being cut from the elims because they lucked into the strongest division, etc.

Anyways, we have all of this information - anyone invested in this idea could do some programming and spit out 8 hypothetical divisions this year sorted by this algorithm, and then analyze these divisions using different metrics (OPR, etc) to see how unbalanced they are. I'd love to see this sort of analysis.

[BrennanB]

Originally Posted by AGPapa

I don't think "most people" would say that, and the OP definitely didn't. He said "improve the Championship experience for everyone", not just the few teams that make Einstein. Many teams want to play with the best in the world, not watch them compete in another division.

Fair enough, how does balancing divisions help improve the championship experience? Is there a viable way to have elite teams play with elite teams? Doesn't that happen already?

Originally Posted by Chris is me

The implication of your post, Brennan, is that the primary downside of a stacked division is an unfair advantage on Einstein. I don't think that's the main reason at all that unbalanced divisions are a bad thing. It's mainly the worse playing experience playing in an overly strong or weak division, good teams being cut from the elims because they lucked into the strongest division, etc.

Chris brings up a decent point. I can definitely seeing that have an impact on some teams. I wonder how much of that is a hard or easy division and more just people making strange picks during alliance selection? And even if we balance it. Who is to say we will do much better at creating a more balanced division? Perhaps slightly. Robots change. Things happen.

[Knufire]

As Jaci mentioned, since (bar bad scouting) you only have two "top" robots per alliance, stacked divisions don't end up with a stacked alliance on Einstein. Stacked Einstein alliances are almost entirely due to bad scouting by the majority of the division they come from. What they do result in is many stacked alliances in divisional playoffs. Some of those alliances that lose in the semis or finals of a stacked division could be objectively better than alliances that win weaker divisions, and it sucks for those teams to know that they missed out on Einstein just by being unlucky in their assigned division.

What's the opportunity cost for making this change? I've seen some people debating that it would make a positive change, and some arguing that there wouldn't be a significant effect at all. Does anybody thing this would cause a negative change?