Championship Event - Where the "Random" Match Sorting Really "Shines"

[Lil' Lavery]

While the goals of the system may be controversial, and are certainly questionable, there is little question that this algorithm meets those goals. If you look down Ed's generated match list, there are only 2 or 3 matches with a clear cut winner, the rest will all be very competitive. Even the godly alliance of 67, 469, and 1501 won't have a terribly easy time with 74, 498, and 1523. The qualification rounds are well balanced, with both alliances having a good shot to win. While number isn't always a good indicator of the strength of a team, the fact that people are complaining that high numbers teams are being artificially inflated and low numbers are being crushed shows that it often is. As any assumption, there are exceptions. An occasional blow-out match is bound to happen, and I don't think FIRST wanted to completely erase them.
A disturbing bi-product is that it not only creates competitive qualification matches, it creates competitive elimination matches. By having some lesser-skilled teams inflated to the alliance captain status, it dilutes the talent of the pool, and creates parity among the alliances. It greatly reduces the chances that two powerhouse teams will pair up and dominate(once again, there are exceptions). Perhaps this was the motivation of FIRST? It serves the same function as the serpentine draft after all, creating competitive elimination matches, where very few alliances are a clear-cut favorite.

[Ian Mackenzie]

Originally Posted by Tom Bottiglieri

I propose this:

1. Assign all teams a "regional success score". 1 point for a top 8 seed, 3 points for a regional finalist, 5 points for a regional win.
2. Rank all teams based on this score. If teams have the same score, randomize them within this subset.
3. Using the current list ranked "good" to "not so good", assign teams into divisions using the 1,2,3,4 drag and drop method, like previous years.

I really disagree with this - this system effectively punishes teams for doing well at their regionals, and raises the spectre of hyper-competitive teams getting into the Championship by first-come, first-serve (instead of actually qualifying) and then deliberately not doing well at their regionals to get a more favourable spot in Atlanta. (Not saying anybody would do this, but it's still not the sort of thing one wants to encourage.)

On the other hand, if our hypothetical hyper-competitive team doesn't register early enough to get into the Championship, they might be encouraged to go for Chairman's or Engineering Inspiration so they don't 'have' to win a regional

I disagree with any sort of attempted match-balancing at all (either number- or merit-based); you'll certainly get imbalanced blowouts either way (e.g. match 15 in Toronto, 188-1114-1680 vs. 519-1353-1564, 338 to 4), and with some sort of balancing system I think teams get more upset because they have something concrete to be angry at (an imperfect balancing system) as opposed to plain old luck of the draw. Also, I can't think of any other competition which tries to artificially balance matches - if it truly led to more exciting matches, one would think that it would have been done before.

Originally Posted by The Lucas

I suggest pregenerating optimized random matrices (ie check all combinations) with constraints for cycle time for every combination of number of teams at a regional and reasonable number of matches. Then randomize the team list to matrix spots and drop them in the corresponding spots in the matrix.

This is what I've been advocating for a while. 'Check all combinations' is a LOT of combinations, though. I tried to set up match scheduling as a binary programming problem once - effectively an optimized way of checking all combinations - and there are simply too many to check. I think you'll always need some sort of algorithm to generate schedules, but at least if they're pre-generated everyone can see them ahead of time and submit better ones of their own if they want.

[meaubry]

It's really quite simple when you consider that, EVERYONE deserves the opportunity to be paired up with EVERYONE else.

Anything short of being allowed the chance to play WITH some of our longtime friends (that also happen to be in the same grouping of low numbers) is unreasonable and disrespectful to our teams.

If this continues, I can assure you that it will be the end of us registering as Team 47, as we will register as a new team next year (if we return).

Mike Aubry
Engineering Lead Chief Delphi

[Tom Bottiglieri]

Originally Posted by Ian Mackenzie

I really disagree with this - this system effectively punishes teams for doing well at their regionals, and raises the spectre of hyper-competitive teams getting into the Championship by first-come, first-serve (instead of actually qualifying) and then deliberately not doing well at their regionals to get a more favourable spot in Atlanta. (Not saying anybody would do this, but it's still not the sort of thing one wants to encourage.)

Ian,

I'm sorry, but I don't see where you are coming from. I'd really like to believe that NO team would throw a regional just to have a good alliance partner in qualifications in Atlanta.

You're saying that this system punishes teams who perform well at regionals. I can assume you mean 'punish' in the scope that they will paired with "not so good" teams in their division. Now, what makes more sense? Creating equal alliances based on power, or continually punishing teams for having a low team number? If you don't know what I'm talking about, take a look at the match schedule from Boston. Most of the low numbered teams (121, 125, 126, etc) did not have alliances partners with numbers lower than 1500ish. Now, look at the top 4 seeded teams. 1626, 2079, 126, 1511. Notice anything? There you have 3 very good teams who, as you might say, were NOT punished for having a high team number. Were they there on merit, or were they there from the schedule? You make the call.

The best possible solution is to have a totally random match schedule. But obviously, this new algorithm is here to stay. I believe the system I proposed (or something similar.. basically anything that uses power to split teams and not number..) is a good way to make everyone happy in Atlanta. We get our competition, and FIRST gets their co-opertition.

[Corey Balint]

No matter how anyone can try to make this work and try to make a system to make everyone happy, it won't work. There is no way everyone can be happy. Someone will and moan to FIRST about this, and FIRST will try to do something to change it, and more people will and moan about that. There is no freakin' way everyone can be happy.

I would love to see the Championship Event, as a true Championship, and have everything based off merit, but with the way FIRST is set up. Its not gonna happen. Unless the game was 1 v 1, these bad alliance pairings, and the likes are gonna continue to happen. In life, there will always be a flaw, there will always be a weak wheel, its just a matter of what you do with it, and how you can use them to make it all work.
Prime example. No one ever thought a dumper bot would make it to Einstein last year. Look what happened. One fell into 25's laps, and we successfully used them in a strategy few had used during the year, and ultimately, got us as far as we did.

I do hate the current system, and think there is a better way to approach it, but with what FIRST is trying to do, what there goals are, how it is now, is just dandy.

(more to come of my thoughts about this...still deciding/doing a paper)

[Ian Mackenzie]

Originally Posted by Tom Bottiglieri

I'm sorry, but I don't see where you are coming from. I'd really like to believe that NO team would throw a regional just to have a good alliance partner in qualifications in Atlanta.

I agree - as I said, I don't think anybody would actually do it. I'd also like to think that nobody would intentionally push someone into a lower goal last year to get them disqualified, but that doesn't mean that rules that allow that possibility are good ones.

There are also lots of ways to avoid a high "regional success score" without actively throwing a regional, several of which are neither particularly non-gracious or unprofessional. First is to not go to as many regionals in the first place, and concentrate on other things (using the money to build a practice bot, buy nicer parts, do community outreach, etc.). Second is to concentrate on building at a regional - could you really blame a team with a weak arm for taking it off and overhauling it at a regional, even if that meant they spent most of their matches as a box on wheels running around playing defence?

Some of the above is admittedly a bit sensationalist, but in general I worry about what happens when FIRST tries to artificially manipulate things to try to balance veterans and rookies.

Originally Posted by Tom Bottiglieri

You're saying that this system punishes teams who perform well at regionals. I can assume you mean 'punish' in the scope that they will paired with "not so good" teams in their Division. Now, what makes more sense? Creating equal alliances based on power, or continually punishing teams for having a low team number. If you don't know what I'm talking about, take a look at the match schedule from Boston. Most of the low numbered teams (121, 125, 126, etc) did not have alliances partners with numbers lower than 1500ish. Now, look at the top 4 seeded teams. 1626, 2079, 126, 1511. Notice anything? There you have 3 very good teams who, as you might say, were NOT punished for having a high team number. Were they there on merit, or were they there from the schedule? You make the call.

Again, I don't really like either system, so I'm hesitant to argue for one over the other. I personally don't think any teams need a lot of balancing - in most things in life, one can't simply jump in and immediately be on anywhere near a level standing with the experienced veterans, and I'm not sure FIRST needs to be any different. I think most rookies can accept that they might not do so well in their first year or two, but that just makes it that much more of an accomplishment when they finally 'make it'. I'm worried that artificially helping rookie or low-performing teams could have the opposite effect when teams manage to take their robot and team to the next level and then realize that it hasn't really helped them that much.

As a bit of an aside, how do you propose doing the schedule for regionals if you use regional data to make the schedules for the Championship? Use the previous year's data? Default back to numbering?

Originally Posted by Tom Bottiglieri

The best possible solution is to have a totally random match schedule. But obviously, this new algorithm is here to stay. I believe the system I proposed (or something similar.. basically anything that uses power to split teams and not number..) is a good way to make everyone happy in Atlanta. We get our competition, and FIRST gets their co-opertition.

We're certainly in agreement that a totally random match schedule is the best solution. I don't think that the new system is necessarily here to stay, though; I think FIRST has done a pretty good job over the years of letting good decisions stand and correcting bad ones. (In my opinion, the move to alliances and the addition of bumpers were good decisions that were kept; the move to 4-vs.-0 was a bad decision that was reversed. On the other hand, the serpentine draft is still around.)

[Jacob Plicque]

Let's look at the results instead of talking about random matches. At the Florida Regional 5 midrank or junior teams (1902-2nd, 1027-3rd,1251-4th, 801-6th, 1270-7th) were among the top 8. The current algorithm assumes that a junior team will have a weak unit while a veteran team will produce a strong robot. If that were true, then 5 or six veteran team should be in the top 8. Look also at the finals 1902 (with 108, 179, & 365-replacement) was a finalist while 1251 and 1270 were winners with Team 86. The midrank teams were not weak if they were the champions. The difference was having a practical strategy and employing a good scouting system that allowed 1251 to create a strong alliance.
There are only three kinds of fair - county fair, state fair & worlds fair. All pairing systems have a bias but the cream will rise to the top. We were a low number team (86 - Rank 21/51) with only a 4W- 4L record partly because of pairings with three no show teams who did not get to thru their inspections in time for the match. However, we could have gotten these no shows with any other pairing system. I blame the regional officials for not providing a substitute or rescheduling these matches. I would prefer a more random system of matches but a winner succeeds in all cases with good design, determination, desire, discipline, and dedication. I have not checked other regionals but I am aware that Team 45 had a lot of pairing problems and went on to win a regional.

[Lil' Lavery]

Originally Posted by Jacob Plicque

Let's look at the results instead of talking about random matches. At the Florida Regional 5 midrank or junior teams (1902-2nd, 1027-3rd,1251-4th, 801-6th, 1270-7th) were among the top 8. The current algorithm assumes that a junior team will have a weak unit while a veteran team will produce a strong robot. If that were true, then 5 or six veteran team should be in the top 8.

That statement isn't quite true. With the current scheduling system, the lower numbered teams each face a [theoretically] harder schedule as they are never paired with another low numbered team, and will typically have to carry their alliance. Thus why low numbered teams rarely emerge captains. You cannot use evidence tampered by this system to advocate against it.

[Salik Syed]

Why can't we have a dynamic team sorting algorithm? Start out randomly then model team sucess rates and divide teams based on how well they actually perform as the day progresses! Weight wins and losses based on the "power" value of the opponents so that the system is even more accurate. Create alliances so the "power" value of each alliance is relatively similar. Add in a little randomness factor to keep things interesting.

We could extend this -- create an 3X3 matrix
Set up a fixed equation which determines a probablity value of a team ( with "power" value X) winning against another team (of power Y) if played head to head.
Plug in these values into the matrix. If teams are evenly matched the determinant should be close to 0. (right??)

This still doesn't take into account the fact that 3 ramp bots together will probably not do well even if they are very powerful from a statistical standpoint (lots of wins)



what do y'all think?

[Vogel648]

My problem is that SOS is not taken into account when determining the top 8 for aliances so it seems that all teams should have a similar SOS optimally so to have the top 8 teams picking. Either this or have the standings based on an RPI like system.

[Jacob Plicque]

Originally Posted by Lil' Lavery

That statement isn't quite true. With the current scheduling system, the lower numbered teams each face a [theoretically] harder schedule as they are never paired with another low numbered team, and will typically have to carry their alliance. Thus why low numbered teams rarely emerge captains. You cannot use evidence tampered by this system to advocate against it.

Read my words carefully. I stated that all systems are unfair to one group or another. I agree that low number teams have to work harder to win under the current match system. My thesis was as follows:
a) 1902, 1251 and 1270 were strong teams since they reached the finals
b) the current match pairing system incorrectly assumes they are weak
which gives them a stronger team in the round robins
c) good scouting will find good teams for alliances - #4 at LA, #45, & #86
even if they do not have high ranks.

[julivan]

time 1860 indo para atlanta, (from brasil)para Championship...


la vamos nós.......!!!!!!

aguarde-nos.

[Ian Mackenzie]

As another thought, if FIRST is determined to some sort of balancing, perhaps there could be a compromise. Instead of splitting teams into three tiers and having each alliance be one robot from each tier, there could be a looser restriction that every alliance have robots from at least two different tiers (or, equivalently, that no alliance may have three robots from the same tier). This would make it possible for every team to play with and against every other team, but still prevent situations like 3 veterans vs. 3 rookies.

More drastically, the restriction on what teams can be on any one alliance could be dropped, but a restriction could be added that the two alliances be balanced relative to one another - so one could have 3 veterans vs. 3 veterans or 3 rookies vs. 3 rookies, but not 3 veterans vs. 3 rookies.

Actually, on second thought, while that might get some really exciting matches with 6 great robots, there would also be some pretty boring matches with 6 mediocre robots. I still like a completely random schedule, with the emphasis being on reducing duplication (so one team doesn't see any other team more than a couple times) instead of trying to artificially balance matches, and trusting that with enough matches and enough variety in partners and opponents, things will average out.

[Francis-134]

It is apparent that FIRST's methodology in this algorithm is the following:

Teams of a lower number will generally outperform teams of a higher number, specifically very low numbers (generally less than 400) will do better than teams between 400 and 1700, and both will do better than teams greater than 1800.

Therefore: teams above 1700 are at a statistical disadvantage to those with team numbers less than 1700, and those between 400 and 1700 have a disadvantage to those numbered below 400.

Similarly: teams above 1700 deserve a chance to not only play with teams of a lower team number, but also deserve the opportunity to win more matches with these lower number teams. Also, teams of numbers less than 1700 should not have to play with two robots of a much lower statistical advantage, nor should they play against two or more robots of a much lo wer statistical advantage

Therefore: a system must be implemented in order to even the playing field such that no alliance will have more than one robots from the same team number range. Doing this will theoretically spread out the pairings such that no alliance faces a blowout. For example, three veterans against three rookies, or even one veteran with two rookies against three mid-level teams.

Because of this, a ranking system must be created that will split teams into these three groups, and make sure that every alliance has a team from each of the three categories. Indeed, some robots will never face each other, but this is an acceptable loss so that no team will be statistically, based on team number, outmatched.

In my opinion, each of these points is somewhat flawed. First and foremost, team number is not an indicator of the team's potential performance. Instead of beating a dead horse, I'll simply name teams like 1902, 1114, 1503 and 1680 as examples. The statistical disadvantage of rookies and the advantage that veterans have is a fallacy.

I'm going to sound cold when I say this, but rookies do not intrinsically deserve to be paired with veteran teams? No. Should they? Yes. Will they? More than likely they will, no matter the algorithm. Must they (in every match)? No. Rookies and young teams need help, but artificially forcing them with veterans will not do the trick. How are they to learn about FIRST if they are forever cast into the role of not playing as the "lead team". If you think about it, FIRST would have to grow enormously in order for rookies to become the "veteran" team on an alliance in the future (essentially, triple in size).

Should every alliance be statistically fair? In my opinion, no. The randomish system of previous years would indeed cause blowout matches, but at least you knew that it was fate that caused it, not some master system that some individuals believe is better for you and your alliance members.

Should teams be grouped into three or even six or nine groups? No, I don't think this is a good idea. To be frank, FIRST is grouping teams into the good, the alright, and those that are not so hot. It is stereotyping teams, in a sense. If you are in the bottom third of the team numbers at your event, you are preconceived to be of a lower caliber than two-thirds of the other teams at the regional.

I do understand that it's "not about the robot", and that the competition is merely the means to an end (an end that I most whole-heartedly agree with), but if we cannot have faith in how things are to go at a competition, how can you focus on making science and technology be cool? Maybe it's because I'm from New Hampshire, but I think that the less meddling FIRST does, the better. Why make rules about things that teams should be making or learning about on their own? Why change things that aren't actually broken?

What saddens me the most is that if this system continues, I will most likely never get the opportunity to play with my old teams. Anyone who has been on more than one teams knows how great is is to see that you will be playing with your old friends and mentors, working on strategy together, and enjoying the thrill or disappointment of victory or defeat once again, just like the "old days". However, both of my previous teams (134 and 40) will always be in the same grouping as my current team (190). The only way we would get to play together is if we were picked in the elimination rounds, something that becomes more unlikely if a high number team rides their alliance partners to the top 8. In closing, I really hope FIRST fixes this blunder (there is no question that it is a blunder, in my opinion). I'm not going to leave FIRST over this, but for the good of the program, I wish that cool, calm, and thoughtful heads prevail in the discussion that is sure to occur at some point at FIRST HQ.

[GaryVoshol]

FIRST has decided, with the help of "Bob the FTA", that a tiering system is desired. That decision may be suspect, but it has a noble goal, to even out competition. It is similar to the serpentine draft.

The problem is deciding which teams get into which tiers. Team number is not the way to go. If you go to a regional with few rookies, the 3rd tier gets distorted - such as including 1114 at GLR. At the other end, there are teams with very low numbers who have a poor year, or who have had a long-term decline due to various factors (losing sponsors or mentors comes to mind) and are just shells of their former glory. When a team signed up for FIRST does not indicate how well or poorly that team will perform.

We need a pre-defined tournament match structure, something like
Round 1, Match 1 - Teams A/B/C play teams D/E/F
Match 2 - Teams G/H/I play teams J/K/L
Etc.

Rounds 2 through X are defined with constraints - no multiple partners / opponents, maximize time between rounds, etc. Once a program is written that can do these permutations, it can be run for any number of teams in a regional or Championship division, from 24 to 100. Now we have defined match schedules, and all that remains is to randomly assign team numbers to the A, B, C, D ... positions in the schedule.