Championship Event - Where the "Random" Match Sorting Really "Shines"

[Travis Hoffman]

So now that everyone has had a chance to experience the joy and fun that is the patently-horrible match sorting algorithm at the regionals (why can't we have a truly random algorithm implemented for the Championship again? - We've had five weeks to work on one - is this parasitic algorithm that much bored into the flesh of the otherwise great scoring software that it takes more time than that to dig it out?), let's take a theoretical look at one of the as-yet unsettled divisional team lists for the Championship and see how the teams break down. I took this from the most recent Divisional List posted in Koko Ed's 2007 Championship Divisions thread.

Newton Team Categories
 
V = Veteran
M = Mid-Level
Y = Young  
 
V    M    Y
20  357  1305
34  365  1341
48  395  1418
60  456  1502
67  469  1510
74  498  1523
86  527  1535
97  558  1563
108 612  1617
121 648  1647
126 694  1675
141 716  1700
159 771  1714
176 830  1732
180 862  1811
195 910  1850
224 967  1872
234 1001 1929
246 1015 2010
269 1033 2046
291 1087 2067
314 1114 2100
330 1137 2137
339 1155 2182
343 1218 2252
   1270 2283

That veteran list is packed with a much higher quantity of successful 2007 teams relative to the other 2 lists, so if this division were actually finalized, "relatively inexperienced" teams like 365, 469, and 1114 would be ecstatic to reap the bountiful harvest of favorable qualifying round matchups coming their way.

365 is right on the edge of entering the "veteran" category, lowering their odds of a favorable qualifying match list. Any more higher-numbered teams signing up for the Championship and entering Newton severely jeopardizes their favorable status as a mid-level team. How messed up is a system that could potentially have a team's mentors hoping for FEWER higher-numbered teams signing up for the Championship? I know everyone on MOE is above that kind of thinking (congrats on Philly Chairman's!), but just the fact that a FIRST-sponsored algorithm could potentially introduce anti-FIRST thoughts in teams, well it's just so BLAH. BLAH in action - there are a few strong mid and high-number teams who have admitted to me they like the extra artificial "kick" this algorithm gives to their chances during qualifying. It's a guilty pleasure for them, I think. I can't necessarily blame them for feeling that way - it's hard to resist denouncing something that is to your benefit. The low-numbered vets see this and are quietly frustrated.....

WE WANT AS MANY TEAMS AS POSSIBLE TO ATTEND THESE EVENTS, AND WE WANT TO KNOW THE MATCHUPS WILL BE AS RANDOM AS POSSIBLE!

The odds of quality mid-level and high-numbered teams (who need and should receive no artificial help) seeing quality partners at the Championship during qualifying will be much greater relative to what they experienced at the regionals, and it was already bad enough at the regionals.

It will be interesting to see just how many lower-numbered teams reach the finals on Einstein.....

Could anyone with insider knowledge report on the current status of FIRST's efforts to correct this self-inflicted gaping flesh wound? I expect the Championship Team Forums to be filled with comments from team representatives who are vehemently opposed to the current algorithm. If FIRST wanted to minimize this type of chatter to provide more time for other less-publicized topics of discussion, I'd suggest they send out a communication prior to the Championship explaining what algorithm will be in place in Atlanta. If the same algorithm will be used, at least people will know. I'd also like a verbal commitment from them that this abomination will be purged from the program in 2008 before it comes time for veteran-numbered teams to start deciding how much money they will spend on official FIRST events next year.....

As a final thought, what are the chances/harm of FIRST exposing this software to other qualified professionals on teams throughout the community? If you admit there is a problem but feel you don't have enough time and could use some more manpower to revise it, then I'm sure many in the community would love to come to your aid to help iron out the wrinkles in this stuff.

[Bongle]

One easy solution to the tiering problem: Rather than split the teams into 3 tiers based on a sorted order, just randomize your input list so that the three groups will be approximately equal in quality. You would still have the problem of never playing with anyone in your tier as an ally, but at least you wouldn't have the big helping of rookies and hurting of veterans.

So rather than

1 4 7
2 5 8
3 6 9

You might have lists like

1 6 5
3 9 4
7 2 8

On average, they will even out to have the same number of powerhouse teams.

The whole tiering thing seems to be a wierd solution to a problem that didn't really exist anyway. Now instead of a few teams sometimes legitametely complaining that they got a bad deal in the pairings, you've got a whole group (the veteran teams) who get a bad deal, and they can prove it.

[The Lucas]

Originally Posted by Travis Hoffman

That veteran list is packed with a much higher quantity of successful 2007 teams relative to the other 2 lists, so if this division were actually finalized, "relatively inexperienced" teams like 365, 469, 1114, 1305 should be ecstatic to reap the bountiful harvest of favorable qualifying round matchups coming their way.

We (365) have been at the first team in the M pool at every regional this year. I looked into it after FLR and I knew a week in advance going into Philly. I think this "Team Number Fairness" algorithm is most unfair. I probably was the most vocal critic of the algorithm at Philly (I talked the scorekeeper's ear off and told pretty much anyone else who would listen). I noted the partner repetition, cycle (some teams got only 1 match off between matches), and the unfair advantage received by teams (like mine) simply based on their team number (not even years of experience). In another odd twist of the algorithm, the first teams in every pool seem to always have the first match generally again the second team in every pool. Is this meant to showcase the "algorithm determined Best teams". At Philly, the first cycle of the Match List literally went:

first in every pool vs second in every pool
third vs fourth
5th vs 6th
etc... for basically the WHOLE first cycle, so it didn't even try to "randomize" the first cycle

However your list is slightly wrong. The V pool is the last to fill up so 357 is currently at the top of the M pool. I know this because this is the only reason 365 was always in the M pool. If one team hadn't dropped out (tragic story I don't want to share w/o their permission) from Philly we would have had 45 teams (equally divisible by 3) and 365 would be at the end of the V pool. Same thing at FLR if it had 36 teams instead of 35 assuming the 36th team had # > 365.

I know FIRST probably will not fix this by Championship, but I hope they will by next year. I suggest pregenerating optimized random matrices (ie check all combinations) with constraints for cycle time for every combination of number of teams at a regional and reasonable number of matches. Then randomize the team list to matrix spots and drop them in the corresponding spots in the matrix.

[Travis Hoffman]

Originally Posted by Steve W

removed

Steve:

I do not wish to assign blame, nor do I want this thread to devolve into something equally as BLAH as the algorithm is - I simply wish that FIRST will release a public update on this matter so we know what is going on behind the scenes to address it. I don't care who created the parasite; I just want to find out if someone is currently developing the cure. I don't want teams to waste time railing on about the algorithm at the Team Forums if there are already plans in place to correct the matter - believe me, teams already have a huge laundry list of items they wish to cover, and it would be great if they could check one off the list prior to attending. If there AREN'T any FIRST-laid plans to correct the algorithm, however.......well, it won't be pretty. This is why it's so important for them to COMMUNICATE WITH US - to dispel any rumo(u)rs and set the record straight.

[Travis Hoffman]

Originally Posted by The Lucas

However your list is slightly wrong. The V pool is the last to fill up so 357 is currently at the top of the M pool. I know this because this is the only reason 365 was always in the M pool. If one team hadn't dropped out (tragic story I don't want to share w/o their permission) from Philly we would have had 45 teams (equally divisible by 3) and 365 would be at the end of the V pool. Same thing at FLR if it had 36 teams instead of 35 assuming the 36th team had # > 365.

Fixed in the original post - thanks. 357 - welcome to the mid-level euphoria. And 1305, well you guys are just flipping out right now, you inexperienced youngins, you . To now have the chance at seeing the algorithm put you with that somewhat-inexperienced 1114 all day long? Go Canada.

I know FIRST probably will not fix this by Championship, but I hope they will by next year. I suggest pregenerating optimized random matrices (ie check all combinations) with constraints for cycle time for every combination of number of teams at a regional and reasonable number of matches. Then randomize the team list to matrix spots and drop them in the corresponding spots in the matrix.

Thank you for the honest and respectful feedback and commentary.

[JudyVandy]

Without any background info on the above division into V, M and Y, I'm going to take issue with the numerical break points. Most of the under 1000 numbers were generated by order of registration during a year when there were only about 600-800 teams. Prior to that point, each team received a new number each year.

To truly apply these descriptions to present teams, you'd have to do a bit of research into the teams given these numbers to see when they really began operation.

[Travis Hoffman]

Originally Posted by JudyVandy

Without any background info on the above division into V, M and Y, I'm going to take issue with the numerical break points. Most of the under 1000 numbers were generated by order of registration during a year when there were only about 600-800 teams. Prior to that point, each team received a new number each year.

To truly apply these descriptions to present teams, you'd have to do a bit of research into the teams given these numbers to see when they really began operation.

Judy:

Your comments are exactly those shared by so many - team number is not an effective indicator of team quality.


For anyone who might not be aware of how all this works....

It has been proven by many that this Veteran, Mid-Level, Young categorical breakdown based on team number is exactly what is used as the basis for the match generating algorithm for the 2007 regionals:

• A team is guaranteed to never be allied with anyone within their own category during qualifying.
• Given the complete team list, you can predict with 100% accuracy the bulk of the 1st round of qualifying matches at any event - I've personally predicted this successfully at Buckeye and GTR. All you have to do is go down the columns of the correctly-sorted category list and split off every 2 rows into individual matches (see below).
• It's a safe bet that the 2 lowest-numbered teams at an event/division will play against each other in Match #1 of the event. 48 has had the "pleasure" of going first at both Buckeye and GTR. While the 1st round match list pairings at Pittsburgh followed the above prediction procedure, we weren't in the first match there - we were in the 3rd match - perhaps the regional staff was finessing the ordering of the matches around a bit?

1st Round Match List for Theoretical Newton Division 
(Listed in 1st Thread Post)
 
Match #1
20  357  1305
34  365  1341
 
 
Match #2
48  395  1418
60  456  1502
 
 
Match #3  
67  469  1510  
74  498  1523
 
Match #4
86  527  1535
97  558  1563
 
Match #5
108 612  1617
121 648  1647
 
 
Match #6
126 694  1675
141 716  1700
 
 
Match #7
159 771  1714
176 830  1732
 
Match #8
180 862  1811
195 910  1850
 
Match #9
224 967  1872
234 1001 1929
 
Match #10
246 1015 2010
269 1033 2046
 
Match #11
291 1087 2067
314 1114 2100
 
Match #12
330 1137 2137
339 1155 2182
 
Match #13
343 1218 2252
??? 1270 2283

[OZ_341]

I think we all know low numbered teams that consistently struggle from year to year. We also have seen high numbered teams that become instant powerhouses.

An alogrithm based on team numbers makes the assumption that time is the only factor in team success. As we all know it is one of many factors.

Hopefully this will change for 2008.

[johnr]

i may not express this right,but no matter what first does it will still come down to "luck of the draw". you will never make everyone happy, unless you plan to extend championships a few days to play every possible combination. or make sure there is a climb-able ramp on each team each match, since that seems to be the most important thing this year. or if you score keeper that would multiply your rack score by 2. or,or ,or. when all is said and done it will be the quality of your robot, your ability to sell yourself and other teams scouting that will get you to the big show.

[BandChick]

I'm assuming this is just a typo, but team 1089 isn't listed in your original post (and I know we're going). ~__^ I'm assuming we'd be a "mid-level" team, since all the numbers surrounding us are.

I agree with Alan, there are a LOT of factors that contribute to teams building a successful machine (and for that matter, drive team). I honestly think numbers play a small, if not insignificant role.

[Travis Hoffman]

Originally Posted by johnr

i may not express this right,but no matter what first does it will still come down to "luck of the draw". you will never make everyone happy, unless you plan to extend championships a few days to play every possible combination. or make sure there is a climb-able ramp on each team each match, since that seems to be the most important thing this year. or if you score keeper that would multiply your rack score by 2. or,or ,or. when all is said and done it will be the quality of your robot, your ability to sell yourself and other teams scouting that will get you to the big show.

You can go out and perform well individually during qualifying regardless of match pairings, as we've been fortunate enough to do at Buckeye and GTR, but your chances of getting picked are directly affected by the quality of the scouting the Top 8 seeds employ. More youngins in the Top 8 may increase the risk of poorly-assembled team pick lists. We've also been fortunate in that area at our regionals this year, as have many other veterans who've finished outside the top 8 who've been picked by the top seeds.

However, these events aren't only played on Saturday. The overall experience can be greatly enriched if you get to play with an against a greater of variety of teams during qualifying on Friday. Right now, a lot of lower-numbered teams are artificially being forced into less than fun situations on Friday.

If FIRST wants to find a way to balance alliance pairings based upon team quality, I'm game, if they can find a way to make it WORK - I've actually suggested it in other threads in the past. However, the method they've chosen to implement that desire in 2007 is just plain ineffective and misguided. If a new and truly effective method can't be found, then going back to a plain old random algorithm is the way to go.

[Travis Hoffman]

Originally Posted by BandChick

I'm assuming this is just a typo, but team 1089 isn't listed in your original post (and I know we're going). ~__^ I'm assuming we'd be a "mid-level" team, since all the numbers surrounding us are.

I agree with Alan, there are a LOT of factors that contribute to teams building a successful machine (and for that matter, drive team). I honestly think numbers play a small, if not insignificant role.

As I stated in the original post, the division list I used was an intermediate one pulled from the 2007 Championship Divisions thread - it is used as an example only. The final divisional lists will undoubtedly be different. I will post my categorical breakdowns and 1st round match list predictions for all divisions once they are finalized and announced.

[OZ_341]

Originally Posted by johnr

i may not express this right,but no matter what first does it will still come down to "luck of the draw". you will never make everyone happy .............

I can verify this sentiment. We are never happy about our match pairings.

I guess we are talking about how this algorithm increases our level of overall unhappiness! The goal is always complete satisfaction, but in the end we are trying to control our level of unhappiness.

[Tom Bottiglieri]

I think this match generation model could work, given the proper set of input data. Since there has been no official statement from FIRST on this, all we can do is assume their intentions. So, I am assuming the point of this algorithm was to match "better" teams with teams who are just getting on their feet.

I don't disagree with this methodology for qualification rounds. It makes for less blowouts, and a more consistent and exciting set of matches. The problem comes with drawing the border between "good" and "not so good" teams. Regionals have been broken up into three categories (veteran, mid level, and rookie) based on team number, which is a very rough implication for experience. Due to this, alot of "good" mid level and rookie teams are getting paired with other "good" veterans. While this happens, "not so good" rookies and mid level teams are getting paired with other "not so good" veterans, making the original point of the algorithm completely null and void.

Travis has already shown how the current method of creating divisions breaks the match creation model. Ranking random teams by number will hurt more than it will help. If, on the other hand, the splitting up of "veterans", "mid level", and "rookie" teams was based on merit and not an arbitrary number, the system would work. So, for the Championship, I would love to see the use of the regional results in creating the divisions.

I propose this:

1. Assign all teams a "regional success score". 1 point for a top 8 seed, 3 points for a regional finalist, 5 points for a regional win.
2. Rank all teams based on this score. If teams have the same score, randomize them within this subset.
3. Using the current list ranked "good" to "not so good", assign teams into divisions using the 1,2,3,4 drag and drop method, like previous years.

Boda Bing! We're one step closer to parity. This assignment is something that can whipped up in excel in about 10 minutes. Now, this exact method may not be perfect. There will still be some flukes. But I can guarantee you it will work at least 10 times better than what we assume the division assignment system will be.

The bottom line is that you cannot split teams up based off of implied "experience". If you really want to create a tiered match generation system, you will need to base it off of true power.

This is the Championship event. The best of the best. Everyone wants to see the best teams on Einstein. Let's not let Rack n' Roll fall to a fluke.

[Steve Kaneb]

There is a problem with the fundamental assumption that young and rookie teams directly benefit from being in matches with very veteran teams.

Young teams will get as much experience from being paired up with a veteran in a couple matches as being paired with any veteran for every match.

This algorithm performs its function perfectly but the function is based on incorrect assumptions. The basic goal, to have young teams paired with veteran teams, doesn't further the competition or the teams in any way. Teams do not get a good feeling of the way that robots work together if their possible pairings are limited. Some teams have no chance of ever meeting others because of these split pools.

Isn't it better for the good rookies and young teams if they are the ones carrying their alliance in a randomly paired match? They'll be lauded, and rightfully so. The problem with having the same teams playing each other is that you see several matches that play exactly the same: one good team scores well because they told one of the other teams to defend the good team they've faced twice already. The chances that there is only one good team on an alliance are much greater with a split pool.

Secondly, the current match pairing hurts the standings. If there is a dominant team at a regional, chances are very high that they will be first ranked, because there's a smaller chance that there will be two excellent robots going against it in a qualification round. What happens is that standings get changed because the same basic matches are being played over and over again. A good team won't be able to consistently beat a very good team with two robots defending them every round.

From these pairings, little strategy evolves. An excellent team will just have its partners play defense or interference, and those teams that aren't as good at scoring will never get the chance to get better because they will be playing defense in almost all of their matches.

Young teams have found themselves pushed up into the standings because they were on good alliances. Some of these teams don't have the resources to be a picker, sometimes they don't have enough people, or they just don't have the experience of strategy. This propels the better teams further into the elimination rounds because they are able to exploit alliances that were crafted without specific strategy in mind. Young alliances in the finals might find themselves without a strategy at all because they haven't had to strategize throughout the qualification rounds.

Teams new to the competition will get to see the one or two teams beat up on the competition, pick the best other team, and romp through the finals. It gives good rookies less of a chance to be on one of these alliances, because now they're a captain.


Splitting teams into pools doesn't lead to more learning, innovation or strategy. It doesn't make teams feel better about themselves. It doesn't allow for matches with an alliance of veterans getting beaten by a bunch of young teams. Wouldn't we all like to see a match where one of the teams that has been consistently doing well has to squeak out a win against three rookies? We don't get to see any surprising upset matches where you don't get to predict what will happen. The same teams keep winning in qualifications, and you have to wait for eliminations before they see different teams. It's not like eliminations alliances have changed much.

A random pairing gets rookies on alliances with veteran teams, sometimes as often as every match. It allows for teams to be with or against teams that they didn't see last match. Teams are exposed more to different robot designs and strategies. Some strategies evolve in the middle of a match where you find yourself playing people you hadn't seen before.

Doesn't it further FIRST to have many different matches in qualifications rather than the same 2 or 3?