Championship Event - Where the "Random" Match Sorting Really "Shines"
 

So now that everyone has had a chance to experience the joy and fun that is the patently-horrible match sorting algorithm at the regionals (why can't we have a truly random algorithm implemented for the Championship again? - We've had five weeks to work on one - is this parasitic algorithm that much bored into the flesh of the otherwise great scoring software that it takes more time than that to dig it out?), let's take a theoretical look at one of the as-yet unsettled divisional team lists for the Championship and see how the teams break down. I took this from the most recent Divisional List posted in Koko Ed's 2007 Championship Divisions thread.


That veteran list is packed with a much higher quantity of successful 2007 teams relative to the other 2 lists, so if this division were actually finalized, "relatively inexperienced" :rolleyes: teams like 365, 469, and 1114 would be ecstatic to reap the bountiful harvest of favorable qualifying round matchups coming their way.

365 is right on the edge of entering the "veteran" category, lowering their odds of a favorable qualifying match list. Any more higher-numbered teams signing up for the Championship and entering Newton severely jeopardizes their favorable status as a mid-level team. How messed up is a system that could potentially have a team's mentors hoping for FEWER higher-numbered teams signing up for the Championship? I know everyone on MOE is above that kind of thinking (congrats on Philly Chairman's!), but just the fact that a FIRST-sponsored algorithm could potentially introduce anti-FIRST thoughts in teams, well it's just so BLAH. BLAH in action - there are a few strong mid and high-number teams who have admitted to me they like the extra artificial "kick" this algorithm gives to their chances during qualifying. It's a guilty pleasure for them, I think. I can't necessarily blame them for feeling that way - it's hard to resist denouncing something that is to your benefit. The low-numbered vets see this and are quietly frustrated.....

WE WANT AS MANY TEAMS AS POSSIBLE TO ATTEND THESE EVENTS, AND WE WANT TO KNOW THE MATCHUPS WILL BE AS RANDOM AS POSSIBLE!

The odds of quality mid-level and high-numbered teams (who need and should receive no artificial help) seeing quality partners at the Championship during qualifying will be much greater relative to what they experienced at the regionals, and it was already bad enough at the regionals.

It will be interesting to see just how many lower-numbered teams reach the finals on Einstein.....

Could anyone with insider knowledge report on the current status of FIRST's efforts to correct this self-inflicted gaping flesh wound? I expect the Championship Team Forums to be filled with comments from team representatives who are vehemently opposed to the current algorithm. If FIRST wanted to minimize this type of chatter to provide more time for other less-publicized topics of discussion, I'd suggest they send out a communication prior to the Championship explaining what algorithm will be in place in Atlanta. If the same algorithm will be used, at least people will know. I'd also like a verbal commitment from them that this abomination will be purged from the program in 2008 before it comes time for veteran-numbered teams to start deciding how much money they will spend on official FIRST events next year.....

As a final thought, what are the chances/harm of FIRST exposing this software to other qualified professionals on teams throughout the community? If you admit there is a problem but feel you don't have enough time and could use some more manpower to revise it, then I'm sure many in the community would love to come to your aid to help iron out the wrinkles in this stuff.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

One easy solution to the tiering problem: Rather than split the teams into 3 tiers based on a sorted order, just randomize your input list so that the three groups will be approximately equal in quality. You would still have the problem of never playing with anyone in your tier as an ally, but at least you wouldn't have the big helping of rookies and hurting of veterans.

So rather than
You might have lists like
On average, they will even out to have the same number of powerhouse teams.

The whole tiering thing seems to be a wierd solution to a problem that didn't really exist anyway. Now instead of a few teams sometimes legitametely complaining that they got a bad deal in the pairings, you've got a whole group (the veteran teams) who get a bad deal, and they can prove it.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

We (365) have been at the first team in the M pool at every regional this year. I looked into it after FLR and I knew a week in advance going into Philly. I think this "Team Number Fairness" algorithm is most unfair. I probably was the most vocal critic of the algorithm at Philly (I talked the scorekeeper's ear off and told pretty much anyone else who would listen). I noted the partner repetition, cycle (some teams got only 1 match off between matches), and the unfair advantage received by teams (like mine) simply based on their team number (not even years of experience). In another odd twist of the algorithm, the first teams in every pool seem to always have the first match generally again the second team in every pool. Is this meant to showcase the "algorithm determined Best teams". At Philly, the first cycle of the Match List literally went:

first in every pool vs second in every pool
third vs fourth
5th vs 6th
etc... for basically the WHOLE first cycle, so it didn't even try to "randomize" the first cycle

However your list is slightly wrong. The V pool is the last to fill up so 357 is currently at the top of the M pool. I know this because this is the only reason 365 was always in the M pool. If one team hadn't dropped out (tragic story I don't want to share w/o their permission) from Philly we would have had 45 teams (equally divisible by 3) and 365 would be at the end of the V pool. Same thing at FLR if it had 36 teams instead of 35 assuming the 36th team had # > 365.

I know FIRST probably will not fix this by Championship, but I hope they will by next year. I suggest pregenerating optimized random matrices (ie check all combinations) with constraints for cycle time for every combination of number of teams at a regional and reasonable number of matches. Then randomize the team list to matrix spots and drop them in the corresponding spots in the matrix.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Steve:

I do not wish to assign blame, nor do I want this thread to devolve into something equally as BLAH as the algorithm is - I simply wish that FIRST will release a public update on this matter so we know what is going on behind the scenes to address it. I don't care who created the parasite; I just want to find out if someone is currently developing the cure. I don't want teams to waste time railing on about the algorithm at the Team Forums if there are already plans in place to correct the matter - believe me, teams already have a huge laundry list of items they wish to cover, and it would be great if they could check one off the list prior to attending. If there AREN'T any FIRST-laid plans to correct the algorithm, however.......well, it won't be pretty. This is why it's so important for them to COMMUNICATE WITH US - to dispel any rumo(u)rs and set the record straight.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Fixed in the original post - thanks. 357 - welcome to the mid-level euphoria. And 1305, well you guys are just flipping out right now, you inexperienced youngins, you :rolleyes: :) . To now have the chance at seeing the algorithm put you with that somewhat-inexperienced 1114 all day long? Go Canada. :rolleyes: :rolleyes: :p

Thank you for the honest and respectful feedback and commentary.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Without any background info on the above division into V, M and Y, I'm going to take issue with the numerical break points. Most of the under 1000 numbers were generated by order of registration during a year when there were only about 600-800 teams. Prior to that point, each team received a new number each year.

To truly apply these descriptions to present teams, you'd have to do a bit of research into the teams given these numbers to see when they really began operation.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Judy:

Your comments are exactly those shared by so many - team number is not an effective indicator of team quality.


For anyone who might not be aware of how all this works....

It has been proven by many that this Veteran, Mid-Level, Young categorical breakdown based on team number is exactly what is used as the basis for the match generating algorithm for the 2007 regionals:

• A team is guaranteed to never be allied with anyone within their own category during qualifying.
• Given the complete team list, you can predict with 100% accuracy the bulk of the 1st round of qualifying matches at any event - I've personally predicted this successfully at Buckeye and GTR. All you have to do is go down the columns of the correctly-sorted category list and split off every 2 rows into individual matches (see below).
• It's a safe bet that the 2 lowest-numbered teams at an event/division will play against each other in Match #1 of the event. 48 has had the "pleasure" of going first at both Buckeye and GTR. While the 1st round match list pairings at Pittsburgh followed the above prediction procedure, we weren't in the first match there - we were in the 3rd match - perhaps the regional staff was finessing the ordering of the matches around a bit?

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I think we all know low numbered teams that consistently struggle from year to year. We also have seen high numbered teams that become instant powerhouses.

An alogrithm based on team numbers makes the assumption that time is the only factor in team success. As we all know it is one of many factors.

Hopefully this will change for 2008.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

i may not express this right,but no matter what first does it will still come down to "luck of the draw". you will never make everyone happy, unless you plan to extend championships a few days to play every possible combination. or make sure there is a climb-able ramp on each team each match, since that seems to be the most important thing this year. or if you score keeper that would multiply your rack score by 2. or,or ,or. when all is said and done it will be the quality of your robot, your ability to sell yourself and other teams scouting that will get you to the big show.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I'm assuming this is just a typo, but team 1089 isn't listed in your original post (and I know we're going). ~__^ I'm assuming we'd be a "mid-level" team, since all the numbers surrounding us are.

I agree with Alan, there are a LOT of factors that contribute to teams building a successful machine (and for that matter, drive team). I honestly think numbers play a small, if not insignificant role.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

You can go out and perform well individually during qualifying regardless of match pairings, as we've been fortunate enough to do at Buckeye and GTR, but your chances of getting picked are directly affected by the quality of the scouting the Top 8 seeds employ. More youngins in the Top 8 may increase the risk of poorly-assembled team pick lists. We've also been fortunate in that area at our regionals this year, as have many other veterans who've finished outside the top 8 who've been picked by the top seeds.

However, these events aren't only played on Saturday. The overall experience can be greatly enriched if you get to play with an against a greater of variety of teams during qualifying on Friday. Right now, a lot of lower-numbered teams are artificially being forced into less than fun situations on Friday.

If FIRST wants to find a way to balance alliance pairings based upon team quality, I'm game, if they can find a way to make it WORK - I've actually suggested it in other threads in the past. However, the method they've chosen to implement that desire in 2007 is just plain ineffective and misguided. If a new and truly effective method can't be found, then going back to a plain old random algorithm is the way to go.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

As I stated in the original post, the division list I used was an intermediate one pulled from the 2007 Championship Divisions thread - it is used as an example only. The final divisional lists will undoubtedly be different. I will post my categorical breakdowns and 1st round match list predictions for all divisions once they are finalized and announced.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I can verify this sentiment. We are never happy about our match pairings. ;)

I guess we are talking about how this algorithm increases our level of overall unhappiness! The goal is always complete satisfaction, but in the end we are trying to control our level of unhappiness. :)

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I think this match generation model could work, given the proper set of input data. Since there has been no official statement from FIRST on this, all we can do is assume their intentions. So, I am assuming the point of this algorithm was to match "better" teams with teams who are just getting on their feet.

I don't disagree with this methodology for qualification rounds. It makes for less blowouts, and a more consistent and exciting set of matches. The problem comes with drawing the border between "good" and "not so good" teams. Regionals have been broken up into three categories (veteran, mid level, and rookie) based on team number, which is a very rough implication for experience. Due to this, alot of "good" mid level and rookie teams are getting paired with other "good" veterans. While this happens, "not so good" rookies and mid level teams are getting paired with other "not so good" veterans, making the original point of the algorithm completely null and void.

Travis has already shown how the current method of creating divisions breaks the match creation model. Ranking random teams by number will hurt more than it will help. If, on the other hand, the splitting up of "veterans", "mid level", and "rookie" teams was based on merit and not an arbitrary number, the system would work. So, for the Championship, I would love to see the use of the regional results in creating the divisions.

I propose this:

1. Assign all teams a "regional success score". 1 point for a top 8 seed, 3 points for a regional finalist, 5 points for a regional win.
2. Rank all teams based on this score. If teams have the same score, randomize them within this subset.
3. Using the current list ranked "good" to "not so good", assign teams into divisions using the 1,2,3,4 drag and drop method, like previous years.

Boda Bing! We're one step closer to parity. This assignment is something that can whipped up in excel in about 10 minutes. Now, this exact method may not be perfect. There will still be some flukes. But I can guarantee you it will work at least 10 times better than what we assume the division assignment system will be.

The bottom line is that you cannot split teams up based off of implied "experience". If you really want to create a tiered match generation system, you will need to base it off of true power.

This is the Championship event. The best of the best. Everyone wants to see the best teams on Einstein. Let's not let Rack n' Roll fall to a fluke.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

There is a problem with the fundamental assumption that young and rookie teams directly benefit from being in matches with very veteran teams.

Young teams will get as much experience from being paired up with a veteran in a couple matches as being paired with any veteran for every match.

This algorithm performs its function perfectly but the function is based on incorrect assumptions. The basic goal, to have young teams paired with veteran teams, doesn't further the competition or the teams in any way. Teams do not get a good feeling of the way that robots work together if their possible pairings are limited. Some teams have no chance of ever meeting others because of these split pools.

Isn't it better for the good rookies and young teams if they are the ones carrying their alliance in a randomly paired match? They'll be lauded, and rightfully so. The problem with having the same teams playing each other is that you see several matches that play exactly the same: one good team scores well because they told one of the other teams to defend the good team they've faced twice already. The chances that there is only one good team on an alliance are much greater with a split pool.

Secondly, the current match pairing hurts the standings. If there is a dominant team at a regional, chances are very high that they will be first ranked, because there's a smaller chance that there will be two excellent robots going against it in a qualification round. What happens is that standings get changed because the same basic matches are being played over and over again. A good team won't be able to consistently beat a very good team with two robots defending them every round.

From these pairings, little strategy evolves. An excellent team will just have its partners play defense or interference, and those teams that aren't as good at scoring will never get the chance to get better because they will be playing defense in almost all of their matches.

Young teams have found themselves pushed up into the standings because they were on good alliances. Some of these teams don't have the resources to be a picker, sometimes they don't have enough people, or they just don't have the experience of strategy. This propels the better teams further into the elimination rounds because they are able to exploit alliances that were crafted without specific strategy in mind. Young alliances in the finals might find themselves without a strategy at all because they haven't had to strategize throughout the qualification rounds.

Teams new to the competition will get to see the one or two teams beat up on the competition, pick the best other team, and romp through the finals. It gives good rookies less of a chance to be on one of these alliances, because now they're a captain.


Splitting teams into pools doesn't lead to more learning, innovation or strategy. It doesn't make teams feel better about themselves. It doesn't allow for matches with an alliance of veterans getting beaten by a bunch of young teams. Wouldn't we all like to see a match where one of the teams that has been consistently doing well has to squeak out a win against three rookies? We don't get to see any surprising upset matches where you don't get to predict what will happen. The same teams keep winning in qualifications, and you have to wait for eliminations before they see different teams. It's not like eliminations alliances have changed much.

A random pairing gets rookies on alliances with veteran teams, sometimes as often as every match. It allows for teams to be with or against teams that they didn't see last match. Teams are exposed more to different robot designs and strategies. Some strategies evolve in the middle of a match where you find yourself playing people you hadn't seen before.

Doesn't it further FIRST to have many different matches in qualifications rather than the same 2 or 3?

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

While the goals of the system may be controversial, and are certainly questionable, there is little question that this algorithm meets those goals. If you look down Ed's generated match list, there are only 2 or 3 matches with a clear cut winner, the rest will all be very competitive. Even the godly alliance of 67, 469, and 1501 won't have a terribly easy time with 74, 498, and 1523. The qualification rounds are well balanced, with both alliances having a good shot to win. While number isn't always a good indicator of the strength of a team, the fact that people are complaining that high numbers teams are being artificially inflated and low numbers are being crushed shows that it often is. As any assumption, there are exceptions. An occasional blow-out match is bound to happen, and I don't think FIRST wanted to completely erase them.
A disturbing bi-product is that it not only creates competitive qualification matches, it creates competitive elimination matches. By having some lesser-skilled teams inflated to the alliance captain status, it dilutes the talent of the pool, and creates parity among the alliances. It greatly reduces the chances that two powerhouse teams will pair up and dominate(once again, there are exceptions). Perhaps this was the motivation of FIRST? It serves the same function as the serpentine draft after all, creating competitive elimination matches, where very few alliances are a clear-cut favorite.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I really disagree with this - this system effectively punishes teams for doing well at their regionals, and raises the spectre of hyper-competitive teams getting into the Championship by first-come, first-serve (instead of actually qualifying) and then deliberately not doing well at their regionals to get a more favourable spot in Atlanta. (Not saying anybody would do this, but it's still not the sort of thing one wants to encourage.)

On the other hand, if our hypothetical hyper-competitive team doesn't register early enough to get into the Championship, they might be encouraged to go for Chairman's or Engineering Inspiration so they don't 'have' to win a regional :)

I disagree with any sort of attempted match-balancing at all (either number- or merit-based); you'll certainly get imbalanced blowouts either way (e.g. match 15 in Toronto, 188-1114-1680 vs. 519-1353-1564, 338 to 4), and with some sort of balancing system I think teams get more upset because they have something concrete to be angry at (an imperfect balancing system) as opposed to plain old luck of the draw. Also, I can't think of any other competition which tries to artificially balance matches - if it truly led to more exciting matches, one would think that it would have been done before.

This is what I've been advocating for a while. 'Check all combinations' is a LOT of combinations, though. I tried to set up match scheduling as a binary programming problem once - effectively an optimized way of checking all combinations - and there are simply too many to check. I think you'll always need some sort of algorithm to generate schedules, but at least if they're pre-generated everyone can see them ahead of time and submit better ones of their own if they want.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

It's really quite simple when you consider that, EVERYONE deserves the opportunity to be paired up with EVERYONE else.

Anything short of being allowed the chance to play WITH some of our longtime friends (that also happen to be in the same grouping of low numbers) is unreasonable and disrespectful to our teams.

If this continues, I can assure you that it will be the end of us registering as Team 47, as we will register as a new team next year (if we return).

Mike Aubry
Engineering Lead Chief Delphi

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Ian,

I'm sorry, but I don't see where you are coming from. I'd really like to believe that NO team would throw a regional just to have a good alliance partner in qualifications in Atlanta.

You're saying that this system punishes teams who perform well at regionals. I can assume you mean 'punish' in the scope that they will paired with "not so good" teams in their division. Now, what makes more sense? Creating equal alliances based on power, or continually punishing teams for having a low team number? If you don't know what I'm talking about, take a look at the match schedule from Boston. Most of the low numbered teams (121, 125, 126, etc) did not have alliances partners with numbers lower than 1500ish. Now, look at the top 4 seeded teams. 1626, 2079, 126, 1511. Notice anything? There you have 3 very good teams who, as you might say, were NOT punished for having a high team number. Were they there on merit, or were they there from the schedule? You make the call.

The best possible solution is to have a totally random match schedule. But obviously, this new algorithm is here to stay. I believe the system I proposed (or something similar.. basically anything that uses power to split teams and not number..) is a good way to make everyone happy in Atlanta. We get our competition, and FIRST gets their co-opertition.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

No matter how anyone can try to make this work and try to make a system to make everyone happy, it won't work. There is no way everyone can be happy. Someone will and moan to FIRST about this, and FIRST will try to do something to change it, and more people will and moan about that. There is no freakin' way everyone can be happy.

I would love to see the Championship Event, as a true Championship, and have everything based off merit, but with the way FIRST is set up. Its not gonna happen. Unless the game was 1 v 1, these bad alliance pairings, and the likes are gonna continue to happen. In life, there will always be a flaw, there will always be a weak wheel, its just a matter of what you do with it, and how you can use them to make it all work.
Prime example. No one ever thought a dumper bot would make it to Einstein last year. Look what happened. One fell into 25's laps, and we successfully used them in a strategy few had used during the year, and ultimately, got us as far as we did.

I do hate the current system, and think there is a better way to approach it, but with what FIRST is trying to do, what there goals are, how it is now, is just dandy.

(more to come of my thoughts about this...still deciding/doing a paper)

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I agree - as I said, I don't think anybody would actually do it. I'd also like to think that nobody would intentionally push someone into a lower goal last year to get them disqualified, but that doesn't mean that rules that allow that possibility are good ones.

There are also lots of ways to avoid a high "regional success score" without actively throwing a regional, several of which are neither particularly non-gracious or unprofessional. First is to not go to as many regionals in the first place, and concentrate on other things (using the money to build a practice bot, buy nicer parts, do community outreach, etc.). Second is to concentrate on building at a regional - could you really blame a team with a weak arm for taking it off and overhauling it at a regional, even if that meant they spent most of their matches as a box on wheels running around playing defence?

Some of the above is admittedly a bit sensationalist, but in general I worry about what happens when FIRST tries to artificially manipulate things to try to balance veterans and rookies.

Again, I don't really like either system, so I'm hesitant to argue for one over the other. I personally don't think any teams need a lot of balancing - in most things in life, one can't simply jump in and immediately be on anywhere near a level standing with the experienced veterans, and I'm not sure FIRST needs to be any different. I think most rookies can accept that they might not do so well in their first year or two, but that just makes it that much more of an accomplishment when they finally 'make it'. I'm worried that artificially helping rookie or low-performing teams could have the opposite effect when teams manage to take their robot and team to the next level and then realize that it hasn't really helped them that much.

As a bit of an aside, how do you propose doing the schedule for regionals if you use regional data to make the schedules for the Championship? Use the previous year's data? Default back to numbering?

We're certainly in agreement that a totally random match schedule is the best solution. I don't think that the new system is necessarily here to stay, though; I think FIRST has done a pretty good job over the years of letting good decisions stand and correcting bad ones. (In my opinion, the move to alliances and the addition of bumpers were good decisions that were kept; the move to 4-vs.-0 was a bad decision that was reversed. On the other hand, the serpentine draft is still around.)

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Let's look at the results instead of talking about random matches. At the Florida Regional 5 midrank or junior teams (1902-2nd, 1027-3rd,1251-4th, 801-6th, 1270-7th) were among the top 8. The current algorithm assumes that a junior team will have a weak unit while a veteran team will produce a strong robot. If that were true, then 5 or six veteran team should be in the top 8. Look also at the finals 1902 (with 108, 179, & 365-replacement) was a finalist while 1251 and 1270 were winners with Team 86. The midrank teams were not weak if they were the champions. The difference was having a practical strategy and employing a good scouting system that allowed 1251 to create a strong alliance.
There are only three kinds of fair - county fair, state fair & worlds fair. All pairing systems have a bias but the cream will rise to the top. We were a low number team (86 - Rank 21/51) with only a 4W- 4L record partly because of pairings with three no show teams who did not get to thru their inspections in time for the match. However, we could have gotten these no shows with any other pairing system. I blame the regional officials for not providing a substitute or rescheduling these matches. I would prefer a more random system of matches but a winner succeeds in all cases with good design, determination, desire, discipline, and dedication. I have not checked other regionals but I am aware that Team 45 had a lot of pairing problems and went on to win a regional. :D :) :rolleyes:

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

That statement isn't quite true. With the current scheduling system, the lower numbered teams each face a [theoretically] harder schedule as they are never paired with another low numbered team, and will typically have to carry their alliance. Thus why low numbered teams rarely emerge captains. You cannot use evidence tampered by this system to advocate against it.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Why can't we have a dynamic team sorting algorithm? Start out randomly then model team sucess rates and divide teams based on how well they actually perform as the day progresses! Weight wins and losses based on the "power" value of the opponents so that the system is even more accurate. Create alliances so the "power" value of each alliance is relatively similar. Add in a little randomness factor to keep things interesting.

We could extend this -- create an 3X3 matrix
Set up a fixed equation which determines a probablity value of a team ( with "power" value X) winning against another team (of power Y) if played head to head.
Plug in these values into the matrix. If teams are evenly matched the determinant should be close to 0. (right??)

This still doesn't take into account the fact that 3 ramp bots together will probably not do well even if they are very powerful from a statistical standpoint (lots of wins)



what do y'all think?

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

My problem is that SOS is not taken into account when determining the top 8 for aliances so it seems that all teams should have a similar SOS optimally so to have the top 8 teams picking. Either this or have the standings based on an RPI like system.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Read my words carefully. I stated that all systems are unfair to one group or another. I agree that low number teams have to work harder to win under the current match system. My thesis was as follows:
a) 1902, 1251 and 1270 were strong teams since they reached the finals
b) the current match pairing system incorrectly assumes they are weak
which gives them a stronger team in the round robins
c) good scouting will find good teams for alliances - #4 at LA, #45, & #86
even if they do not have high ranks.
:D

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

time 1860 indo para atlanta, (from brasil)para Championship...


la vamos nós.......!!!!!!

aguarde-nos.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

As another thought, if FIRST is determined to some sort of balancing, perhaps there could be a compromise. Instead of splitting teams into three tiers and having each alliance be one robot from each tier, there could be a looser restriction that every alliance have robots from at least two different tiers (or, equivalently, that no alliance may have three robots from the same tier). This would make it possible for every team to play with and against every other team, but still prevent situations like 3 veterans vs. 3 rookies.

More drastically, the restriction on what teams can be on any one alliance could be dropped, but a restriction could be added that the two alliances be balanced relative to one another - so one could have 3 veterans vs. 3 veterans or 3 rookies vs. 3 rookies, but not 3 veterans vs. 3 rookies.

Actually, on second thought, while that might get some really exciting matches with 6 great robots, there would also be some pretty boring matches with 6 mediocre robots. I still like a completely random schedule, with the emphasis being on reducing duplication (so one team doesn't see any other team more than a couple times) instead of trying to artificially balance matches, and trusting that with enough matches and enough variety in partners and opponents, things will average out.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

It is apparent that FIRST's methodology in this algorithm is the following:

Teams of a lower number will generally outperform teams of a higher number, specifically very low numbers (generally less than 400) will do better than teams between 400 and 1700, and both will do better than teams greater than 1800.

Therefore: teams above 1700 are at a statistical disadvantage to those with team numbers less than 1700, and those between 400 and 1700 have a disadvantage to those numbered below 400.

Similarly: teams above 1700 deserve a chance to not only play with teams of a lower team number, but also deserve the opportunity to win more matches with these lower number teams. Also, teams of numbers less than 1700 should not have to play with two robots of a much lower statistical advantage, nor should they play against two or more robots of a much lo wer statistical advantage

Therefore: a system must be implemented in order to even the playing field such that no alliance will have more than one robots from the same team number range. Doing this will theoretically spread out the pairings such that no alliance faces a blowout. For example, three veterans against three rookies, or even one veteran with two rookies against three mid-level teams.

Because of this, a ranking system must be created that will split teams into these three groups, and make sure that every alliance has a team from each of the three categories. Indeed, some robots will never face each other, but this is an acceptable loss so that no team will be statistically, based on team number, outmatched.

In my opinion, each of these points is somewhat flawed. First and foremost, team number is not an indicator of the team's potential performance. Instead of beating a dead horse, I'll simply name teams like 1902, 1114, 1503 and 1680 as examples. The statistical disadvantage of rookies and the advantage that veterans have is a fallacy.

I'm going to sound cold when I say this, but rookies do not intrinsically deserve to be paired with veteran teams? No. Should they? Yes. Will they? More than likely they will, no matter the algorithm. Must they (in every match)? No. Rookies and young teams need help, but artificially forcing them with veterans will not do the trick. How are they to learn about FIRST if they are forever cast into the role of not playing as the "lead team". If you think about it, FIRST would have to grow enormously in order for rookies to become the "veteran" team on an alliance in the future (essentially, triple in size).

Should every alliance be statistically fair? In my opinion, no. The randomish system of previous years would indeed cause blowout matches, but at least you knew that it was fate that caused it, not some master system that some individuals believe is better for you and your alliance members.

Should teams be grouped into three or even six or nine groups? No, I don't think this is a good idea. To be frank, FIRST is grouping teams into the good, the alright, and those that are not so hot. It is stereotyping teams, in a sense. If you are in the bottom third of the team numbers at your event, you are preconceived to be of a lower caliber than two-thirds of the other teams at the regional.

I do understand that it's "not about the robot", and that the competition is merely the means to an end (an end that I most whole-heartedly agree with), but if we cannot have faith in how things are to go at a competition, how can you focus on making science and technology be cool? Maybe it's because I'm from New Hampshire, but I think that the less meddling FIRST does, the better. Why make rules about things that teams should be making or learning about on their own? Why change things that aren't actually broken?

What saddens me the most is that if this system continues, I will most likely never get the opportunity to play with my old teams. Anyone who has been on more than one teams knows how great is is to see that you will be playing with your old friends and mentors, working on strategy together, and enjoying the thrill or disappointment of victory or defeat once again, just like the "old days". However, both of my previous teams (134 and 40) will always be in the same grouping as my current team (190). The only way we would get to play together is if we were picked in the elimination rounds, something that becomes more unlikely if a high number team rides their alliance partners to the top 8. In closing, I really hope FIRST fixes this blunder (there is no question that it is a blunder, in my opinion). I'm not going to leave FIRST over this, but for the good of the program, I wish that cool, calm, and thoughtful heads prevail in the discussion that is sure to occur at some point at FIRST HQ.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

FIRST has decided, with the help of "Bob the FTA", that a tiering system is desired. That decision may be suspect, but it has a noble goal, to even out competition. It is similar to the serpentine draft.

The problem is deciding which teams get into which tiers. Team number is not the way to go. If you go to a regional with few rookies, the 3rd tier gets distorted - such as including 1114 at GLR. At the other end, there are teams with very low numbers who have a poor year, or who have had a long-term decline due to various factors (losing sponsors or mentors comes to mind) and are just shells of their former glory. When a team signed up for FIRST does not indicate how well or poorly that team will perform.

We need a pre-defined tournament match structure, something like
Round 1, Match 1 - Teams A/B/C play teams D/E/F
Match 2 - Teams G/H/I play teams J/K/L
Etc.

Rounds 2 through X are defined with constraints - no multiple partners / opponents, maximize time between rounds, etc. Once a program is written that can do these permutations, it can be run for any number of teams in a regional or Championship division, from 24 to 100. Now we have defined match schedules, and all that remains is to randomly assign team numbers to the A, B, C, D ... positions in the schedule.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I suggest a modification to your idea which sorts the teams the same way as the divisions at the championships.
Current method
V M R
1 504 1207
2 505 1208
3 506 1209
A V team can never play another V team in the random pairings
Division Method
A B C
1 2 3
4 5 6 Low Pool
7 8 9
501 502 503
504 505 506 Mid Pool
507 508 509
1201 1202 1203
1204 1205 1206 High Pool
1207 1208 1209
This distributes the allegedly strong lower number teams (< 400) into three groups. Now a veteran team will still be paired with a midrank and a rookie in its division but it has the possibility of playing against 67% of the veteran teams:D

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

My team, 1100, was actually in the lowest tier at Boston. At least I think we were, since the lowest number team we played with was 1403, and we only played with teams under 1900 twice (once was 1403).

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Indeed, you were the last team in the A (or Vet) pool. The lowest number you could have been paired with was 1350. That is not a fun position to be in but you faired well at 5-4 and got picked 9th overall. Congrats.

There is good news and bad news regarding Championships.

The good news is you will be around the middle of the B (or Mid) pool in Galileo.

The bad news is your first match is likely to be:

250 1100 2026 v 279 1126 2056

Ouch, Tech Fusion, SPARX, and Patriotics in one match. You have FLR champ 250 on your side, but that is a tough task for even their defensive prowess. This the fake match 11 Alex Cormier starts talking about starting with this post

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

We'll take the preceeding scenarios over what we had in Vegas...team 987 was never allied with a team number less that 1000 during the qualification rounds...

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

The championships are over, and I thought it might make more sense to comment about the match schedule on an existing thread, rather than a generic championships likes/dislikes thread.

We were on Curie and it appears that the schedule achieved what it set out to do. A few observations:

1) Old and new teams were very mixed in the final standings and the alliances that were formed were all very close to each other. A little better strategy or luck and any of the alliances formed on Curie could have won that field. Watching an 8th seeded alliance win it all on Einstein, I have to assume the situation was similar on the other fields.

2) There were 8 rookies in the Championship alliances. For the record we teamed with a rookie at all three of our events this year. I don't know, but I doubt there have ever been this many rookies in the eliminations since the four division system started.

3) This game was very much a game of alliances. No machine was able to dominate unless their partners could help in some fashion. We won many rounds last year almost single handedly. We did not win any rounds this year unless we had help from our alliance partners, I suspect most people had similar experiences.

4) The match schedule employed seems to increase the likelihood of having a tough time in the qualification rounds. At one regional we almost breezed through quals because of excellent partners. At the next regional we struggled to win half our qual matches and we were almost left out of the elmins despite some excellent scoring by our team. At the championships we had a lot of close matches and won with a lot of good play by our partners. According to our scouting there were half a dozen teams on Curie ranked below 50 that belonged on the short list for picking. I know there have always been a couple really good picks way down in the standings, but almost a third of the top 24 sounds like a lot.

5) For the record I don't like the algorithm because in a field of 86 teams we should not have to face the same team twice. We were against one low numbered team twice, we were with a mid numbered team twice, and we were against and then with four more teams. That means we saw 29 different teams out of 86 when we could have seen 35, if the algorithm focused more on mixing up the match pairings.

FIRST seems to have accomplished what they wanted with the algorithm (see 1 and 2). But by limiting the variety of partners too many teams get locked into unfortunate schedules.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I saw a lot of teams get bitten by the "random" algorithm monster again. This hurt a lot of great teams dearly... some super-great teams were waaay overlooked. Some teams weren't even given a chance to impress because they had partners that couldn't compliment each other... but that's how the game's played.

For the record, team 648 played against team 16 three times. Of 86 teams, what are the chances? I don't know the stats like that for other teams, but I'm going to assume we weren't the only ones that that happened to.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Again, just to set the record straight, the match pairing algorithm is NOT what FIRST wanted. It was implemented by the software developer without the concurrence of FIRST. It was not corrected during the season, despite multiple FIRST requests.

FIRST knows the match pairing algorithm is a problem. They know it is an intolerable situation that caused significant problems with allowing teams to fairly display their capabilities and performance. It will NOT be continued next year.

-dave

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

yes 540 got bitten really hard at championship and regionals when they had to go against 539 every single match.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I hear what you’re saying Dave but it was a problem from the first day of the first regional and it was never fixed and it should have been.

You can blame the developer all you want but the buck has to stop with FIRST.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Perhaps. But I have to give them a little bit of slack when they tell the developer to make a change to the system (for which they are paying a significant price), and the developer just says "no - I won't do it. My way is so much better, and you (the customer) are just not smart enough to understand that."




.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I have an idea: We all can send this developer a note, tell him how much we liked or did not like match generating program he wrote.

Is it appropriate to post his email address?

Andy B.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Thank you, Dave, for getting this situation out into the open for all of us to see. I have understood for a couple of months now that the contractor FIRST hired has his own agenda. I think we've all been waiting for the real FIRST (the board and the GDC) to raise its voice about that.

My primary concern about the algorithm, which I expressed to my own favorite board member just after Week 1, was never that it is unfair, capricious, stupid, flawed, bogus, etc; although it is all that. My concerns were that it did not reflect the published intent of FIRST as stated in the manual, and that it in fact reflects the agenda of a software system contractor who fails to understand that role; i.e., who openly defies direction from his employer. The $20B firm that employs me would not employ such a contractor more than once.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

As much as I like this idea, I do not think posting an email address is the right thing. This person, while they may have not listened to FIRST, does not deserve the massive amounts of hate mail that will invariably be sent. Not to mention, posting an email address on a public forum is generally asking for trouble.

I do, however think that this particular software developer does need to get feedback from the teams. I cannot think of an easy way to do this currently, but I'm sure someone out there can.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

If this is the case, then I guess you're right that FIRST should get some slack. However, isn't this the same (or similar) story that we've heard a few times in the past, such as with Hatch? At some point FIRST needs to accept the fact that they are putting themselves in this position by selecting these vendors in the first place.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Andy,
I'm sorry, but posting the persons email address is NOT appropriate (at least not on this website). If we allowed it, just imagine how many others would need to be posted whenever someone had a beef against something. We really don't want to be associated with that kind of behavior.

As much as I disliked the algorithem, I think Dave has done a great job trying to explain that the supplier failed to, and refused to meet the customers (FIRST) requirements.

As in business, that supplier should not be used again, if what David has stated is correct, and I have no reason NOT to believe him.
If I ran FIRST, I'd not ask him to return and I'd throw out the algorith never using it again.

We all understand where the "buck stops", I'm sure that they learned something themselves this time and we will not see a repeat next year.

Thanks Dave for trying to clear up what happened. I appreciate knowing this.

Mike Aubry

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

OK, no problem. That's why I didn't post it.

Thanks,
AB

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I don't know if the mentors of Chief Delphi would agree to this but here is one suggestion.
Create a thread for this purpose. It can be moderated if need be.
Have some criteria for posting.
Some suggestions off the top of my head:
- pros/cons of match generating program
- impact on the team(s) competition experience(s)
- summary

In business, one learns to deal with less than satisfactory situations. It is something that should be handled professionally by each team or team member that participates in a way that does not detract from the team. The developer will get the message loud and clear, esp. if presented in a manner that expresses the frustration but does not come across as 'hate mail' It could be an opportunity to act as young professionals of FIRST teams expressing serious concerns.

The thread can be forwarded to the developer.
Mike, if this is a no as well, I fully understand.

Jane

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I would like to thank Dave L for his post. From what I have learned by speaking to the developer I can believe fully Dave's statements. Dave, again thanks for posting and keeping us informed.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Jane,
I fully endorse creating a thread where users can discuss ideas and post suggestions, relative to a "better algorithem" (as you have stated).

The thread can be forwarded to FIRST and they can forward it to the supplier/programmer/individual - if they choose to.

Nuttyman,
This site should NOT be used to get this specific software developer direct feedback. If you want to send this person direct feedback, I suggest you do that without involving this website.

CD doesn't need the hassles associated with individuals complaining about having their personal email account bombarded because someone posted their emal addy here.

Sorry - but I suggest you find another way.

Mike Aubry

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Interesting. I know the developer was very proud of his algorithm. Incidentally, he believed that he had convinced FIRST that the algorithm was a valid way to load-level teams.

He also wrote the PDA/scoring program, which was certainly an achievement, much better than last year. Although to be fair to Hatch, counting "hits" on a PDA is easier than counting balls rolling through a chute. Still, very few scoring failures, and certainly nothing like the fiasco last year when the program couldn't even calculate standings correctly.

Btw, for those that wanted to email him, he does have a CD account. Perhaps he's reading.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I'll speak up on this one more time on this matter. We don't need a "match scheduling algorithm" that springs its results on us Thursday morning, only to have a statistical analysis demonstrate that it is totally messed up.

We need a set of match scheduling tables, one for each possible team count at an event. The entries in these tables should be 1 to N where N is the number of teams at the event. These tables should not be held secret and in fact should be published for all to inspect, and possibly improve upon. The table for 47 teams (to pick an example) at an event need never be changed, unless an alternative table is found that is better, replacing the prior one after it is vetted publicly.

On that Thursday that we actually know how many teams actually make it to the event, one can use popsicle sticks labled with team numbers shaken in a jar, or any other high quality random permutation of the team number list (I urge care here, as low quality computer based random number generators abound), to map the indices in the match scheduling table to the team numbers to create the match schedule.

So, the match schedules can be vetted publicly and the mapping of the team numbers to the indices in the schedule can be completely random. The complaints can then stop and we can then spend more time in this forum helping students.

Eugene

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I do think that a truely random match listing would be unfair, but one idea is to take previous teams placements a point value, and make rookies random.

AC1 - 10pts
AC2 - 9pts
AC3 - 8pts
AC4 - 7pts
AC5 - 6pts
AC6 - 5pts
AC7 - 5pts
AC8 - 5pts

Pick 1 - 15pts
Pick 2 - 12pts
Pick 3 - 10pts
Pick 4 - 8pts
Pick 5 - 8pts
Pick 6 - 5pts
Pick 7 - 5pts
Pick 8 - 5pts

All Second Round Picks - 3pts

DNQ - 1pt

From there, decide upon ties as being random. I think this system would need some work done to it to make it effective, but it is just a concept.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Actually, this year's program failed to calculate rankings correctly as well. St. Louis was down for over an hour as they did it by hand. They fixed this after week 1, but this was a major defect that should have been caught in testing before it ever got to a regional.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

I don't think the teams at the championship should be split into different divisions based on team number, as they have all managed to get to the championship, meaning that they must all have done something well.

On the topic of randomness, matches should have been a lot more random. In 7 matches, we were paired with one team 3 times, against another team 3 times, and with the second team another time. It would have been nice to play with/against more teams.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

for the most part, this is not true. and also, what improvement was made by having a team play another 4 times in one regional that only had 7 matches and playing another team 3 times.
We work so hard all year round and try to attend one of the biggest regionals in NJ, only to find that we play the same teams over again and vice versa as they become our partners also.
WHY ATTEND A REGIONAL OF 59 IF YOU WILL ONLY SEE A FEW!?!

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

It's clear that most of us agree that this year's algorithm was a big problem and that our concerns were communicated through a number of channels starting in week one. However, before lambasting anyone we should recognize a few important facts.

First, and foremost, because of the efforts of FIRST, 4fx and their subcontractors, this year's field control and scoring systems were a tremendous improvement over previous years in terms of ease of setup, reliability and maintainability. One only has to look at how few matches had to be re-run due to field fault this year compared with last to see a clear benefit to the teams and event staff.

Next, according to "the developer", work started on this year's system in Sept. '06 and was completed on time and on what amounts to a shoestring budget, at least in the world of control systems. So, in four months they went from a GDC concept to delivery of the first system, and then replicated it in time to ship to the week one regionals by the end of February.

Finally, with regards to making a major overhaul of the qualification match scheduling algorithm, FIRST probably can't afford to pay for the kind of resources that would have been needed to make significant changes to the Field Management System mid-season. Remember that the people who would make that change were already working full-time fixing bugs that caused ranking problems and FMS database access conflicts - when they weren't running the field at regionals and providing field support. I suspect that the main reason we didn't see a big change mid-season was that it would have increased the risk of FMS failure at events and taken resources away from more critical tasks.

I've debated the team ranking philosophy with "the developer" a few times and at this point I'd say we've agreed to disagree. It's obvious that FIRST has to clearly specify to him what they expect for next year's system and I hope they do it earlier than September. It is not clear to me that everyone at FIRST headquarters agrees that this year's algorithm is all that bad, so if people want their complaints acted on, they should go there, not vent at the guy who wrote it.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

Well, not to whine and complain, but considering the fact that during the championships, my team was paired with one of the same teams for 3 of our matches in Curie. I don't think it was that random, considering that there were about 80 other teams in the division, and we were never against that team.
Granted one of the matches was during the practice day, but even at Regionals I have found similar things happening.
And to reitterate, many of the same teams do win reoccuringly because of the pairings, and many wonderful teams lose because of the same reason.
However, it does help to have some of these pairings because some of the teams just don't have the experience. Then again, rookie teams win regioanls so, it all depends on the team, I guess.
I am not "hating on F.I.R.S.T." I am just voicing what I think it true.

Re: Championship Event - Where the "Random" Match Sorting Really "Shines"
 

From Team Update #21:

1) The Director of FRC is very aware of the concerns regarding the Alliance Selection
algorithm used throughout the 2007 competition season. Unfortunately, we ran out of time
and the resources necessary to properly implement a change and meanwhile mitigate any
risk to the FIRST Championship.

********************************