Is the qualification match robot randomizer really "random"

[coalhot]

Today and yesterday, in the Philly regional, i noticed that top-tier(better designed, better equipped) robots seemed to be paired up together; while low-tier (teams that didnt have much funding, big names, etc...) were paired together AGAINST the top-tier alliances. Im including the match records (courtesy of TBA) so you can see for yourself:

Match# | Team# | Final Score

Qualifications 1 2539 433 103 1143 1634 304 |66 16
Qualifications 2 708 484 2229 816 321 272 |32 52
Qualifications 3 237 1640 834 709 84 2559 |56 40
Qualifications 4 467 316 341 284 486 1601 |62 44
Qualifications 5 1727 563 1712 539 423 357 |46 12
Qualifications 6 444 1370 1391 1495 1885 381 |64 62
Qualifications 7 809 2607 1647 2558 365 2668 |24 22
Qualifications 8 2180 1640 84 1511 484 103 |26 88
Qualifications 9 272 2559 467 486 816 304 |64 8
Qualifications 10 2229 237 284 708 357 1727 |54 24
Qualifications 11 1634 341 1712 381 1391 423 |64 32
Qualifications 12 2558 709 1143 365 1495 1370 |40 44
Qualifications 13 2180 834 316 321 1885 809 |32 22
Qualifications 14 1647 433 1601 1511 563 2607 |0 74
Qualifications 15 2668 539 284 444 2539 484 |28 14
Qualifications 16 381 103 84 304 708 1391 |120 38
Qualifications 17 423 365 709 1727 237 341 |4 76
Qualifications 18 357 272 316 467 1143 1370 |96 48
Qualifications 19 1640 486 1885 2607 321 1712 |46 34
Qualifications 20 444 539 433 2180 809 1601 |28 14
Qualifications 21 1495 1634 2558 816 2559 563 |4 52
Qualifications 22 2229 2539 2668 834 1511 1647 |36 78
Qualifications 23 304 381 316 484 341 357 |28 66
Qualifications 24 1885 284 84 486 272 423 |34 36
Qualifications 25 1391 709 1727 467 433 1712 |48 66
Qualifications 26 1495 563 103 816 237 539 |70 36
Qualifications 27 1647 1143 444 2607 1634 1640 |32 12
Qualifications 28 1511 2668 321 2229 1370 2558 |66 14
Qualifications 29 2539 1601 2559 365 708 2180 |20 92
Qualifications 30 834 341 423 809 272 433 |40 34
Qualifications 31 84 1712 1391 316 1495 284 |30 42
Qualifications 32 709 237 1647 304 484 1885 |64 14
Qualifications 33 321 467 103 444 381 2668 |96 42
Qualifications 34 1601 365 1511 1634 486 357 |72 64
Qualifications 35 809 1640 1370 816 1727 2539 |40 44
Qualifications 36 2607 2229 2559 2180 563 539 |34 48
Qualifications 37 708 2558 237 1143 834 284 |38 26
Qualifications 38 433 484 423 444 467 1495 |28 42
Qualifications 39 486 321 365 709 103 357 |50 122
Qualifications 40 816 809 316 84 1647 2668 |60 42
Qualifications 41 2180 1885 2229 381 1634 2539 |36 56
Qualifications 42 1370 2559 1727 1601 341 2607 |42 54
Qualifications 43 272 708 563 1143 1511 1712 |54 82
Qualifications 44 304 539 1640 1391 834 2558 |46 36
Qualifications 45 357 365 444 84 316 484 |106 64
Qualifications 46 467 809 284 237 321 1634 |88 34
Qualifications 47 2607 103 486 381 433 2229 |122 58
Qualifications 48 341 2180 2539 1712 816 1495 |58 32
Qualifications 49 1370 834 1885 1647 563 304 |36 50
Qualifications 50 423 708 1143 1511 539 709 |10 20
Qualifications 51 1601 2668 1391 2558 272 1640 |10 58
Qualifications 52 1727 484 321 2559 103 316 |36 82
Qualifications 53 381 816 357 284 365 433 |98 72
Qualifications 54 1712 444 486 304 237 1370 |10 60
Qualifications 55 1143 563 2229 809 84 341 |24 76
Qualifications 56 2539 1511 272 1495 709 2607 |102 56
Qualifications 57 1885 1647 2558 467 1727 539 |14 52
Qualifications 58 423 2180 1634 708 2668 1640 |46 40
Qualifications 59 1601 834 444 1391 2559 357 |26 60
Qualifications 60 304 1712 284 365 809 1143 |58 42
Qualifications 61 84 237 433 1370 2539 316 |62 54
Qualifications 62 2558 103 539 1885 272 341 |78 44
Qualifications 63 709 486 563 2668 1727 2180 |64 50
Qualifications 64 1495 423 1647 1601 381 321 |34 68
Qualifications 65 1640 2559 1511 2229 467 1391 |66 48
Qualifications 66 1634 484 816 2607 708 834 |68 68
Qualifications 67 433 357 1370 341 103 284 |26 64
Qualifications 68 563 316 365 272 444 709 |90 62
Qualifications 69 2558 486 381 2539 809 423 |60 42
Qualifications 70 1511 304 1727 84 321 539 |84 30
Qualifications 71 1391 237 1143 816 2180 1647 |58 78
Qualifications 72 834 484 1712 1634 1601 2229 |72 4
Qualifications 73 1495 2559 708 2668 1885 467 |12 60
Qualifications 74 2607 365 272 1640 316 423 |118 64
Qualifications 75 341 381 709 444 284 1511 |58 20
Qualifications 76 321 2539 304 563 1391 433 |46 12
Qualifications 77 1370 1634 539 237 486 484 |24 78
Qualifications 78 809 357 1495 708 1647 103 |70 54
Qualifications 79 2180 2607 467 84 834 2229 |18 46
Qualifications 80 1143 816 2668 1885 1601 1727 |90 42
Qualifications 81 1640 1712 381 2558 2559 304 |54 36

Is this just me, or does this look fixed to all of you who have been at Philly (or not)? Has this been happening in other places?

Thanks for your input...

~Philip

[Rick TYler]

I know the people that wrote the match scheduler. I've read their paper on how it works. I ran the match scheduler as scorekeeper in Seattle. Any pattern you've discerned is not a result of intention on the part of the algorithm. Trust me. I can't think of anyone in FIRST whose integrity I trust more than Tom's and Cathy's.

[Dillon Compton]

Philly's pairings did seem distinctly inconsistent with the excellent match scheduling I've seen at the other regionals so far this year.

It honestly felt like we were using last years algorithm (a poor version of it) to generate the matches - we were with or against the same teams 2-3 times/team over the course of the regional. In 11 matches with a field of 44, this shouldn't have happened to the extent it did.

My only thought is that perhaps someone did not understand the algorithm entirely and put improper constraints on it, locking it into poor schedule choices.

No complaints, though - such is the nature of FIRST. Luck is all part of the game.

//Dillon

[coalhot]

Im not questioning someone's integrity, but i'm just saying that maybe the computer was set up wrong or something, and gave out inconsistent matches...

~Philip

[Lil' Lavery]

I find your observation interesting, I didn't see that at all.
103 played 272, 341, 357, and 365.
272 played 103, 341, 365 and 1511.
341 played 103, 272, 357, 365, and 1511.
357 played 103, 341, 365, and 1511.
365 played 103, 272, 341, and 357.
1511 played 272, 341, and 357.

I'm not really sure how much more the "big name teams" can play each-other.

[coalhot]

103, 272, 357, 365, 1511 were all in the finals...

~Philip

[Danny McC]

All of those teams were in the eliminations but the finals, no.

[coalhot]

thanx for the correction..

~PHilip

[Lil' Lavery]

103, 272, 357, 365, 1511 were all in the finals...

~Philip

That's exactly my point. They were all playing against each-other often in the qualifications.

[coalhot]

because they eliminated all the others, because they were together in the qualification matches 1-80...

~Philip

[Dillon Compton]

I think the claim that the "big name" teams were together for the entire time are entirely spurious - I will, however, say that a number of teams had clustered plays - one team playing another multiple times, or one team playing with another multiple times.

This was NOT limited to the "big name" teams playing with/against one another, and indeed did NOT have any appreciable effect on the outcome of the event.

The teams that got into the afternoon would've gotten there regardless of the qualification matches (for the most part). They were chosen, or chose, because they had the best 'bots at the regional, not because they did or did not play each other.

[TheBestOfFIRST]

because they eliminated all the others, because they were together in the qualification matches 1-80...

~Philip

Sorry to say this...but i think this thread has gotten a little off topic.

I too have noticed that the randomizer recently has been a little "lax" to say the least.

[Richard Wallace]

This post (http://www.chiefdelphi.com/forums/showthread.php?p=724756#post724756) by Chris Husmann provides some insight.

If the minimum time between matches is set higher than necessary, teams will tend to oppose and ally with each other more often because they are on the same time interval.

[SayDeeM8194]

We just competed in both the Peachtree and Bayou Regional and we did find that we were playing either with or against many of the same teams. I do not question the integrity of FIRST, however, I do think the algorithm could be adjusted. As someone earlier stated...the luck of the game is the nature of FIRST. One just hopes they get the better end of the deal!!

[TheBestOfFIRST]

That's just it I can't do the math becasue we don't know the algorithm. If the algorithm is so truly random what does it matter if they supply it to the teams. I would be interested in running a few test situations of my own to see how the matches play out.

Also we need to identify "RANDOM" because is it ever truly possible to define random with a simple algorithm the human mind understands when the mind can not grasp the concept random.

[Lil' Lavery]

because they eliminated all the others, because they were together in the qualification matches 1-80...

~Philip

Uhhh... I was referring to in the qualifications. All those match-ups I mentioned were qualification matches. I was using "big name" teams that made the eliminations as my example six teams of playing one another in the qualifications. As you can see, all six of those teams faced at least three of the other five at least once in qualifications. 341 played against all five others.
When you play eleven matches, you will be paired with 22 teams and against 33 others. With 55 spots, and only 44 teams, it's literally impossible to not play with/against the same team(s) multiple times. With the time and other constraints added, it means that you will miss the opportunity to play a few teams, and you will play with few teams multiple times.

[Richard Wallace]

That's just it I can't do the math becasue we don't know the algorithm. If the algorithm is so truly random what does it matter if they supply it to the teams. I would be interested in running a few test situations of my own to see how the matches play out.... The 2008 match generation algorithm is not a secret. It was discussed extensively on the FIRST Forum (http://forums.usfirst.org/showthread.php?t=7805) before the 2008 FRC season started. The improvements that FIRST introduced this year were suggested by the people mentioned in an earlier post in this thread (http://www.chiefdelphi.com/forums/showthread.php?p=726571#post726571).

[smurfgirl]

At the Connecticut Regional, they did a good job of putting us with a team once and against them once, if we were in a match with them twice. I felt like we played a lot of matches with young teams on our alliance and against us, so it basically evened out the competition. Older teams seemed to be with and against older teams, primarily. So it wasn't really that unfair. There were also matches that seemed to be a good mix of teams. Maybe the Philly matches were just a coincidence?

[XaulZan11]

The problem I have with this 'random' match schedule, is that it can be redone if they feel there are 'too many repeat'. By doing this it is no longer a random schedule. While I don't think this is likely, it leaves the door open for someone to rerun the program because of an unfavorable schedule.

[Rick TYler]

The problem I have with this 'random' match schedule, is that it can be redone if they feel there are 'too many repeat'. By doing this it is no longer a random schedule. While I don't think this is likely, it leaves the door open for someone to rerun the program because of an unfavorable schedule.

Except that the scheduler is run by volunteers who are mostly under major pressure to get the match list printed (on a printer that only works about one in four tries) and distributed to dozens of anxious teams as soon as possible. We don't give a big furry rat what the match list is as long as it will finish on time and print the first time. Carefully scrutinizing the match list to give one or another team a huge advantage is so far from being important that -- well, analogies fail me. We want to run a fair tournament that finishes on schedule. The volunteers, for the most part, have zippy-de-doo-dah interest in who actually wins. I, for one, could not even tell you who won Seattle, and I watched every single match from about four feet away. I can tell you that every team I talked to was happy about the venue, the officiating, and the volunteers. That's a win to a volunteer.

The idea that we would keep re-running the match list to meet some competitive profile is something I literally never considered until you suggested it. I know for sure that my scorekeeping partner and I would have laughed at the idea. (Although, as I posted elsewhere, we did redo the match schedule once to give a particular team an early match to meet the needs of the television coverage.)

[Dreadfrost]

Last year i was determined that it wasn't random at all because at the Greater Toronto Regional, waterloo regional and curie division qualification match we never played 1114 until the finals at the regionals and quarter finals in curie. However this year we played them at waterloo and then not at Toronto. Therefore I've come to the conclusion that its all luck.

[ChrisH]

The problem I have with this 'random' match schedule, is that it can be redone if they feel there are 'too many repeat'. By doing this it is no longer a random schedule. While I don't think this is likely, it leaves the door open for someone to rerun the program because of an unfavorable schedule.

While this is possible, it is not likely. For one thing the Scorer, FTA and Field Supervisor are all likely to be present and all will have different ideas about just what an "unfavorable" schedule is. For another it takes about 15 min. to generate a new schedule. We just don't have time to generate more than 2 or 3.

The auditing screen gives the following information about each team:

Number of unique partners - this should be #of rounds x 2 and generally is
Number of unique opponents - ideally this would be # of rounds x 3 but there is some variaition here. This is the number checked most closely.
Number of matches as Surrogate - should be no more than 1 and not many of those.
Teams played and number of times - listed in numeric order, not by what match they are in. # of times played should be no more than 2 for any one team, unless the event is really small and there are lots of matches, in which case there should be lots of 3s not only one or two.

None of this gives very much information about the difficulty of the schedule for a particular team. To figure that out you have to go over the schedule itself which would be very time consuming. Then if you cause the schedule to be re-generated, the schedule you have will disappear. So you are taking a very significant risk that the new scedule will be even worse that the one you just blew away, and you have to do all the analysis all over again to find out. The costs outweigh the benefits very quickly.

In practice, you run the scheduler once and check the output against the criteria above. If it passes you go with it. If not you change one parameter and try again. Once you get somehng that is "good" you lock it and attempt to fire up the printer. But that is a whole different story...

[XaulZan11]

To answer this thread's title, no the qualification matches are not random and never were intended to be random. In order to be random, every team needs to have an equal chance of being in every match. Thus, a team can be in multiple matches in a row. Since having a team play 3 matches in a row in not desired, perameters were set so allow time inbetween matches. Within the certain perameters, however, the matches are random (if they don't redo the match list). There is nothing wrong with it not being random, but it shouldn't be called random because its not.

[Rick TYler]

There is nothing wrong with it not being random, but it shouldn't be called random because its not.

Mathematically you are correct. In this context, however, I'm guessing that most people would equate "random" with "not biased in favor of a particular team or class of teams." It might be more valid to ask if the match scheduler produces a fair schedule.

[coalhot]

First, sorry for saying "big-name", i meant to say "sucessful".

another problem was that in qualifications, if a team couldnt go on the field, there was a 2 VS. 3 match, which obviously favors the 3-team alliance.

~Philip

[George A.]

First, sorry for saying "big-name", i meant to say "sucessful".

another problem was that in qualifications, if a team couldnt go on the field, there was a 2 VS. 3 match, which obviously favors the 3-team alliance.

~Philip

That isn't something new. Teams have been missing matches for years now, if not forever.

The scheduling won't help the teams compete on the field...but YOU can. You have the match list at some point on Thursday, then you can go around to your alliance partners and make sure that their bot is working. If it isn't, then try your best to help them make your match.

Also, just because it's 2v3 doesn't automatically mean that it's a lock. I've seen plenty of matches where the alliance with 2 teams, win.

[GaryVoshol]

I've seen plenty of matches where the alliance with 2 teams, win.I saw a practice match on Rack-n-Roll where an alliance with zero robots won. The HP scored one ringer, and that was the only score of the game!

[Rich Kressly]

I thought the "big names" had every bit as tough a road through the qualifications as anyone. Sean's post shows that pretty well. Second, not only is the algorithm not a secret, it's entire testing and development is well documented right here on CD. Third, I thought our (1712) schedule was a lot more balanced this year than it was last year. Last year, because the algorithm was way too stringent, we wound up with a "big name/low number" partner in nearly every match. Thanks to the help of 272, 341, etc we had a great match record in Philly last year. This year we only saw top tier partners a few times and it certainly made our road to Saturday afternoon a lot more difficult, but albeit more fair.

You also need to realize that if you desire to win, you'll have to best the "top tier" anyway. Eight of us on 1712 spent our entire lunch break repairing a claw just for the privelege of going up against the 103 alliance - and I do consider it a privelege.

Next, be careful as to what you perceive about teams and their funding. This is a year when 357 lost a major sponsor. Ask 272 about their major funding and they'll point to a tune up FLL event they run to fund a regional entry fee. All of the time and energy spent on that event takes away time that could be spent on prototyping for the season. 103 seems to be a juggernaut with their number of events and facilities, etc but if you talk to their leadership you'll find two people that started the team with absolutely nothing some 10-11 years ago in the middle of the woods. Every relationship and piece of hardware that exists in Kintnersville is because of hard work and persistence.

Even suggesting that something is "fixed" here is truly unfounded. With a 44 team event (very small) a match schedule with 6 on the field at a time can only look so many different ways.

-Rich

[Kims Robot]

Im not exactly sure what you are seeing in that, we were actually fairly impressed that when we sat down to do our strategy on thursday night, that we played in matches with all but 3 of the teams at the regional, and many of the teams that we played multiple times, we were with in one match and against in the next or one later in qualifications.

And as seen with Sean's examples, Im not sure how you can think too many big name teams had "easy" schedules. I think part of it was that there were some really good teams at Philadelphia, and then there were some that struggled with design as well. We found from our scouting that the divide was really odd this year as opposed to prior years. We normally see a pretty bell shaped curve distribution. This year, it was actually relatively flat in terms of team performance,possibly even curved up on the low end. I think one of the biggest things is that strategy this year isnt anywhere near as simple as past years, and it shows in the teams that have competed already and those that havent. Look at our record from Finger Lakes - while we felt we had some tough matches, we were ranked 27th, we hadnt yet nailed the game strategy. It wasnt due to us fixing our robot, our improvement in Philly was due to us figuring out the strategy. By week 5, many of the big name teams have already competed in other regionals, while many of the other teams havent had the chance. And if they havent been doing strategy off the blue alliance or havent gone to see other regionals, my guess is that they struggled similar to what we did in our first regional.

The other thing too is Im not sure how you can "define" the top-tier teams anymore, and how a match algorithm can know the difference. There are several low number teams that have struggled with building the robot or maintaining sponsors, while there are more and more high number or mid range teams that are really pulling together and becoming better and better. Even some of the rookie or second year teams seem to be more and more impressive.

[Akash Rastogi]

I don't know about Philly but you asked if this happened in other regionals

Chesapeake-good
NJ-it was fair
The matches were pretty good because we got to play w/ and against top teams.

[coalhot]

I'm not saying this is fixed, im just trying to say that the computer that picks the alliances should be more fair. Take for instance the last qualification match in Philly. 304, 2558, and 2559 were paired against, 1640, 1712, and 381. 304 was on an alliance with 2 rookies, one of which had to e-stop their robot a few seconds into the match. Meanwhile, they were against 2 hurdlers and a racer. What im saying i guess is that the best way to work this out would be to have an alliance like 365, 2558, 304 VS. 103, 2559, 84 (providing enough of each tier robot is available). I guess the algorithms would need to be tweaked in that case...

Thanks for all the responses in this topic!!

~Philip

[XaulZan11]

I'm not saying this is fixed, im just trying to say that the computer that picks the alliances should be more fair. Take for instance the last qualification match in Philly. 304, 2558, and 2559 were paired against, 1640, 1712, and 381. 304 was on an alliance with 2 rookies, one of which had to e-stop their robot a few seconds into the match. Meanwhile, they were against 2 hurdlers and a racer. What im saying i guess is that the best way to work this out would be to have an alliance like 365, 2558, 304 VS. 103, 2559, 84 (providing enough of each tier robot is available). I guess the algorithms would need to be tweaked in that case...

Thanks for all the responses in this topic!!

~Philip

Not every match will be perfectly fair. Some matches you get bad partners, some matches you get good partners. In the end, however, it should even out. Personally selecting matches not only would take a lot of time, but be prone to biases and just would cause a ton of problem.

[bduddy]

I saw a practice match on Rack-n-Roll where an alliance with zero robots won. The HP scored one ringer, and that was the only score of the game!It happened at least twice at SVR 08; the fact that an alliance starts with 24 points really helps.

As for the original topic-can anyone figure out what the minimum match separation was? I'm far too lazy to page through that schedule, and teams that were there probably can figure it out pretty quickly.

[coalhot]

Just on a side note, will there be videos of the Philly regional on TBA?

~Philip

PS. i guess i should have called this thread "why did the same teams get picked together so many times in the Philly regional?"

[Dillon Compton]

You also need to realize that if you desire to win, you'll have to best the "top tier" anyway. Eight of us on 1712 spent our entire lunch break repairing a claw just for the privelege of going up against the 103 alliance - and I do consider it a privelege.

-Rich

I'm going to hijack this thread for a second to thank Rich and 1712 for giving 1391 the chance to share in that privilege of going up against 103's alliance. I thought 1712, 1391, and 381 put up an excellent fight against a very competitive alliance (even if we did get eliminated in QFs).

I also want to echo that match pairings and distribution have been MUCH better this year than in my previous experience. This algorithm is a huge step forward in generating good, distributed matches.

//Dillon

[coalhot]

Since we all are getting off-topic, ill do too and give a plug to Metal Moose. So good hurdler they could throw it out of the ring (literally). Great 'bot. And 304 had a match against them, and almost lost were it not for a good matchup. Ill look for you guys in the Atlanta vids!!

~Philip

[Knippschild]

I was a scorekeeper for the Philadelphia Regional. I was present when the match schedule was generated (by the lead scorekeeper).

The FMS (the event management software) uses a very complicated algorithm to generate a completely RANDOM schedule. When FMS calculates who should be paired with whom, and who should be pitted against whom, it has no knowledge of the team's "big name"ship nor of that team's standings at other regionals.

The match parings take about 3-5 minutes to generate. Last year's algorithm only took seconds to generate... this one is definitely a lot more random by definition.

Also, once a match is generated, it is impossible for a human to edit the schedule and rearrange the pairings to fit some sort of side agenda.

I assure you, the Philadelphia Regional's schedule was generated once, and that was the official schedule used for the qualifying rounds.

[demps45]

I have often wondered if the "random" pairings were actually "random" too. We attended the Greater Toronto Regional this past week, and after the pairings were distributed, some teams complained that the pairings were not fair, so the officials ran the program again and came out with pairings that were satisfactory to those complaining. the officials said they had used the wrong algorithm for the number of teams attending the event. That tells me that mistakes do happen. I am not questioning the integrity of any of the officials, only the method used for pairings. As luck would have it, we had much better pairngs "before" the the pairings were rerun. Also, the "better" teams ended up with better pairings "after" the program rerun. Go figure!!

[dlavery]

The FMS (the event management software) uses a very complicated algorithm to generate a completely RANDOM schedule. When FMS calculates who should be paired with whom, and who should be pitted against whom, it has no knowledge of the team's "big name"ship nor of that team's standings at other regionals.

The match parings take about 3-5 minutes to generate. Last year's algorithm only took seconds to generate... this one is definitely a lot more random by definition.
As previously noted, the match pairings are anything but random. Rather, it is a highly constrained process, with many factors influencing how the pairings are established. As outlined in Section 9.3.2 of The Manual, there is a specific set of criteria that are used by the match pairing algorithm to generate the match list. The match pairings generated by the algorithm are evaluated for a "best fit" against these criteria as the list is generated, and adjusted mid-process if necessary. So, by definition, the resulting list is NOT random. Nor would you ever want it to be random. A truly random list will have a significant probability of having teams playing back-to-back matches, being allied with or opposing the same teams, or having uneven numbers of matches. All of these are situations that we clearly want to avoid.

-dave

[Steve W]

I have often wondered if the "random" pairings were actually "random" too. We attended the Greater Toronto Regional this past week, and after the pairings were distributed, some teams complained that the pairings were not fair, so the officials ran the program again and came out with pairings that were satisfactory to those complaining. the officials said they had used the wrong algorithm for the number of teams attending the event. That tells me that mistakes do happen. I am not questioning the integrity of any of the officials, only the method used for pairings. As luck would have it, we had much better pairngs "before" the the pairings were rerun. Also, the "better" teams ended up with better pairings "after" the program rerun. Go figure!!

The issue at GTR was that the match list was printed before being checked. There are defaults in the program that were set quite high. When the matches were looked at, it was discovered that teams were playing with or against all of the same teams. The number of matches between played matches was changed from 9 to 7. This allowed a greater mixing of teams thus not as many with/against matches. This allowed the teams to play with/against a larger number of teams because of a 66 team regional.

As for who gets better pairing, I guess that would depend on everyones definition of what a better pairing is. My definition would be to play with/against as many different teams as possible so that we can have the opportunity to gain knowledge and workability with many teams.

[The Lucas]

This year's algorithm is a tremendous improvement over the previous years IMHO. I have read up on it and downloaded it to test (http://idleloop.com/matchmaker/)(I recommend reading it before posting in this thread). I agree with how it approaches the problems, and love how it is customizable from the command line.

The most important factor in evaluating a schedule, at least in my opinion, is paring uniformity. Here is my grading for the 3 schedules that it generated for my teams:

FLR #365: A Philly #365: C Philly #1495: B

Making broad judgments about the algorithm based only on 3 team schedules at 2 event is flawed because it is a minuscule sample size. However, I think I can provide some analysis. The big problem with 365's Philly schedule is the series of 4 matches (in a row) in which we were with/against 357 (1 with, 2 against). 486 was also involved in back to back matches in this series in a home and home aspect (with one match against the next). This series (or cycle) suggests to me that the algorithm was having trouble maintaining maintaining paring uniformity around the minimum match separation constraint.

I think the default minimum match separation was higher than optimal for Philly. In the 3 schedules, the lowest match separation was 4 at FLR and 5 at Philly so I guess that was the minimum. My guess is that minimum match separation is set to # of teams / 6 (round down) - 2 since FLR had 40 teams and Philly had 44 (42 would yield exactly 5).

I would be willing to sacrifice some match separation for better match uniformity. At events the size of FLR and Philly, I personally would be willing to go for a minimum match separation of 3. The software already gives the scorekeepers the option to specify a minimum match separation (although I don't think many scorekeepers do), but I think the way the program computes a default match separation needs some tuning for the next revision (even it is just subtracting 1).

What does everyone else think about the tradeoff between pairing uniformity and minimum match separation?

I was a scorekeeper for the Philadelphia Regional. I was present when the match schedule was generated (by the lead scorekeeper).

The FMS (the event management software) uses a very complicated algorithm to generate a completely RANDOM schedule. When FMS calculates who should be paired with whom, and who should be pitted against whom, it has no knowledge of the team's "big name"ship nor of that team's standings at other regionals.

Since volunteers from both 357 and 365 were present when a schedule that called for them to face each other 3 times was generated, I think we can all lay the "algorithm paired big names theory" to rest (not that anyone has been talking about that lately)

[Tom Saxton]

Cathy and I have just released a new version of MatchMaker, the schedule generation program, to FIRST. This new version includes several changes to avoid the clumping effect first reported on this thread.

The clumping was caused mostly by setting the minimum gap between matches too high, which was mostly caused by MatchMaker picking a default that was too large for larger tournaments. To a lesser extent, the way the algorithm was seeded contributed to this problem.

The changes are:

1. MatchMaker now limits the values specified for the minimum match separation so that values that will yield poor schedules will not be allowed.

2. The default chosen when MMS is not specified on the command line is improved, especially for larger tournaments.

3. The starting schedule that is used to seed the match generation algorithm is now randomized both to reduce clumping and also to mix teams up to improve fairness even in situations where constraints on the schedule force some amount of clumping.

4. MatchMaker has a new command line option, -x, which will cause it to just print out the lowest and highest allowed value for MMS, which may be used by the scorekeeper interface to show the range of allowed values.

If you want to give it a whirl, you can get a copy of the build from the MatchMaker download page:

http://www.idleloop.com/matchmaker/download.php

There is also a white paper on the same site that describes the algorithm used.

Please let me know by direct message if you find any issues with this release.

[Seth Mallory]

Tom,

I want to thank you and Cathy for your efforts. The improvements in 2008 over 2007 were impressive. I for one was satisfied with that system. You must have spent a lot of time to engineer this new system. Thank you

Seth

[coalhot]

Hey,
I was very tied up during the last few weeks, but i have to say (from seeing it work) good job!!!! It works to perfection. Couldn't be happier!

(now i have to find a way to remotely use the randomizer to our advantage :) )

Again, thanks!!!

--Philip

[Elgin Clock]

In doing some research with the "randomizer" this year to improve our scouting techniques, & while I agree with the rest of the folks here that 2009's version is VASTLY improved over previous years, I only have one suggestion to add to the "randomizer" for future years.

With this paper I generated for the NJ Regional: ( http://www.chiefdelphi.com/media/papers/2224 & testing the match list in NYC which I didn't release a paper for) I found that the placement of teams during qualification matches in the alliance stations were not equal.

For instance, one team could play a total of 8 matches, of which 4 of those would be in one Alliance Station spot (As shown in that paper, Team 103 (http://www.thebluealliance.net/tbatv/team.php?team=103) being in spot Blue 3 a total of 4 times for example).

I'm sure this could be an easy fix or addition to the algorithm if it was approved.

I know, speaking from an operators perspective, it's nice to bounce around your placement on the field rather than always stand in the same spot & have the same view on either red or blue side of the field.

[Jared Russell]

With 41 regional competitions, it's going to happen that sometimes the best bots end up with and against each other consistently. The schedule generator takes a lot of factors into consideration, but robot ability is most definitely not one of them.

But yes, this year in Philadelphia it seemed that there was not a lot of parity in the schedule. The proof is in the pudding: the better robot were selected for the finals, including some with poor qualification records because of who they had to play against. Yet several top 20 seeds missed the playoffs altogether.

The scheduler doesn't have to be "rigged" in order to generate a match list without robot parity.

[Tom Saxton]

With this paper I generated for the NJ Regional: ( http://www.chiefdelphi.com/media/papers/2224 ) I found that the placement of teams during qualification matches in the alliance stations were not equal.

The algorithm does try to balance each team's appearance on the red and blue alliances, but doesn't attempt to balance positions within each alliance.

The red/blue balancing was added because some event arenas are asymmetric where it's better to view the match from one end or the other, so it's a pain if your team is always playing from the less desirable end. The schedules generated before FIRST asked for that (before the 2008 season) were *much* more imbalanced red/blue than what it does now.

[GaryVoshol]

In five weeks of Michigan Districts, I have not noticed any clumping problems. This is especially significant given that there are 40 teams playing 12 matches each; obviously many see each other more than once. But that doesn't take away the perception - I heard of one complaint that they had to play with "that rookie team" (said disparagingly) 3 times. I hadn't noticed - and sure enough, a check of the schedule showed only one such pairing.

The only downfall with that size of events is the minimum match spacing has to be set to 3 - meaning sometimes a team barely has time to return to the pits before they are being requeued. But even that doesn't "clump" - if a team has a 3 or 4 game separation, likely there will be 11 or more games before their next match.

Good job, Tom.

[Elgin Clock]

The algorithm does try to balance each team's appearance on the red and blue alliances, but doesn't attempt to balance positions within each alliance.

Oh, I definitely understand that.
All I ask is that for next year, (if possible) for it to be stepped up just a bit to include randomization within the red or blue as well a bit more.
What fun is advancing the program, if it will stay the same every year?
I guess I'm just an Engineer by nature (but not degree yet) and always want to see something improved. lol It's a curse... & a blessing all in one. :cool:
The jump from last year to this year's alliance pairing system was progress by leaps & bounds no doubt, and I congratulate you on that!!! :)
For next year, I only have that one request if possible with that said.
Just a suggestion!

The red/blue balancing was added because some event arenas are asymmetric where it's better to view the match from one end or the other, so it's a pain if your team is always playing from the less desirable end. The schedules generated before FIRST asked for that (before the 2008 season) were *much* more imbalanced red/blue than what it does now.

I think the regional I went to this past weekend (CT Regional) is a perfect example of one of those where teams can play on the "less desirable end" as you noted.

The main screen which shows the field (and thus real time scoring, & video) is behind the Red Alliance station, so I can see where that request came from - Blue has an advantage in that scenario by being able to see the HUGE screen very nicely with a quick glace up forward, while the Red Alliance has a rather tiny LCD screen to glance over at in retrospect, or has to look behind the or rely on their coach to look behind them.

I guess it's true, every simple request has a good reason behind it!

[martin417]

I believe that the algorithm is written as fairly as can be achieved, given the limited number of matches teams play. However, I did see some oddities that caught my attention. The most unusual was one team that had a drive-train problem on arrival, and missed their first 5 matches (I saw many people from many teams lending a hand to get them up and running, FIRST at its best). They did send a human player for each match, and had very good alliance partners, so they were highly ranked, even though they hadn't had a robot on the field.

[Steven Sigley]

Why is there an emphasis on reducing matches with surrogate teams in the algorithim?

Because of time constraints at regionals?

I know i personally enjoy playing surrogate matches, you get to enjoy playing on the field without the personal stress that comes with losing a match. :)

[EricH]

Why is there an emphasis on reducing matches with surrogate teams in the algorithim?

Because of time constraints at regionals?

I know i personally enjoy playing surrogate matches, you get to enjoy playing on the field without the personal stress that comes with losing a match. :)
Because if you've got too many with a surrogate, you're doing something wrong.:)

It's that teams like lots and lots of matches that count. Matches that don't count are a bonus. If you have too many surrogate matches, then you don't get those counting matches. Plus, if every team has a surrogate match, then why have a surrogate match at all?

[Ice Berg]

Back in 2007 (not relevant to this years match algorithm but still interesting) we were at the Trenton Regional, and we were against team 637 for every single one of our qualifying matches. This was definitely the least "random" case I have seen.

http://www.thebluealliance.net/tbatv/team/694/2007

[artdutra04]

Back in 2007 (not relevant to this years match algorithm but still interesting) we were at the Trenton Regional, and we were against team 637 for 6 out of our 7 qualifying matches. This was definitely the least "random" case I have seen.

http://www.thebluealliance.net/tbatv/team/694/2007The 2007 algorithm was seriously flawed that year, and vastly different than the algorithm used in any other year in FIRST. That year, one team was randomly chosen from a low group of team numbers, one from a middle group of team numbers, and one from a high group of team numbers. As a result, there was a much more limited sample size of available teams in the algorithm, leading to match schedules with many of the same teams constantly playing with or against each other, and no one else.

[waialua359]

The 2007 algorithm was seriously flawed that year, and vastly different than the algorithm used in any other year in FIRST. That year, one team was randomly chosen from a low group of team numbers, one from a middle group of team numbers, and one from a high group of team numbers. As a result, there was a much more limited sample size of available teams in the algorithm, leading to match schedules with many of the same teams constantly playing with or against each other, and no one else.
I agree.
We went to NJ that year and played Miss Daisy 4 times and Robotic Plague 3 times. We had a total of 7 matches.
My only complaint is that we went to NJ to meet many new teams. Why go through an entire weekend only to meet a few.
This happened though in '08 also for a few regionals. In Chesapeake '08, we saw the Robo Raiders (75) 4 matches in a row. 3 Against and 1 with them.

[Tom Saxton]

My only complaint is that we went to NJ to meet many new teams. Why go through an entire weekend only to meet a few.
This happened though in '08 also for a few regionals. In Chesapeake '08, we saw the Robo Raiders (75) 4 matches in a row. 3 Against and 1 with them.

Given a finite amount of time to generate the schedule, no schedule will be perfect. In the Chesapeake '08 regional, 75 and 359 were the only pair of teams that saw each other 4 times, all other pairs occurred three or fewer times. Even those two teams saw a total of 30 (359) or 32 (75) different teams.

Last year's implementation did a poor job of guiding the scorekeepers in choosing the minimum gap between matches. In the case of Chesapeake '08, that parameter was set too high (8) thus the scheduling algorithm was overly constrained. Because of the constraint to give teams at least 8 matches between rounds, the order of teams could not change very much from one round to the next. This caused excessive duplication among pairings at several of the larger regionals in 2008, and was not brought to my attention until I was waiting in the Houston airport for a connecting flight to Ecuador which kept me out of the country until after Atlanta.

That problem was fixed for this year. For example, the Chesapeake '09 regional had a minimum gap between matches of 4, and the resulting schedule was much better, with just two pairs that appeared 3 times and all other teams never seeing another team more than twice.