Is the qualification match robot randomizer really "random"

[coalhot]

I'm not saying this is fixed, im just trying to say that the computer that picks the alliances should be more fair. Take for instance the last qualification match in Philly. 304, 2558, and 2559 were paired against, 1640, 1712, and 381. 304 was on an alliance with 2 rookies, one of which had to e-stop their robot a few seconds into the match. Meanwhile, they were against 2 hurdlers and a racer. What im saying i guess is that the best way to work this out would be to have an alliance like 365, 2558, 304 VS. 103, 2559, 84 (providing enough of each tier robot is available). I guess the algorithms would need to be tweaked in that case...

Thanks for all the responses in this topic!!

~Philip

[XaulZan11]

Originally Posted by coalhot

I'm not saying this is fixed, im just trying to say that the computer that picks the alliances should be more fair. Take for instance the last qualification match in Philly. 304, 2558, and 2559 were paired against, 1640, 1712, and 381. 304 was on an alliance with 2 rookies, one of which had to e-stop their robot a few seconds into the match. Meanwhile, they were against 2 hurdlers and a racer. What im saying i guess is that the best way to work this out would be to have an alliance like 365, 2558, 304 VS. 103, 2559, 84 (providing enough of each tier robot is available). I guess the algorithms would need to be tweaked in that case...

Thanks for all the responses in this topic!!

~Philip

Not every match will be perfectly fair. Some matches you get bad partners, some matches you get good partners. In the end, however, it should even out. Personally selecting matches not only would take a lot of time, but be prone to biases and just would cause a ton of problem.

[bduddy]

Originally Posted by GaryVoshol

I saw a practice match on Rack-n-Roll where an alliance with zero robots won. The HP scored one ringer, and that was the only score of the game!

It happened at least twice at SVR 08; the fact that an alliance starts with 24 points really helps.

As for the original topic-can anyone figure out what the minimum match separation was? I'm far too lazy to page through that schedule, and teams that were there probably can figure it out pretty quickly.

[coalhot]

Just on a side note, will there be videos of the Philly regional on TBA?

~Philip

PS. i guess i should have called this thread "why did the same teams get picked together so many times in the Philly regional?"

[Dillon Compton]

Originally Posted by Rich Kressly

You also need to realize that if you desire to win, you'll have to best the "top tier" anyway. Eight of us on 1712 spent our entire lunch break repairing a claw just for the privelege of going up against the 103 alliance - and I do consider it a privelege.

-Rich

I'm going to hijack this thread for a second to thank Rich and 1712 for giving 1391 the chance to share in that privilege of going up against 103's alliance. I thought 1712, 1391, and 381 put up an excellent fight against a very competitive alliance (even if we did get eliminated in QFs).

I also want to echo that match pairings and distribution have been MUCH better this year than in my previous experience. This algorithm is a huge step forward in generating good, distributed matches.

//Dillon

[coalhot]

Since we all are getting off-topic, ill do too and give a plug to Metal Moose. So good hurdler they could throw it out of the ring (literally). Great 'bot. And 304 had a match against them, and almost lost were it not for a good matchup. Ill look for you guys in the Atlanta vids!!

~Philip

[Knippschild]

I was a scorekeeper for the Philadelphia Regional. I was present when the match schedule was generated (by the lead scorekeeper).

The FMS (the event management software) uses a very complicated algorithm to generate a completely RANDOM schedule. When FMS calculates who should be paired with whom, and who should be pitted against whom, it has no knowledge of the team's "big name"ship nor of that team's standings at other regionals.

The match parings take about 3-5 minutes to generate. Last year's algorithm only took seconds to generate... this one is definitely a lot more random by definition.

Also, once a match is generated, it is impossible for a human to edit the schedule and rearrange the pairings to fit some sort of side agenda.

I assure you, the Philadelphia Regional's schedule was generated once, and that was the official schedule used for the qualifying rounds.

[demps45]

I have often wondered if the "random" pairings were actually "random" too. We attended the Greater Toronto Regional this past week, and after the pairings were distributed, some teams complained that the pairings were not fair, so the officials ran the program again and came out with pairings that were satisfactory to those complaining. the officials said they had used the wrong algorithm for the number of teams attending the event. That tells me that mistakes do happen. I am not questioning the integrity of any of the officials, only the method used for pairings. As luck would have it, we had much better pairngs "before" the the pairings were rerun. Also, the "better" teams ended up with better pairings "after" the program rerun. Go figure!!

[dlavery]

Originally Posted by Knippschild

The FMS (the event management software) uses a very complicated algorithm to generate a completely RANDOM schedule. When FMS calculates who should be paired with whom, and who should be pitted against whom, it has no knowledge of the team's "big name"ship nor of that team's standings at other regionals.

The match parings take about 3-5 minutes to generate. Last year's algorithm only took seconds to generate... this one is definitely a lot more random by definition.

As previously noted, the match pairings are anything but random. Rather, it is a highly constrained process, with many factors influencing how the pairings are established. As outlined in Section 9.3.2 of The Manual, there is a specific set of criteria that are used by the match pairing algorithm to generate the match list. The match pairings generated by the algorithm are evaluated for a "best fit" against these criteria as the list is generated, and adjusted mid-process if necessary. So, by definition, the resulting list is NOT random. Nor would you ever want it to be random. A truly random list will have a significant probability of having teams playing back-to-back matches, being allied with or opposing the same teams, or having uneven numbers of matches. All of these are situations that we clearly want to avoid.

-dave

[Steve W]

Originally Posted by demps45

I have often wondered if the "random" pairings were actually "random" too. We attended the Greater Toronto Regional this past week, and after the pairings were distributed, some teams complained that the pairings were not fair, so the officials ran the program again and came out with pairings that were satisfactory to those complaining. the officials said they had used the wrong algorithm for the number of teams attending the event. That tells me that mistakes do happen. I am not questioning the integrity of any of the officials, only the method used for pairings. As luck would have it, we had much better pairngs "before" the the pairings were rerun. Also, the "better" teams ended up with better pairings "after" the program rerun. Go figure!!

The issue at GTR was that the match list was printed before being checked. There are defaults in the program that were set quite high. When the matches were looked at, it was discovered that teams were playing with or against all of the same teams. The number of matches between played matches was changed from 9 to 7. This allowed a greater mixing of teams thus not as many with/against matches. This allowed the teams to play with/against a larger number of teams because of a 66 team regional.

As for who gets better pairing, I guess that would depend on everyones definition of what a better pairing is. My definition would be to play with/against as many different teams as possible so that we can have the opportunity to gain knowledge and workability with many teams.

[The Lucas]

This year's algorithm is a tremendous improvement over the previous years IMHO. I have read up on it and downloaded it to test(I recommend reading it before posting in this thread). I agree with how it approaches the problems, and love how it is customizable from the command line.

The most important factor in evaluating a schedule, at least in my opinion, is paring uniformity. Here is my grading for the 3 schedules that it generated for my teams:

FLR #365: A Philly #365: C Philly #1495: B

Making broad judgments about the algorithm based only on 3 team schedules at 2 event is flawed because it is a minuscule sample size. However, I think I can provide some analysis. The big problem with 365's Philly schedule is the series of 4 matches (in a row) in which we were with/against 357 (1 with, 2 against). 486 was also involved in back to back matches in this series in a home and home aspect (with one match against the next). This series (or cycle) suggests to me that the algorithm was having trouble maintaining maintaining paring uniformity around the minimum match separation constraint.

I think the default minimum match separation was higher than optimal for Philly. In the 3 schedules, the lowest match separation was 4 at FLR and 5 at Philly so I guess that was the minimum. My guess is that minimum match separation is set to

# of teams / 6 (round down) - 2

since FLR had 40 teams and Philly had 44 (42 would yield exactly 5).

I would be willing to sacrifice some match separation for better match uniformity. At events the size of FLR and Philly, I personally would be willing to go for a minimum match separation of 3. The software already gives the scorekeepers the option to specify a minimum match separation (although I don't think many scorekeepers do), but I think the way the program computes a default match separation needs some tuning for the next revision (even it is just subtracting 1).

What does everyone else think about the tradeoff between pairing uniformity and minimum match separation?

Originally Posted by Knippschild

I was a scorekeeper for the Philadelphia Regional. I was present when the match schedule was generated (by the lead scorekeeper).

The FMS (the event management software) uses a very complicated algorithm to generate a completely RANDOM schedule. When FMS calculates who should be paired with whom, and who should be pitted against whom, it has no knowledge of the team's "big name"ship nor of that team's standings at other regionals.

Since volunteers from both 357 and 365 were present when a schedule that called for them to face each other 3 times was generated, I think we can all lay the "algorithm paired big names theory" to rest (not that anyone has been talking about that lately)

[Tom Saxton]

Cathy and I have just released a new version of MatchMaker, the schedule generation program, to FIRST. This new version includes several changes to avoid the clumping effect first reported on this thread.

The clumping was caused mostly by setting the minimum gap between matches too high, which was mostly caused by MatchMaker picking a default that was too large for larger tournaments. To a lesser extent, the way the algorithm was seeded contributed to this problem.

The changes are:

1. MatchMaker now limits the values specified for the minimum match separation so that values that will yield poor schedules will not be allowed.

2. The default chosen when MMS is not specified on the command line is improved, especially for larger tournaments.

3. The starting schedule that is used to seed the match generation algorithm is now randomized both to reduce clumping and also to mix teams up to improve fairness even in situations where constraints on the schedule force some amount of clumping.

4. MatchMaker has a new command line option, -x, which will cause it to just print out the lowest and highest allowed value for MMS, which may be used by the scorekeeper interface to show the range of allowed values.

If you want to give it a whirl, you can get a copy of the build from the MatchMaker download page:

http://www.idleloop.com/matchmaker/download.php

There is also a white paper on the same site that describes the algorithm used.

Please let me know by direct message if you find any issues with this release.

[Seth Mallory]

Tom,

I want to thank you and Cathy for your efforts. The improvements in 2008 over 2007 were impressive. I for one was satisfied with that system. You must have spent a lot of time to engineer this new system. Thank you

Seth

[coalhot]

Hey,
I was very tied up during the last few weeks, but i have to say (from seeing it work) good job!!!! It works to perfection. Couldn't be happier!

(now i have to find a way to remotely use the randomizer to our advantage )

Again, thanks!!!

--Philip

[Elgin Clock]

In doing some research with the "randomizer" this year to improve our scouting techniques, & while I agree with the rest of the folks here that 2009's version is VASTLY improved over previous years, I only have one suggestion to add to the "randomizer" for future years.

With this paper I generated for the NJ Regional: ( http://www.chiefdelphi.com/media/papers/2224 & testing the match list in NYC which I didn't release a paper for) I found that the placement of teams during qualification matches in the alliance stations were not equal.

For instance, one team could play a total of 8 matches, of which 4 of those would be in one Alliance Station spot (As shown in that paper, Team 103 being in spot Blue 3 a total of 4 times for example).

I'm sure this could be an easy fix or addition to the algorithm if it was approved.

I know, speaking from an operators perspective, it's nice to bounce around your placement on the field rather than always stand in the same spot & have the same view on either red or blue side of the field.