How is matchmaking done at competitions? (Kind of a rant)

+1. The above is a frequent problem. It usually arises when the well-intentioned volunteer running the “MatchMaker” program to generate the schedule has set the “Minimum Match Separation” parameter too high for the number of teams at the event. Having pre-canned excellent schedules would avoid this problem entirely.

Facing a specific team 4 times when there are other teams that are never faced is exactly the kind of situation that should be avoidable with pre-generated excellent schedules.

It seems to me that schedules should be possible so that all opponents are faced either N times or N+1 times. (Or, if you’re a glass half empty person, N times or N-1 times. :wink: ) Furthermore, that should be possible while also having all teams be partners M times or M+1 times, too.

2 Likes

Volunteers have zero input on setting those parameters.

The inputs that are available prior to generating the schedule assignment are the event schedule and the team list. The schedule is determined by these inputs:

  • Match cycle times (time between starting match X and match X + 1)
  • Start/end times each day
  • Breaks (e.g. lunch)
  • Number of matches per team

The work of determining match separation, quality, etc. is 100% done by match maker. No volunteers can say “I want lower/higher quality matches this weekend.”

The idea of pre-generated schedules is certainly interesting and worth discussing, but it wouldn’t address any of the above issues- it would only serve to minimize the time spent generating the schedule (which, while it can be annoying, isn’t a significant hinderance).

4 Likes

What issues are you referring to? As this most definitely solves many of the issues above. The ability to generate “all” or many many more schedule options and choosing the best of those will improve the schedule dramatically over the relatively small pool that the machine can generate with the small allotted resources and time pool.

That makes the assumption that all of the schedules that come out the MatchMaker that meet the minimum MatchMaker rules are all equally optimized to minimized replaying the against/with the same team and time between matches, which simply isn’t true.

1 Like

That’s a fair point- pre-generating the theoretical best possible schedule would be feasible.

I don’t think it will solve everyone’s conception of their schedules and the fairness of playing against team X, but it would help remove some of the points of contention. I just wanted to call out that even with an ideal pre-generated schedule, you will still run into many of the issues raised here. You may still play against the same team multiple times, and you may still play in matches only 1 match apart.

Cheesy Arena already has such pre-generated schedules and I am 100% on board with FIRST using these or similar (but I’d like them to be published so we can independently verify).

I independently compared the Cheesy Arena schedules against 2018 schedules, and found that they were of comparable quality (although they seem to prioritize different aspects of a good schedule differently). For the record, I personally don’t think volunteers are biasing schedules, but I can see how reasonable people could come to that conclusion. I’d like to make the scheduling process as transparent as possible to minimize the number of people that have this view. Also, it just seems dumb to me to waste so much processing power and time at each event generating millions of theoretical schedules that just get thrown out.

I think any discussions about generating “balanced schedules” in official competitions are premature until the FMS implements pre-generated schedules or something similar first. I really don’t want to add any more criteria to the current scheduler as any improvements in “balance” will come at the expense of either unique opponents/partners, color, or match gap. If you want to try out “balanced” schedules for your offseason comp though, I have made them.

6 Likes

Agree that it may be mathematically impossible to generate an ideal schedule but many (maybe all) of the reported problems are well outside the expected error. Doing this and making the pairings public would also allow the community to ensure fairness.

This will not help teams that think there should be a rookie “advantage” by not being paired with other rookies or something silly like that. That is inherently “unfair” and I think a completely bad idea to treat any team different than any other because of their perceived quality or lack thereof.

1 Like

I don’t think pre-generation of schedules will significantly change the quality of the schedules produced, and the data that Caleb linked above seems to bear that out. We’re long past the time where events weren’t running the matchmaking algorithm on “best” each time.

1 Like

And the order in which the teams are entered into the schedule.

This has an interesting effect. It is readily apparent that groups of teams stay together and roll through the schedule playing amongst each other. As Sean, pointed out, much of this occurs because of the constraints match separation and limit on replaying with and against teams. If your team happens to be in the “wrong” group that is loaded with powerhouse teams, your ranking can take a serious hit or gain a significant boost.

An alternative approach, instead of keeping the team order fixed and running the match scheduling algorithm (which fairly often has to be rerun for a satisfactory outcome), would be to have a set of fixed match schedules for different sized events, and then to submit a certain team order to come up with a desired characteristic. The predetermined schedules could ensure both of the constraints are maintained (and probably limiting the number of team rematches to a minimum). I played around with one version of rotating assignment among three pools of teams and got pretty far, but didn’t have the time to iron out how assigning teams after the first rotation through. Teams could be assign to the pools in any number of ways, including random or by a measure of team strength or by balancing strength within each pool.

This approach would make the match schedule easier to prepare at local events, and opens up the possibilities for achieving better schedule balance than currently occurs.

Interesting. I haven’t been involved “behind the scorer’s table” with schedule generation in a few years. It sounds like things have changed a bit. Indeed, the parameters you mention above, with the exception of “number of matches per team,” aren’t even inputs to the “MatchMaker” algorithm (see https://idleloop.com/matchmaker/ ), but are rather “don’t cares” for the purpose of determining which teams play in which matches. Clearly, there must be some sort of “wrapper” program which provides those parameters to assign match times to the match list generated by MatchMaker.

Interesting. It didn’t used to be that way. If the match separation parameter is chosen poorly, then unbalanced match lists are a necessary result. If whatever “wrapper” program is being used to add the match times (derived from match cycle times, start/end times each day, breaks, etc.) from the MatchMaker-generated match list is selecting the “minimum match separation” time poorly, then that could be part of the problem.

From my perspective, what you mention is an even bigger reason to have the match list pre-generated / pre-vetted, and have the “at the event” computation simply do two things (1) randomization of which team gets which “spot” in the match list and (2) assignment of specific match times to each match in the match list (turning the “match list” into a “schedule”) based upon match separation times, daily start/finish times, breaks, etc.

Your comment makes me realize that there are multiple different issues being discussed in this one thread. Because of that, the details of each problem (and possible solutions) are different for the distinct issues.

I was focusing on what I think is a solvable problem, in that currently, tournaments are using “unbalanced” match lists, where team A may play against team B as many as 4 times while they may play against team C 0 times! (see @MoistRobot’s post above). Similarly, at 1519’s Week 1 event this year, we faced an opponent 3 times, while there were other teams we never played against. Such a situation shouldn’t happen, and I believe it can be fixed with pre-generated match lists.

There are other folks in this same thread discussing a similar, but related issue, of whether or not the MatchMaking algorithm does anything to take into account the “level of play” of individual teams when generating the match list. It does not do this. There were some years that schedules attempted to do that, which resulted in the “algorithm of death” AOD referenced above in a few posts. (Edit: I’ve subsequently learned that was only one year, 2007, in a post from Citrus_Dad, further below.) Personally, I don’t think the MatchMaker algorithm should try to balance “difficulty of matches” given strengths of individual teams, other than avoiding things like playing some teams N times while playing other teams N+2, N+3, and N+4 times!

Rather, I’d randomize assignment of teams into a “balanced” match list.

3 Likes

This isn’t true. From the IdleLoop MatchMaker page:

While the random order that teams are dumped into the first round can impact the schedule throughout the event, the order in which teams are entered into the schedule has no impact.

2 Likes

I maybe wasn’t clear- when I said the “event schedule” takes in those parameters, I was referring to the match schedule without team numbers. Match maker takes the schedule and plops the team numbers in, so by the transitive proper they are inputs, but not directly. For example, I don’t believe a lunch break would ever impact a decision on match separation.

That’s interesting- do you have a source for that? I didn’t see it in the match maker docs, but I did see this note:

The algorithm begins by seeding the match schedule with a simple schedule: the teams are dumped in the schedule in random order for each round. Thereafter, teams are only rearranged within rounds.

I do recall some discussion a few years ago that seemed to indicate that championship division assignment may have suffered from this issue, but I hadn’t seen this in reference to match maker before.

1 Like

As an O(N!) operation where N is the number of teams, feasible is perhaps a bit of a strong characterization.

2 Likes

The question is what is defined as “random”? As far as I know, no “random” computer algorithm is truly random. It works from a preset list of numbers in a particular order, and then a seed value is chosen that selects at what point the “randomnzation” begins in that number list. The order in which the teams are “dumped” then affects the order in which the teams are randomized and then assigned.

In any case, I read Caleb’s TBA post which was a fascinating solution to this problem and appears to eliminate the “outlier” matches and the misrankings produced solely through scheduling flukes. Since we use Chezy Arena for our offseason event, we’d be interested in trying this out.

BTW, Chezy Arena appears in general to perform better than the standard FMS. Our FTAs and other field admin are generally happier with Chezy Arena.

Is there a certain event size (teams, matches) that is better at achieving a strength of schedule balance? How would you balance an event with x rookie teams?

I’d hate to have pre generated schedules for a 40 team, 12 match district where eventually everyone knows the possible schedules and they all have figured out the seed #15 has the easiest schedule. Everyone would then start arguing about where they are seeded, just like in the NCAA basketball tournament.

While I am sure this is extremely rare, it isn’t completely unheard of for a volunteer to re-run the schedule generator until e.g. two top teams play each other, etc. I haven’t heard of this happening in the last several years though.

2 Likes

There are probably several ways to address this, including grouping seeds of teams and rotating the schedule assignments among that smaller group.

For the past several years, Scorekeepers have been required to contact FIRST support prior to running matchmaker a second time.

4 Likes

It was done only in one year, 2007, and done VERY badly with very obvious errors. I suggest reading Caleb’s TBA post which addresses the repeated team problem from 2007, and also moves toward a more balanced schedule.

I don’t understand why we use this convoluted scheduling system that so often leaves people baffled and upset when better solutions that meet fairness criteria AND can make life easier for the field staff are readily available.

The first step to using pregenerated schedules is for someone to develop an algorithm that given a longer (but finite) time, can generate schedules that are better then matchmaker and cheesy arena. Until someone steps up to do this, like the Saxton’s did with Matchmaker, all this talking is a moot point.

1 Like