How is matchmaking done at competitions? (Kind of a rant)

While I am sure this is extremely rare, it isn’t completely unheard of for a volunteer to re-run the schedule generator until e.g. two top teams play each other, etc. I haven’t heard of this happening in the last several years though.

2 Likes

There are probably several ways to address this, including grouping seeds of teams and rotating the schedule assignments among that smaller group.

For the past several years, Scorekeepers have been required to contact FIRST support prior to running matchmaker a second time.

4 Likes

It was done only in one year, 2007, and done VERY badly with very obvious errors. I suggest reading Caleb’s TBA post which addresses the repeated team problem from 2007, and also moves toward a more balanced schedule.

I don’t understand why we use this convoluted scheduling system that so often leaves people baffled and upset when better solutions that meet fairness criteria AND can make life easier for the field staff are readily available.

The first step to using pregenerated schedules is for someone to develop an algorithm that given a longer (but finite) time, can generate schedules that are better then matchmaker and cheesy arena. Until someone steps up to do this, like the Saxton’s did with Matchmaker, all this talking is a moot point.

1 Like

Wouldn’t it be possible to generate a set number of schedules per number of teams that only takes an input of team numbers in numeric order? After the scheduler takes in the team number it could randomly number off each team competing to assign them to a set of matches.

To add in to some complaining about the current scheduler, I heard about several teams who had back-to-back matches then 8+ match breaks at the Sundome event last week. Every single one of our matches was within 3 qualification matches of another one of our qualification match, and we had at least 3 sets of matches with a 2 match turnaround.

Very informative thread? We had 254 twice as an opponent this weekend in SFR :frowning: . I have wondered how this happens.

Personally, all I want for the Matchmaker algorithm is the ability to include some additional, event-specific constraints. Quite often, by the time the schedule is generated the LRI is going to have a good idea of which teams might need a little more time. Being able to ensure those teams are not in the first few matches could make all the difference for their experience. This past weekend I also had a case where a team’s red bumpers were going to be arriving on the bus… but the bus wasn’t going to get up there until 11 on Friday, due to the blizzard. An exceptional circumstance, but it would have been nice to be able to ensure they would only be on the blue alliance in the morning. Instead, they ended up on red for their very first match and we had to scramble to get them something that would let them play in that match (Luckily, their red bumpers showed up early, and they were able to swap them out while in the queue!).

I understand what you’re saying. However, I don’t think we necessarily need a better algorithm than MatchMaker, as much as we need to have a way to store away and keep the “best schedule” that MatchMaker ever comes up with, for a given # of teams with a specific # of rounds.

I guess that is kind of a “meta-algorithm.”

Let’s say that maybe last week, an event “got lucky” and MatchMaker happened to generate the “perfect” match list for an event of 36 teams playing 12 rounds. Maybe there was only a 1 in 1,000,000 chance of that match list ever being generated. However, the way things are currently, if it ever happened, the “perfect match list” will be lost again. Just keeping the “best ever” match lists, and using them, instead of new random trials every time MatchMaker is run at an event, would make things better, even if not perfect.

Just having one decent computer run MatchMaker repeatedly for 24 hours, and keeping the “best” match list every time, would generate a set of better match lists for 12-round events (what are played in most districts) of between 35 and 48 teams than what can be generated at any given event in the 15-or-so minutes available for that purpose.

2 Likes

Me neither.

However, I think part of the reason that the fixes don’t happen is the discussion in threads like this one, when folks that are unable to distinguish the solvable portions of the problem (e.g. having Team A play team B 4 times while there is a Team C that Team A never plays) from some of the unsolvable ones (e.g. having to play against the 4-time world champions while never having them as a partner.)

Another part of the reason is what Joe Ross mentioned:

I’m confident that the CD community could implement a way to “save off and reuse” the best schedules generated by MatchMaker. However, without knowing if the decision-makers at FIRST would be willing to use such a solution, it is hard to justify an investment of time in an effort that would make things better, but might be rejected for other reasons of which we are unaware.

Seems like I should try to chat with folks at FIRST about this next time I have their ear for a bit.

See Caleb’s data earlier in the thread comparing CheesyArena to IdleLoop Matchmaker schedules. No substantial difference in quality of schedules produced.

I have read Caleb’s articles on TBA regarding “strength of schedule.” Are those the articles you are talking about?

Unless I’m misunderstanding something, I’m pretty sure what Caleb has been focusing on has been comparing schedules that are “hard” or “easy” based upon how good the teams were that one faced. As I understood it, he is searching for “forward-looking” metrics that could be used to generate schedules that are more equally “hard” or “easy” based upon the specific teams at an event.

From my perspective, unless I’m missing something, what is being sought in that discussion is something similar to the “handicapping” that is done in some sports and games, such as horse racing and golf. Wikipedia defines handicapping as “the practice of assigning advantage through scoring compensation or other advantage given to different contestants to equalize the chances of winning.” https://en.wikipedia.org/wiki/Handicapping

Such an “equal strength of schedule” objective would end up doing things like having top-tier teams play a greater share of their matches with bottom-tier teams as partners, and having top-tier teams play against one another a disproportionately high percentage of the time. If the “handicapping” were done perfectly, then every team would go into an event with an equal chance of winning.

That’s not what I’m suggesting.

Rather, I’m suggesting that match lists be such that teams don’t have extra matches with or against other teams, at least not more than is minimally possible. Ideally, each team would play against all other teams N or N+1 times. (i.e. play all teams either N or N+1 times. No team should be played against N+2 times if there is a team that would be played against only N times.) The same would be true for partners – all teams would have a specific partner either M times or M+1 times.

I believe this latter goal could be accomplished readily, and without folks opposing that goal, at least if they understood what it is intended to fix (and don’t try to have it fix things it doesn’t fix…)

Even with such a “near-ideal” match list, there will still be “easier” and “harder” spots - for instance, by the “luck of the draw” a team could come up with a schedule that happens to play against the best teams N+1 times and the worst teams N times. That would be a hard schedule, but such is unavoidable unless one either uses a “handicapping” approach, or only uses “perfect” schedules.

2 Likes

More matches per team give more balanced schedules:
image

Smaller events also give better schedules, but that’s basically the same as saying more matches per team give better schedules, as they are strongly correlated (and the outliers are higher levels of competition so they can’t be directly compared in this context for other reasons):
image

This isn’t a concern if using pre-generated schedules and randomized team indices. If you are use my “balanced” schedules with seeds going into the event, all teams at the event should have very similar schedule strengths, I did not design it to give better or worse schedules to higher or lower seeded teams. You are free to review those schedules though and let me know if any of the seeds seem like they are getting a drastically better/worse schedule. There are also simple ways to alleviate this concern, one that comes to my mind is shuffling all seeds up or down up to 5 positions, which would might make things slightly less balanced but makes “gaming” the seeding provide negligible benefir since you can’t actually know what your seed will be.

1 Like

This is a misunderstanding of what my objective is/was. The optimization criteria of the “balanced” schedules I made was broadly that for every team, the overall quality of their partners is as close as possible to the overall quality of their opponents when looking at their entire schedule. So high-seeded teams should be getting similar schedules as low-seeded teams with this methodology.

The 2007 algorithm in contrast, and others that have been proposed, sought to provide balance on a per-match basis. The only way to achieve this criteria is to consistently give low-seeded teams high-seeded partners, and vice versa. This actually would go in the opposite direction of what I tried to do, as this would increase schedule imbalance, since some teams would obviously have much better schedules than other teams.

EDIT: Deleted edit

1 Like

Do you have an issue with a rookie team paired up with another rookie team even if the quality of teammates is close to the quality of opponents?

Like qual 69 at SFR(https://www.thebluealliance.com/match/2019casf_qm69) two rookie teams were partners.

I’m not sure exactly what you are asking. I’m not intrinsically opposed to rookies being partnered up, and the “balanced” schedules I created do not prohibit this kind of situation from occurring.

As mentioned earlier in the thread, the last time the scheduler took team age into account, it resulted in the Algorithm of Death where you’d end up facing an incredibly strong rookie team even more than expected, because the system thought all young teams were weaker teams.

For clarity, in this post you’re defining “better” based upon your strength of schedule metric, correct? You’re not defining “better” based on minimizing/balancing repeat appearances or maximizing teams interacted with (aka pairing uniformity).

Just want to clarify this, as smaller events actually do worse for pairing uniformity.

1 Like

IIRC I thought that FiM has an algorithm for MSC where rookies aren’t paired up with rookies and wondered if that would be considered for other FRC matchmaking algorithms.

Correct, we should be careful with terminology here when there are so many (often competing) criteria a good schedule should have.

1 Like