How is matchmaking done at competitions? (Kind of a rant)

5 Likes

Here is the algorithm FMS uses.

10 Likes

Thanks! This will be very interesting to read.

“It’s especially desirable for a given team not to have the same team as a partner more than once, nor as an opponent more than once. If match duplication does occur, it’s preferable that a team that is seen twice is seen once as a partner and once as an opponent, rather than twice in either role.”
33 twice as an opponent is very unlucky then…

In the past our team has gone against team #33 twice (Arial Assist) and team #494 same year, different year. Its just your luck on that day. Also we have played with and against #33 in DCMP.

Something to think about is with districts (smaller events and more matches) it gets much harder to hold those ideas. With 12 matches, you play against 36 other robots and with 24 other robots. It is impossible to build a perfect schedule that adheres to the harder rules (no back to back matches for a team) so you’re very likely to replay a few opponents. The scheduler doesn’t have a concept of powerhouse teams, so it just chooses teams the teams that happens to at random. It’s just luck, partially attributed to the event sizes as well. And if your event has less then 36 teams, you are guaranteed duplicates.

4 Likes

I think many teams have felt your pain. Although it may be small consolation after your Centerline experience, it may help to know that situations like yours were once even more common.

Quoting from the link in an earlier post,

Note the final clause above, about not knowing in advance which teams will turn out to be more effective than others. A dozen years ago, FIRST experimented with a matchmaking algorithm that substituted “low team number” for “more effective”. There were some absurd results; one of those was grouping 1114 among the “less effective” teams despite their history of competitive results, which was quite strong even back then. The scheme was roundly criticized, and dubbed the “Algorithm of Death.” The more balanced, but still imperfect, algorithms that FIRST has used since that time have been, at least in part, developed in response.

10 Likes

I feel your pain. My team was easily one of the best hatch bots and the most consistent lvl 2 climb (second most consistent climb) we ended friday rank 37/40. Saturday we ranked up to 24. According to scouting data we were top 20.

1 Like

For scouting…We for past five seasons have tracked scheduling of every single “partners” and “foes” and “likely captains” I will just say IMO the algorythm is not unbiased and leave it at that. Patterns dont lie.

1 Like

Uh …

If you mean that your observations are documented somewhere else, please point to that.

If you meant that you don’t want to start a discussion about matchmaking algorithm bias, then it might have been better not to respond.

5 Likes

Its a thread about matchmaking I stated my observations of the matchmaking over time. I am not complaining just my observation from scouting events.

Do you have any data you would be willing to share? I’m sure it would be very interesting to a lot of people.

Though algorithms are written by human, they are executed by machines and normal machines have no feeling or bias. I don’t believe humans went out of their way to write algorithms specific to an event or to un-favor a specific team.

3 Likes

No , what we do is put the schedule in ecxel highlight every partner and foe up until we finish playing them and on day 2 add in top 16 likely captains. So I notice patterns over the years that cannot be expalined away easily by a truly random weighting. Little things like allaince 1 high profile teams in Late Qual in OC together are interesting and another data point. As are other past match ups. I’m perseptive to scheduling data since I highlight all the teams we watch painstackenly.

I could be wrong, it may be random as random does have patterns too. I’m competitive so perhaps I’m biased. So take what I say with a grain of salt.

Some times the match Gods smile on you and some times they smack you upside the head. It happens to everyone (although it seems that some teams are luckier than others

Just be glad you weren’t around for the AOD (algorithm of death)

4 Likes

Given that it takes a human chosen initial seed, it is probably possible to introduce bias.

There are problems with the match scheduling system that allows too many blowouts. That has been discussed in several threads including this fall.

Does anyone believe that TBA’s OPR rankings (or something similar) should play a part in ranking teams, or at the very least be shown with the team alongside their ranks at events?

OPR isint the greatest metric to begin with, and I sure wouldnt want it to determine my rank, esp in years like 2018.

4 Likes

Would you say that you are against the best teams like 50% more than you’re with them?

Because on my team we’ve noticed that almost every match we have 50% more opponents than partners.

26 Likes

While this is true, keep in mind that you have 50% more opponents over the course of an event than partners, and pairing uniformity is not the highest priority of the algorithm. You’re more likely to play the same team twice than to partner with them twice.

Every team will almost certainly play someone twice, you happened to draw 33.

2 Likes

I think there’s a fair bit that people need to realize about the match scheduling algorithm. @plnyyanks posted it in the second post of this thread, and I suggest everyone read and ponder its implications. Namely how round uniformity and minimum match separation are baked into how the algorithm runs before pairing uniformity is graded. In other words, round uniformity and minimum match separation are guaranteed criteria, while pairing uniformity is optimized for in the subset of schedules produced.

That’s to say, every schedule will adhere by spacing teams’ matches out based on the minimum match separation criteria defined by the FTA. At smaller events this may be 2 matches, at larger ones it may be more. After that is achieved, THEN the algorithm attempts to balance the pairing uniformity as best it can (and the best results are used).

In practice, because of the priority given to round uniformity and minimum match separation, a schedule with zero “repeats” is often not possible. There are simply scenarios in which team A and team B will end up in the same subset of a round uniformity section at some point due to having to avoid either team playing back-to-back matches.

As @Thad_House mentioned, this is significantly exacerbated by smaller events. Not only is there the defined criteria of needing 36 opponents in a 12 match schedule, but having fewer teams at an event also shrinks the length of each “round,” meaning teams will have fewer matches between each scheduled match, which makes the minimum match criteria even more difficult to achieve (which further limits the pool of options to be optimized for pairing uniformity).

And as @SoftwareBug2.0 and @thatnameistaken is taken mention, you play AGAINST 3 teams in each match and WITH 2 teams in each match. No matter how you shake it out, almost every match schedule will be stacked with more high level opponents than high level partners simply because of that. Moreover, this also means that when violations of the pairing uniformity happen, theres a greater chance of it being in your opponent column that partner column.

5 Likes