Section 11.6.2 of the manual remains incorrect, as it has for years.
I just reviewed MatchMaker 1.5 and the scoring algorithm for schedules remains the same as it was when I reviewed it here. If you don’t want to read that entire post here is the tl:dr:
The schedule “penalty” increases by 5 when you have the same partner twice, but only increases by 1 when you have the same opponent twice. Schedules with lower overall penalties are always selected instead of schedules with higher overall penalties. Therefore, you are disproportionally (that is, even beyond a 3:2 expected ratio) likely to have duplicated opponents over duplicated partners. The disproportional bias is so strong that, of the following options:
A. See the same team as an opponent 3x and as a partner 0x
B. See the same team as an opponent 0x and as a partner 2x
The scheduler strongly prefers option A (penalty = 4) to option B (penalty = 6)
Based on these results, priorities 2 and 3 of section 11.6.2 of the manual are clearly in the wrong order and should be changed:
Sent another email, I basically copied the above post (with some nicer wording) so I won’t repost unless someone wants to see it.
I am not affiliated with any team, so I do not have access to the Q&A, if someone with access wants to submit a question to the system I would appreciate it.
There appears to be a discrepancy between the match scheduling software (MatchMaker 1.5) and the game manual. According to section 11.6.2, it is a higher priority of the schedule to “minimize the number of times a team plays opposite any team” than it is to “minimize the number of times a team is allied with any team”. According to Caleb Sykes’ analysis, the MatchMaker scheduler scoring favors duplicate opponents over duplicate partners by a ratio of 5 to 1, which is contradictory to 11.6.2.
Hooray! The link to idleloop is a nice bonus as well.
Small part of me was hoping they would change the scoring criteria instead of changing the manual, but I still will definitely take the win, at least now the manual and the algorithm match.
I really don’t like this change, the manual had the right order, playing with the same team more than 1 time is much better than playing against the same team more than 1 time
Last year we played against the 1st seed team 3 times in our DCMP and it really ruined our chances to be an alliance captain.
They need to change the algorithm, not the manual.
You play against 3 teams per match, you only play with 2 teams per match. Prioritizing minimizing opponents higher than minimizing partners results in a smaller subset of possible match combinations.
I agree, but I prefer a smaller subset of possible matches over playing against the best robot 3 times in an event with 40 teams. there are plenty of weaker robots that can be with or against a certain team and it happens all the time, but there aren’t many robots which you just can’t beat with regular alliance and I think that it is more important to minimize the number of times you will play against the same elite team over and over again. As soon as it happens, even if you are at the top 5 robots in the event, you will probably won’t finish as alliance captain. I’d rather have the same “bad” alliance partner every match but have more matches where my opponent isn’t unbeatable.
Important distinction to be made here: the scheduling algorithm knows nothing about the strength of teams. As such you may luck into or out of a “good” position in the schedule. Furthermore since team performance is not constant throughout a tournament any signal of a “good” schedule is further lost in the noise.
The “all the time” you mention is just observation bias informing some proportionality bias. Which is totally a natural human reaction.
I didn’t mean that the algorithm should consider the strength of the robot.
What I ment that in an avarage event there are much more “weak” robots than “elite” robots, so you might won’t want a weak robot in your alliance every match but it is really hard to avoid and in most of your matches there will be a weaker robot in your alliance, but in most of your matches you shouldn’t get an elite robot to play against you becaue there aren’t so many.
So with the current system with some bad luck you can get the same elite robot to play against you few times which make much more impact than having the same weak robot playing with you few times as you would probably just get another weak robot instead otherwise, and this is why I think the order should be the other way around.
One thing I’m puzzled by is why they’re doing a full generation at each event for what is fundamentally a small set of options (competitions with 24-75 teams). Just generate/analyze them extensively for fairness offline to come up with a set of say 10 schedules for each size (so ~500 total). Then at an event, all the FMS has to do is randomly select one of the 10, and randomly assign teams to slots.
I guess the problem is the solution space is multiplied by the number of matches being run, but there’s only a relatively small set of those too given event durations and needing to do “full rounds”. Maybe have fewer “perfect” schedules per event size to reduce the *10 factor and use it here instead?
With it the other way around you run a higher chance of “super qualification” matches with 3 of the best bots at the tournament being together. This doesn’t help with separability when it comes to ranking.
You are definitely correct in pointing out the distribution of bit performance is skewed to the point where it is probably a poisson distribution rather than a normal one.
Like it or not these are the rules and priorities of first when it comes to schedule. At least there is consistency between the spec and reality now.
I have thought of this as well, best I can come up is first doesn’t want the overhead of some of those ~500 being a “loaded template” in the right circumstances, a template that they approved. But on the flip side they could REALLY check those schedules for internal consistently and actually maximize time between plays instead of these “waves” of 6-10 matches where everyone plays once (not accounting for situations with surrogates).
Some event sizes are far more common, but that could be adjusted for.
Hopefully this post doesn’t become a yearly tradition.