Thanks for the world class insights… SAC how much further is that? LOL, I literally almost fell asleep on the way home…had to pull over to take a 20 minute “not crash car” nap
As someone who was around the last time the match scheduling algorithm attempted to account for strength of schedule, NO. Just NO. 2007 was AWFUL. Yes, these caps are required.
The gripe is about favorable/unfavorable schedules. If you try to balance out the competitive levels of teams, you’re inherently creating favorable/unfavorable schedules. This is exactly what happened in 2007. Older teams had unfavorable schedules, and younger teams had favorable ones. By trying to solve the problem, you’re essentially enshrining the problem.
And as a technical point, *only *the NFL accounts for previous seasons’ results in determining the schedule. And that’s a relatively minor factor (it influences only 2 games of the 16 on each team’s schedule). Conferences/divisions (which are geographically based) are the driving factor in the other 14 games of the NFL schedule, and the lone factor in the NBA, MLB, and NHL schedules.
NFL teams play each other once unless in divisions (or playoffs)
I do think there is a solution…
NO duplicates
Also too many pairings/facings of “known powerhouse teams” should be schedule avoided until eliminations
Powerhouses don’t really need that much help to rank high nor is it good for younger teams to face “near certain doom” twice in a competition hence my no duplicate suggestion
Powerhouse teams will not be left on the bench either they will nearly always be selected regardless. Most other teams not as much.
Correct. That doesn’t change my point. Only 2 of the 16 games on an NFL teams’ schedule are based on the previous seasons’ results. The other 14 games are based on which division a team plays in.
But two games are…perhaps similar to the few match-ups I see as inconsistent?
I wasn’t trying to use the NFL as a model to base our scheduling algorithm off of. I was simply pointing out that the assertion that “all of the professional sports leagues already do this” is factually incorrect. Three of the four major North American professional leagues do not factor in previous performance at all, and the only that does only has two games based off of it.
We could always follow the European soccer leagues’ example and relegate the bottom performers.
/s
Another important thing to mention here is that this year’s volunteer training very heavily emphasized only running the MatchMaker algorithm once and only once. And if you ran it a second time, you had to explain your reasoning on Skype. The reason for this policy is to avoid the exact kinds of bias we’re discussing - because randomness is the fairest decider of them all.
The programmer of the 2007 match algorithm believed the same thing. Whether or not it is correct doesn’t matter much to the results of implementing that belief: with fewer than about 60 teams at an event, the number of times a given team plays with or against a specific other team skyrockets. This is exactly what you’re trying to avoid, isn’t it?
Unless you have an extraordinarily large pool of teams to draw from, “all of the other constraints” conspire to cluster teams into essentially the same groupings over and over again. What problem were you trying to solve again?
I wonder which setting they are using?
Quality The algorithm allows the user to specify a desired quality. This simply determines the number of schedules that are generated and evaluated in the simulated annealing algorithm, as specified in the list below.
Fair: 100,000
Good: 750,000
Best: 5,000,000
If they are running only one perhaps they should choose “Best”
I’m reasonably certain they already run it on “best”, having played around with the standalone program and been an official scorekeeper in the past. “Best” mode takes several minutes to run, while “Fair” only takes a few seconds. The schedule generation step in FMS also takes several minutes.
First I disagree with the assertion that randomness is the fairest decider, especially when we never get to a full probability distribution which would allow schedules to balance out over a one or even several regionals. Second, these algorithms are in fact not completely random. They have inherent patterns that are difficult to erase.
I believe it would be simple add the bin constraint–I doubt that it would have to be run more than once. The only added step which is quite easy is to assign the teams to the separate bins.
As for leaving scheduling to the judgement calls of the officials, I think that’s fraught with danger. For example, how should we have treated 5136 at CVR? They had made it to Einstein in 2014 yet they are only a 3rd year team. I think a transparent algorithm solves the problem the best.
No, we’re not trying to avoid playing with the same teams repeatedly. We’re trying to avoid matches in which one alliance is over strong vs the other simply due to the nature of the composition. We should not have been paired with 254 or 1671 AT ALL at CVR, yet we played with both of them. Excluding such pairings is more imperative than playing with as many teams as possible.
Another option to use a sliding scale of point ratings for each team, and ensure that each alliance has a total that exceeds a minimum value. That would allow for more flexibility in alliance composition. Again an imperative to ensure that every alliance has a minimum level of competence. The flip side of having overly strong alliances is less important if each alliance surpasses a minimum.
I’m interested in what the problem was with the 2007 schedule? What criteria were they using?
BTW, I think it’s interesting to see that it’s only the NFL that uses strength of schedule. The other leagues have so many games (80+) that each team is able to play each other multiple times. (But I will note that the NBA changed its playoff qualification instead as a means of correcting a regional misbalance in schedule strength.) The NFL season is more akin to FRC with relatively few matches compared to the number of competitors.
Thanks Citrus Dad for voicing from the top tier what I witnessed in my two years of analyzing. I think FRC can do better and that would possibly allow a new generation to break through to the necessary experience of a true national World Class competition. I have a feeling that is why its so hard for young teams to break through as captains…usually they get in as second ,third or backup because if multiple powerhouse qual pairings get skewed the randomness skewed its awfully hard to remain #1 in day 2 to have pick of the litter we would have picked 1678 not to discredit 254 or 973 or 1323 and many other worthy teams…just 1678 dominated by never losing and proved they could get 1 ball auto and scale. Plus pretty much un-blockable with arguably the fastest cycle times at CVR… very close was 254 but no scale so (-2 HG equivalent)
I was happy to see in another regional a 5000# captain did break through not sure if that was powerhouse laced or not though…awesome to see.
All I know is going to worlds our first year is the main reason WHY we are so good now as a young third year team. It made an impression on us we will not forget we did not like the taste of finishing 90th or so there. The world championship experience is something every team should have as I believe it helps young teams to become that next generation. You see what it takes and witness first hand the best. I’m just glad CA offers every year a world class regional experience within driving distance.
I think this is a noble goal, but the devil will be in the implementation details. Done poorly, we’ll be even worse off then we are today. People old enough will remember the 2007 “Algorithm of Doom”, which selected teams from three buckets like you’re proposing (although those buckets were populated by team age).
The biggest issue will be correctly populating the buckets - this thread discusses many of the same ideas. You’ll also have to account for buckets that aren’t of equal size (by whatever metric you choose), especially when the number of teams attending is not a multiple of 3. Would this cause more surrogates to be used, or something else?
Are you proposing using previous year’s data, because that runs into trouble with fast team turnover and the potential for powerhouse teams to have down years, only to be paired with other powerhouses more often the following year. Plus, what do you do with rookies? They have no data to rely on yet, and there are always exceptional rookie teams, which would cause the same kind of inconsistency.
If you use current season data, what would you do for early events? You wouldn’t have any data available for the early weeks, so you’d have to resort to a randomness-based algorithm again. This approach would only work for District Champs/World Champs, where all teams are already guaranteed to have played already.
Plus, Ed Law made a great point in the other thread I linked:
So overall, I’ll concede that random selection isn’t perfect, but I’m not totally convinced a team strength based algorithm is better.
I think you simply never allow repeats and treat world champions that got there “as captains or strong world class second bots” or those that won say 4 regionals in past 6 years slightly different from the remaining pool. I think you let teams like 5136 slide and remain in the greater non-powerhouse pool.
Those “powerhouse” teams should not be paired as a “dream alliance” in Quals at all and possibly face each other to provide more entertainment than just running up the score against three way younger teams scratching and clawing for the worlds only to likely get slaughtered (and drop 2 RP each )
Lets say an event such as CVR there are SIX teams that meet that criteria (with four wildcards not a stretch) so you put SIX fake teams in the algorithm as placeholders…you then analyze each fake entry if it look fair balance with other fake entries you then replace the real powerhouse teams randomly in…for the others you check other fake team place holders if you see a discrepancy shift a pool team in and make their games placeholders…rinse repeat. Until all six teams are more evenly sorted. Then randomly assign powerhouses to each fake entry.
-OR_
Re-run the algorithm until the fake placeholders look balanced is another way. Then enter the real teams again randomly for each fake placeholder.
Look at CVR… #1 really took it to #5 in Finals …no surprise to anyone there and #5 was very strong offensively but younger and not as many creds. I’m fairly certain we would have been a similar story as #2…that is how good #1 was.
It really can be like NE always getting to play Cleveland due to lack of young teams at #1 close but not quite at least in stacked regionals. In SD yes a rookie was #1 and got beat in QF SD only had 1 super class bot 399. Our alliance #8 took them out in 2 games because we were ranked low #19 due to missing RP’s for breaches not credited. (Should have been rank 10 or 11 and captain) fine with us worked out not so fine for rookie #1 alliance (although they rightfully earned RAS)
Would treating powerhouses slightly different change the end result?
Who knows but it may have. Same with valid RP sorts.
FRC can do better IMO…lots of brains there figure it out.
You are never going to get truly random but you can try to get more balanced at the top tier dream team pairings and facing repeat low win probability having to face the same powerhouse team that beat you more than once especially when paired with another.
I’ve thought about many of these issues, the first is that it has to rely on past year data, not current so Ed’s concern is not relevant. (The one exception might be for the Champs, but then teams can’t get there by sandbagging.)
I want to take this conversation off line for those interested in addressing this. I have an idea for changing the scheduling process that would both incorporate strength factors AND make it easier. I’m particularly interested in what happened in 2007 as that was before my involvement. Please message me directly if you want to discuss this.
I wouldn’t say that that’s true at all. Just off the top of my head, 1266, 2486, 2102, 3255, and, though I may be biased, 2485 have all been to champs on multiple occasions.
Broader, though, it’s not FIRST’s job to make sure everyone gets to go to champs. The tech world that FRC prepares us for is cutthroat. Not everyone gets equal opportunity.
Those are REALLY good perennial teams I agree but I put 399 above then based on their long consistent history. Starting to approach other CA giants.
Congrats to 2102, 3021 and 2486 for winning SD I’ll be watching you in worlds 2485 made it too with a well deserved award. 5805 was super sleek awesome RAS. 2687 got the WC…awesome job San Diego teams. 3255 was very solid until they broke down at the wrong time they should be a favorite in their next event
Actually I’m glad Champs are hard makes you hungry every year…