![]() |
"New" 2nd Week Scheduling Algorithm
1 Attachment(s)
As a spin off from this thread I started this thread to specifically discuss the "new" 2nd week scheduling algorithm. The other thread was getting too long and was more of a discussion of the 1st week algorithm.
Last night I made a spreadsheet to analyze the Bayou match schedule (see attached). There were 37 teams. All the "veteran" teams (7+ year old) are color coded blue (12 teams). All the "mid year" teams (2-4 year old teams) are color coded green (12 teams). All the "rookie" (1st year) teams are red (13 teams). The lookup tab has team #, location, year started, team age (1st, 2nd, 3rd, etc. year), and an order # by team number (1-37). The schedule tab is the actual match pairings we played in New Orleans ordered from low team number on the left to highest team number on the right for both the red and blue alliances. The other tabs replace the team # with the year the team was started, team age, or order. As you can see there is nothing random about the pairings at all except whether an alliance was red or blue. It is clear that FIRST divided the teams into 3 sub-groups and intentionally partnered a veteran, a mid-year, and a rookie team for every alliance and then only randomly selected whether they were red or blue. This is not at all what I call random. FIRST has effectively segregated the teams into groups based entirely on team number/age. The only reason to do this is that FIRST is now implying that there is some advantage or disadvantage to being in one group or another. FIRST is without doubt introducing a bias for or against one group or another. Depending on whether you are a pessimist or an optimist you can look at it in one of two ways: 1) A veteran team always HAS to play with a rookie team. 2) A rookie team always GETS to play with a veteran team. But either way, at no time does the algorithm allow two veteran teams on the same alliance during qualification rounds. This effectively stacks the deck in favor of the rookies and against the veterans. My warning to all teams heading into the upcoming regionals: We all pay the same money to play in the same regionals and deserve the same opportunity to play with the same teams. FIRST is not giving you what you paid for. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
Like almost everyone, I agreed that the "perpetual opponent" algorithm was flawed. But as long as you get a variety of opponents so you can show your stuff, the new "random" algorithm seems fair enough to me. In St. Louis, 148 got one of the worst draws I can imagine: every other match had them against either 217 or 45. And how did they respond? Undefeated #1 seed. (Well, except for one surrogate match.) And although 45 seeded near the bottom, they showed the scouts clear and convincing evidence of their capabilities, and were picked 2nd in the draft -- their alliance captain was rewarded for that pick with a regional win. |
Re: "New" 2nd Week Scheduling Algorithm
You are correct, the teams are put into pools A/B/C based on team number - and team number only.
At least in your regional, all the C list were rookies. Look at the GLR pools if you want to see the perverse conditions this creates. There were only 2 rookie teams, and 1114 was put in the C pool because 1114 was 2/3 the way down the list. This meant the two rookies could never be allied with 1114, but might have been opponents (it happened they weren't). 70 was a A, while 494 was a B, and indeed they were allies once and opponents once. Given that pool setup, the assignment of "one from A, one from B, one from C" was randomized more than in the 1st week assignments. 1188 played with 3 teams twice, and played against 2 teams twice. There were no other duplicates. I understand other teams had more duplicates, but no one had a perpetual partner or opponent. In smaller regionals, to keep match timing somewhat separated, the duplicates are higher. |
Re: "New" 2nd Week Scheduling Algorithm
My gut feeling when I was told about this was to like this idea. My feelings have not changed. I like the idea of veterans having to work with rookies. It gives the rookies a chance to learn how the veterans do things. It also prevents an alliance of three veterans going up against an alliance of three rookies.
To say that FIRST isn't giving you what you paid for is not correct. Yes, we all pay the same money to play in the same regionals, but this just means the veterans have to face off and then play with each other during the elimination matches. I'm not out to win a competition, nor am I out to play with only the veteran teams. Instead, I'm out to try and show the students what is possible. And, sometimes, the best lessons are learned from the things that are viewed as disappointments and failures. indieFan |
Re: "New" 2nd Week Scheduling Algorithm
Aren't statistical distributions fun? However the algorithm is written, someone won't be happy, and scouting is still very important.
|
Re: "New" 2nd Week Scheduling Algorithm
My thoughts about the alliance picking algorithm... IT"S NEVER GOING TO BE PERFECT! There simply aren't enough matches in a regional for one team to play every team they want. A basically what people are saying are they don't want to play with rookie teams and only want to play with veteran teams. Well not everyone is going to get what they want. Is it fair? No. But it's a randomizer for a reason. I think one reason it is a randomizer is so there isn't an obvious bias. Of course your going to get the near geniuses that will find some bias in there but why anazlyze the randomizer software. FIRST worked on fixing it from week 1. They did, what more do you want?
|
Re: "New" 2nd Week Scheduling Algorithm
Quote:
Even with the "pools" being controversial, the greatest extent of the problem has been fixed, and as it has already been shown, some people support the formation of pools. Perhaps an ideal would be a "weighted schedule", where the pools are not 100%, but teams are more likely to be playing with teams of different ages (although not guaranteed to be every single match). Or maybe having just 2 "pools", and each alliance having a member from each then one completely random team. |
Re: "New" 2nd Week Scheduling Algorithm
My problem is not the three groups they put us in. It is the fact that we had the same team on our alliance the last several matches and they had terminal problem that they and a lot of the other teams could not help them fix in time. So we went 3 vs 2. Just change the partners up for the different matches. This also helped teams that were with a great bot 5 or 6 matches. If you looked at the bots and the rankings you could see this!!!!!!!!!!!!!!!!! Not what we paid for as so many have said before!!!!!!!!!!!!!!!
|
Re: "New" 2nd Week Scheduling Algorithm
Exactly what did you pay for? I keep hearing "not what we payed for," but what exactly is it that you didn't get that you payed for. You got a kit, you got a regional that there is no way you can admit wasn't fun, and you got an experience you'll remember forever. Seriously what more could you ask. If you think you got ripped off because you didnt get chosen for an alliance for the finals, i'm sorry to say and I don't mean to sound rude but a huge portion of that has absolutely nothing to do with your record! Good record or bad record it all comes down to how you perfrom as a robot.
|
Re: "New" 2nd Week Scheduling Algorithm
Quote:
The "so what" is that by introducing team number as a factor in scheduling matches FIRST is intentionally attempting to bias the outcome of the qualifying matches and introducing a factor into the final seeding that is in no way related to how well a team works or how well designed their robot is. Perhaps this would be acceptable if it were stated in advance in the tournament rules that "your team number will be a factor in determining your opponents and alliance partners", or if a win-loss record of qualifying matches was not kept. (Maybe we should just vote for the top eight teams?) By reducing the randomness of the qualifying match draws, FIRST is not only ending up with a qualification process that less adequately represents the relative abilities of the teams in the overall standings, but also allows teams to question whether they are getting a "fair chance" to win matches. To those who say that a biased match-scheduling system "doesn't matter", then if it doesn't matter why not just get rid of it and go back to random draws? The fact is... it does matter, and it hasn't been "beat to death", because the biased scheduling system is not dead yet. Jason P.S. Although I argue this point strenuously, I'm not quite prepared to start marching to New Hampshire with an "Against Team-Number Oppression" banner or suggesting that FIRST isn't making an honest effort to do a good job... this is one of life's smaller annoyances, after all, but when a team does well... or poorly... in qualifying matches, those results should not be prejudiced by what date they first registered. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
|
Re: "New" 2nd Week Scheduling Algorithm
just in relitive terms, the amount of matches has decreased significantly in the past four years.
look at florida fro example Year one 104 Matches Year two 86 Matches Year three 77 Matches This year 68 Matches Thank god super-regionals are starting to pop up, like GTR adn UCF next year. this way, there are more matches, and i doubt UCF will fill up to 80 teams, because GTR only had 71 out of 80 lasy tear. With two fields, you can divide the teams in half, so it would end up being like 35 teams per field. but the only drawback is the 12 alliances instead of 16. Were using human nature here, we're using something that will NEVER BE PERFECT |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
My gripe with the first week schedule was not playing with rookies. Rather it was playing the same teams 5 times over. We were all rookies once. Some of us still are..... WC :cool: |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
2004- 7 (VCU), 7 (Chesapeake) 2005- 8 (Chesapeake) 7 (Championship) 2006- 9 (VCU, 1 as "surrogate"), 10 (Peachtree-smaller regional) 2007- 8 (VCU) |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
Spoiler for Billfred's Pseudo-scientific analysis of the contenders of Q2:
|
Re: "New" 2nd Week Scheduling Algorithm
Quote:
Quote:
Since our team's rookie year in 2004 we have seen a significant decline in the number of matches we play per regional. I don't have any hard numbers for each event, but our rookie year we got between 10 and 12 matches at both the Pacific NW Regional and the Sacremento Regional. This year, despite being 3v3 now over the 2v2 the first year, we only got 8 matches - largely due to the regionals growing. Portland has gone from roughly 36 teams in 2004 to 54 teams this year. - Jeff |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
IMO the object of qualifying matches is to expose every team's strengths and weaknesses so that scouts get the clearest possible picture of the pool of talent from which elimination partners can be drafted. (Clearly this object was poorly served by the "pertpetual opponent" algorithm because that scheme failed to put teams on the field with the largest number of different alliance partners and against the largest number of different opponents.) Given that during 1-1/2 days of qualifying it is not possible to ally every robot with every other robot, nor oppose every robot with every other robot, would any good end be served by making qualifying alliances of low numbered teams? Students of statistics will appreciate that "randomness" in assigning qualifying match alliance partners and opponents is not a realistic goal. There is simply not enough time available. The best we can do is agree on a set of realistic criteria for qualifying match assignments and devise an algorithm that comes as close as possible to satisfying those criteria. Tom Saxton has taken a crack at this; I think his effort is worth detailed study by FIRST engineering staff and such study may move this discussion in the right direction. |
Re: "New" 2nd Week Scheduling Algorithm
When more then a few teams show up in the top 8 who can not score points I think something is wrong. I think the new matching algorithm makes the top 8 too much about luck rather than skill.
I think after qualification matches, the best teams should have more or less the best records. I think it will be discouraging when teams have excellent machines that don't make the top 8, or get picked by a rookie that can't score who was paired with the same dominating veteran the whole competition. Good design, build, strategy, and gameplay are not being adaquately rewarded. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
|
Re: "New" 2nd Week Scheduling Algorithm
I don't care what teams we get paired with, what number they are, what color their eyes are. I want, as accurately as the tournament's dynamics allow, seeding matches to represent the actual abilities of the teams who play them.
An example. Looking through many regional standings and videos, and our own second week experience, I have seen far to many box bots in the top eight than either the game's dynamics or historical precedent allow for. There are always robots who get carried into the top eight, but they used to be relatively sparse. Teams might see one or two teams float up throughout a couple regionals. This year is not the same. I have not analyzed the algorithm extensively, I have not interpreted the match results to find out why a disproportionate amount of non-scoring robots are making it into the top eight. I do however feel that there is a problem that needs to be fixed- and I do not necessarily fix the blame on the new algorithm. There is an issue that is allowing teams who are not leading their alliances to seed extremely high with regularity, and I'm not at all sure why. But regardless of why it's happening, in my opinion, many people of all team numbers are getting the short end of the stick. Teams that are not especially skilled in game play or strategy, and their alliances as explained in my linked post, are at a direct disadvantage going into finals, and it's not beneficial to anyone caught up in it. Teams aren't losing because of bad planning, design, or strategy, but because of good luck. The situation just doesn't seem to sit well in my mind, but it doesn't seem impossible to remedy either. |
Re: "New" 2nd Week Scheduling Algorithm
There are essentially three distinct schools of though I have noticed in this and the other threads dealing with this subject. Each of these has a different idea of the objective of the qualification matches:
*Seeding: that the qualification matches are for seeding teams based on ability, with the best teams emerging as the top seeds *Performance: that the qualification matches are for determining which bots are the best by pitting them against each other and allowing them to showcase their abilities. The W/L/T records are not important, and it's the responsibility of scouts to make the determination of who is the best. *Competition: that the qualification matches are to create dynamic and competitive matches with evenly matched alliances in every match to produce the most entertaining product possible. Unfortunately it is probably impossible to create an algorithm and schedule that fulfills each of these criteria fully, and someone will always be unhappy with the results. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
1) I don’t think that FIRST segregating team by purposely pooling low, medium and high teams is a good thing. By FIRST segregating teams into “pools” they are inherently saying that one group is different than the other. Whether they are “better” or “worse” or otherwise “special” I can only speculate but as a society I thought we learned somewhere back in the 60’s that segregating groups of people is never a good idea. 2) In a regional with a lot rookie teams (more than 1/3 of the teams in this case), many of the rookies don’t yet understand the importance of good scouting and tend to pick the next highest ranked team whether or not that team will be the best alliance partner for them or not. 3) With the current non-random scheduling algorithm rookies are artificially pushed up the rankings. 4) My contention is that is doesn’t make nearly as much of a difference to low number teams with a good arm design as it does to low number teams with a good ramp/lift design. Dave told everyone at Bayou that the GDC wants teams to learn how to work together. They intentionally made the lift part of the game this year to challenge us to think about all the different types of robots that we will compete with and against. They intentionally wanted us to “get in the heads” of the other teams that we have never met. We actually did this. We sat down and thought about how we could make our robot as compatible with as many different unknown robot designs as possible. Then we proceeded to do the best we could to accommodate these designs. Our robot has and effective arm to place ringers and dual lifts that are spring loaded and are actuated by our alliance partner going up the ramp and into position and then are released by the alliance partners robot. Thus we are using part of the energy in our alliance partner to make the actual lift. While it was not compatible with all robot drivetrains we looked at the list of teams that would be competing at our regional and assumed that we would be randomly partnered with veteran teams at least as often as rookie teams. This was a false assumption. Had the rules given to us at kickoff stated “veteran teams will always be paired with rookie teams” I would have read between the lines that the chances of all the rookie teams having solid reliable drivetrains capable of climbing any sort of incline is much lower than veteran teams with established drivetrains. Also, I would have figured in the likelihood that the rookie drivers (with or without as solid, reliable drivetrain) probably didn't have as much driver practice as the veteran drivers who more than likely had a robot from a previous season to train their drivers on. Factoring these in we would have determined that ramps would not be as effective with rookie partners as they would be with veteran partners thus we would have focused more of our resources on a more effective arm and less on our ramps. As it stands we abandoned our usual strategy of do one thing and do it well and tried to do both this year. Unfortunately, this was not the year to change our usual strategy. That was our fault, however, if we had known then what we know now about the scheduling algorithm we would not have made that decision. I guess what I am saying is if you are an arm bot and you show up to a match with out any alliance partners or neither of your alliance partners can even move then at least you are capable of scoring ringer points on your own. You may or may not win but at least you can play the game. If you are in the same situation but are a ramp/lift bot at best you can play defense but you can’t actually score any points unless you have alliance partners. FIRST wants us to play the partner game but then they fix the matches so the chances are much higher that we won’t have any partners to play with. Quote:
Quote:
Quote:
|
Re: "New" 2nd Week Scheduling Algorithm
Quote:
To add some weigh to the "seedings should reflect a teams ability", FIRST themselves has an award specifically to reward the rookie with the highest seed. I doubt they would give such an award if they didn't put faith in the fact that rookies would be awarded it for an excellent robot design, not simply serendipitous matches. |
Re: "New" 2nd Week Scheduling Algorithm
Sean,
I think you missed a 4th school - "Variety". Variety - the desire to mix things up and not repeatedly play with or against the same partner or opponent. The other 3 you mentioned were spot on though. The beauty is that there are probably many solutions - if there was only that one criteria. Mike |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
I'm going to use this as an example at Detroit when I'm lead queuer, to emphasize the importance of getting at least your human player to the field. |
Re: "New" 2nd Week Scheduling Algorithm
Personally I don't think that team pairings should be made with a particular algorithm - pure randomization works for me.
With regards to teams being categorized into different groups based on number - that's more than a little unfair to all teams. I would have loved to have the (purely random) opportunity to play against different teams, regardless of age or status. To stratify teams within a competition is outrageous. Teams did not get what they paid for equally - a spot in a regional should be just that - a spot in a randomized list, the same as any other team. To be separated and segregated really is against the spirit of how the competitions have been arranged in the past - or at least, how I have seen them - a part of the competition is seeing how you due in random pairings against randomly paired opponents. Putting an even mix of differently aged teams on the field accomplishes nothing. Rookie teams, as well as veteran teams, should be viewed as equal when on the field and when determining when and with whom they play. |
Re: "New" 2nd Week Scheduling Algorithm
1 Attachment(s)
I'll briefly state my support for the non-pooled match scheduling system employed in previous years. Sometimes we ended up going against a powerhouse alliance, sometimes we were part of one, but I at least had confidence that overall the alliances would end up balanced. After all, we were being treated the same as everyone else. Now we aren't -- we're being treated the same as teams with the same team number, which is quite a different thing.
In fact, had we (Team 1346) attended the Wisconsin Regional, we would have been grouped in with the "veteran" pool. If we had attended Great Lakes we would have been in the "rookie" pool. At most regionals we would have been pool "B", smack in the middle. Looking at the overall results of the five second weekend regionals (I've left off Brazil, not because I don't like Brazil, but because it is listed as a "Pilot" regional) and sorting the teams into pools based on team numbers and looking at the top eight qualifiers, then comparing the numbers to similar regionals from last year you get: Top Eight Qualifying Spots Sorted by "Pool" for Second Week Regionals Pool.............................................. .......2007............2006 Veterans (lowest 1/3 of team numbers).......14................21 Mid Year Teams......................................16..... ...........10 Rookies (highest 1/3 of team numbers).........10.................9 This would indicate to me that (as one would suspect) the pooling scheme has resulted in a more even distribution of top eight finishes amongst the three pools. Is this good? Is this bad? Are the numbers even significant? All I know, is that if they are signficant, then veteran teams are being denied positions as alliance captains because of their team numbers. If the numbers aren't significant then the pooling system wasn't needed in the first place, was it? Jason P.S. Apologies for not formatting the excel file a little neater, it was mostly for my work and the data is summarized here. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
Allies 597, 1722 599, 1669 702, 1759 399, 2174 597, 1836 702, 2174 254, 1702 606, 1759 691, 1855 Opponents 599, 1138, 1759 687, 812, 1644 4, 1722, 1197 330, 1438, 2272 294, 1266, 2178 599, 1266, 1669 399, 980, 2029 330, 867, 1669 330, 968, 2178 We had nine matches and interacted with a total of 31 teams, many of them multiple times. Getting paired with a few teams multiple times, sure that happens. But we were allied with four teams twice, played against four teams twice, and played against 330 three times. I still had fun, but it seemed repetitive after a while. Quote:
|
Re: "New" 2nd Week Scheduling Algorithm
So, did anyone notice if this algorithm has remained the same since week 2 or if it's been changed through the other events?
The scheduling at Silicon Valley left a lot to be desired during week 3. We played a more randomized set of opponents, but not notably so. What's worse than playing against the same opponent four times, however, was the scheduling of our matches themselves. We had two sets of nearly back-to-back matches separated by a gap of 3.5 hours. Has anyone else had a longer gap between matches? :) The closest we've had two consecutive matches was three apart. |
Re: "New" 2nd Week Scheduling Algorithm
Here's our matches from Boston, week 4:
2099, 1757, 173 vs 2103, 1761, 246 1973, 1568, 173 vs 1922, 1403, 155 2103, 1686, 173 vs 2125, 1761, 69 1975, 1768, 173 vs 2079, 1761, 529 1973, 1626, 173 vs 1975, 1909, 97 2099, 1757, 173 vs 2262, 1831, 125 2110, 1761, 173 vs 1965, 1768, 126 1965, 1350, 173 vs 2124, 1685, 97 2043, 1685, 173 vs 2103, 1735, 155 You can draw your own conclusions. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
|
Re: "New" 2nd Week Scheduling Algorithm
I've posted a white paper with an analysis of both scheduling algorithms, proposed metrics for measuring the desirable qualities of a match schedule, explained a superior solution to the program, and posted a program that generates much better schedules.
|
Re: "New" 2nd Week Scheduling Algorithm
It looks like they tuned the system up for Week 5. They still have the teams divided into 3 parts, but you don't play with/against everyone a whole lot often. In fact, if the regional is big enough, I think it tries to pair you up only once with/against teams. Anyone have a match list to confirm this?
|
Re: "New" 2nd Week Scheduling Algorithm
Well, the analysis from my stat class is not done yet (I decided to wait until data from all of the regional matches is available) but a preliminary look seems to confirm that FIRST did succeed in "levelling the playing field" in terms of rookies and veteran teams. Using this algorithm there is basically no correlation between number of years in FIRST and record after qualifying. It is a completely different question as to whether this is "better" or "worse" than in the past.
The data from the past two years shows that veteran teams do have a significant advantage over non-veterans, but that advantage largely disappears once you get to about 3-4 years in FIRST. This is about what you would expect because by that time you are turning over team members. The really big advantage goes for teams competing in multiple regional competitions. The more competitions, the bigger the advantage. (But the biggest jump comes from the second regional.) I am going to try to compile some data from the past couple of years and find out if the "veteran" correlation gets stronger by comparing the total number of competitions attended to qualifying record. My suspicion is that it will. A couple of other things jump out at first glance. The algorithm seems to work quite differently for larger regionals (over 50 teams) and smaller regionals. At a small regional there are many more repeat partners and opponents. The teams in the top 8 tend to have played multiple matches with other teams in the top 8. It would have been very interesting to keep track of number of points scored (or at least ringers placed, ramps successfully deployed or climbed) by each team. I know we kept data at Pittsburgh and the correlation between points scored by a team and its record in qualifying is a lot weaker than one would expect. Tom Saxton's algorithm looks good to me. I am thinking that something like Arrow's Impossibility Theorem will be in play here in that there can't be a system that does the best under all criteria. So there is a need to look at the problems perceived in the current system and try to find a way to change them without causing other problems. I am also wondering if it might not be advisable for FIRST to incorporate how many total competitions a team has attended rather than just number of years in FIRST and perhaps how many competitions a team has/will attend that year. The real question becomes two part: Should FIRST change the algorithm and will FIRST change the algorithm? Neither answer is really obvious to me. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
We had several other repeating partners/opponents. Draw your own conclusions. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
Played With: Three times each: 665 and 1251 Two times each: 34, 343, 1102, 1293 Once each: 337, 538, 587, 900, 1249, 1390 Played Against: Three times each: 1051, 1522, 2237 Two times each: 337, 538, 1436 Once each: 135, 281, 342, 386, 804, 832, 845, 900, 1026, 1225, 1251, 1287, 1390, 1553, 2187 Take from that what you will. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
|
Re: "New" 2nd Week Scheduling Algorithm
OK, I guess I was wrong. I was working at the LI regional and I "heard" that they improved the match pairing system. The only improvement was the digital clock in the scoring software. Sorry for the confusion. |
Re: "New" 2nd Week Scheduling Algorithm
Quote:
|
Re: "New" 2nd Week Scheduling Algorithm
Code:
Team 494 Partners Opponents71-1 with 3 against 703-2 against (Both with Hammond) 1918-2 with 288-2 with 27-1 with 1 against 2188- 1 with 1 against 1783-1 with 1 against 2054-2 against The system really bugs me. Take 1718 for example, Yes they have on of the best machines this year, but with the scheduling system they also have easier matches. We love 71 alot, but playing them three times is ludicrous. Admit it FIRST, the system is partially broken, go back to last years system or risk having several teams really upset with you |
| All times are GMT -5. The time now is 08:38. |
Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi