Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   Regional Competitions (http://www.chiefdelphi.com/forums/forumdisplay.php?f=10)
-   -   "New" 2nd Week Scheduling Algorithm (http://www.chiefdelphi.com/forums/showthread.php?t=55622)

ChuckDickerson 12-03-2007 15:52

"New" 2nd Week Scheduling Algorithm
 
1 Attachment(s)
As a spin off from this thread I started this thread to specifically discuss the "new" 2nd week scheduling algorithm. The other thread was getting too long and was more of a discussion of the 1st week algorithm.

Last night I made a spreadsheet to analyze the Bayou match schedule (see attached). There were 37 teams. All the "veteran" teams (7+ year old) are color coded blue (12 teams). All the "mid year" teams (2-4 year old teams) are color coded green (12 teams). All the "rookie" (1st year) teams are red (13 teams). The lookup tab has team #, location, year started, team age (1st, 2nd, 3rd, etc. year), and an order # by team number (1-37). The schedule tab is the actual match pairings we played in New Orleans ordered from low team number on the left to highest team number on the right for both the red and blue alliances. The other tabs replace the team # with the year the team was started, team age, or order. As you can see there is nothing random about the pairings at all except whether an alliance was red or blue. It is clear that FIRST divided the teams into 3 sub-groups and intentionally partnered a veteran, a mid-year, and a rookie team for every alliance and then only randomly selected whether they were red or blue. This is not at all what I call random. FIRST has effectively segregated the teams into groups based entirely on team number/age. The only reason to do this is that FIRST is now implying that there is some advantage or disadvantage to being in one group or another. FIRST is without doubt introducing a bias for or against one group or another.

Depending on whether you are a pessimist or an optimist you can look at it in one of two ways:

1) A veteran team always HAS to play with a rookie team.

2) A rookie team always GETS to play with a veteran team.

But either way, at no time does the algorithm allow two veteran teams on the same alliance during qualification rounds. This effectively stacks the deck in favor of the rookies and against the veterans.

My warning to all teams heading into the upcoming regionals: We all pay the same money to play in the same regionals and deserve the same opportunity to play with the same teams. FIRST is not giving you what you paid for.

Richard Wallace 12-03-2007 16:05

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by DeepWater (Post 595900)
... Depending on whether you are a pessimist or an optimist you can look at it in one of two ways:

1) A veteran team always HAS to play with a rookie team.

2) A rookie team always GETS to play with a veteran team.

But either way, at no time does the algorithm allow two veteran teams on the same alliance during qualification rounds. This effectively stacks the deck in favor of the rookies and against the veterans. ...

Hasn't this been beat to death yet? OK, so low numbered teams don't get to play with each other in qualifying. So what?

Like almost everyone, I agreed that the "perpetual opponent" algorithm was flawed. But as long as you get a variety of opponents so you can show your stuff, the new "random" algorithm seems fair enough to me.

In St. Louis, 148 got one of the worst draws I can imagine: every other match had them against either 217 or 45. And how did they respond? Undefeated #1 seed. (Well, except for one surrogate match.)

And although 45 seeded near the bottom, they showed the scouts clear and convincing evidence of their capabilities, and were picked 2nd in the draft -- their alliance captain was rewarded for that pick with a regional win.

GaryVoshol 12-03-2007 16:08

Re: "New" 2nd Week Scheduling Algorithm
 
You are correct, the teams are put into pools A/B/C based on team number - and team number only.

At least in your regional, all the C list were rookies. Look at the GLR pools if you want to see the perverse conditions this creates. There were only 2 rookie teams, and 1114 was put in the C pool because 1114 was 2/3 the way down the list. This meant the two rookies could never be allied with 1114, but might have been opponents (it happened they weren't). 70 was a A, while 494 was a B, and indeed they were allies once and opponents once.

Given that pool setup, the assignment of "one from A, one from B, one from C" was randomized more than in the 1st week assignments. 1188 played with 3 teams twice, and played against 2 teams twice. There were no other duplicates. I understand other teams had more duplicates, but no one had a perpetual partner or opponent. In smaller regionals, to keep match timing somewhat separated, the duplicates are higher.

indieFan 12-03-2007 16:12

Re: "New" 2nd Week Scheduling Algorithm
 
My gut feeling when I was told about this was to like this idea. My feelings have not changed. I like the idea of veterans having to work with rookies. It gives the rookies a chance to learn how the veterans do things. It also prevents an alliance of three veterans going up against an alliance of three rookies.

To say that FIRST isn't giving you what you paid for is not correct. Yes, we all pay the same money to play in the same regionals, but this just means the veterans have to face off and then play with each other during the elimination matches.

I'm not out to win a competition, nor am I out to play with only the veteran teams. Instead, I'm out to try and show the students what is possible. And, sometimes, the best lessons are learned from the things that are viewed as disappointments and failures.

indieFan

MrForbes 12-03-2007 16:14

Re: "New" 2nd Week Scheduling Algorithm
 
Aren't statistical distributions fun? However the algorithm is written, someone won't be happy, and scouting is still very important.

burkechrs1 12-03-2007 16:18

Re: "New" 2nd Week Scheduling Algorithm
 
My thoughts about the alliance picking algorithm... IT"S NEVER GOING TO BE PERFECT! There simply aren't enough matches in a regional for one team to play every team they want. A basically what people are saying are they don't want to play with rookie teams and only want to play with veteran teams. Well not everyone is going to get what they want. Is it fair? No. But it's a randomizer for a reason. I think one reason it is a randomizer is so there isn't an obvious bias. Of course your going to get the near geniuses that will find some bias in there but why anazlyze the randomizer software. FIRST worked on fixing it from week 1. They did, what more do you want?

Lil' Lavery 12-03-2007 16:19

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by GaryV1188 (Post 595918)
You are correct, the teams are put into pools A/B/C based on team number - and team number only.

At least in your regional, all the C list were rookies. Look at the GLR pools if you want to see the perverse conditions this creates. There were only 2 rookie teams, and 1114 was put in the C pool because 1114 was 2/3 the way down the list. This meant the two rookies could never be allied with 1114, but might have been opponents (it happened they weren't). 70 was a A, while 494 was a B, and indeed they were allies once and opponents once.

Given that pool setup, the assignment of "one from A, one from B, one from C" was randomized more than in the 1st week assignments. 1188 played with 3 teams twice, and played against 2 teams twice. There were no other duplicates. I understand other teams had more duplicates, but no one had a perpetual partner or opponent. In smaller regionals, to keep match timing somewhat separated, the duplicates are higher.

The A/B/C pools were also created in week 1. 116 faced our "A" opponent (122) every match. In addition, our "B" opponent became our "B" ally in our next match every single time. That may have resulted from VCU having 66 teams (and therefor divisible by 6), which also resulted in 116 playing exactly every 11th match (1, 12, 23, 34, 45, 56, 67, 78).
Even with the "pools" being controversial, the greatest extent of the problem has been fixed, and as it has already been shown, some people support the formation of pools. Perhaps an ideal would be a "weighted schedule", where the pools are not 100%, but teams are more likely to be playing with teams of different ages (although not guaranteed to be every single match). Or maybe having just 2 "pools", and each alliance having a member from each then one completely random team.

JA-Low 12-03-2007 16:20

Re: "New" 2nd Week Scheduling Algorithm
 
My problem is not the three groups they put us in. It is the fact that we had the same team on our alliance the last several matches and they had terminal problem that they and a lot of the other teams could not help them fix in time. So we went 3 vs 2. Just change the partners up for the different matches. This also helped teams that were with a great bot 5 or 6 matches. If you looked at the bots and the rankings you could see this!!!!!!!!!!!!!!!!! Not what we paid for as so many have said before!!!!!!!!!!!!!!!

burkechrs1 12-03-2007 16:24

Re: "New" 2nd Week Scheduling Algorithm
 
Exactly what did you pay for? I keep hearing "not what we payed for," but what exactly is it that you didn't get that you payed for. You got a kit, you got a regional that there is no way you can admit wasn't fun, and you got an experience you'll remember forever. Seriously what more could you ask. If you think you got ripped off because you didnt get chosen for an alliance for the finals, i'm sorry to say and I don't mean to sound rude but a huge portion of that has absolutely nothing to do with your record! Good record or bad record it all comes down to how you perfrom as a robot.

dtengineering 12-03-2007 16:36

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Richard (Post 595915)
Hasn't this been beat to death yet? OK, so low numbered teams don't get to play with each other in qualifying. So what?

I will first of all agree with the rest of your post, in which you point out that excellent teams can still make it to the top of the qualifying rounds, and that it is possible for a low-seeded team to be picked by a top eight-seeded team. I don't think anyone is arguing that, but I feel obliged to address the question above.

The "so what" is that by introducing team number as a factor in scheduling matches FIRST is intentionally attempting to bias the outcome of the qualifying matches and introducing a factor into the final seeding that is in no way related to how well a team works or how well designed their robot is. Perhaps this would be acceptable if it were stated in advance in the tournament rules that "your team number will be a factor in determining your opponents and alliance partners", or if a win-loss record of qualifying matches was not kept. (Maybe we should just vote for the top eight teams?)

By reducing the randomness of the qualifying match draws, FIRST is not only ending up with a qualification process that less adequately represents the relative abilities of the teams in the overall standings, but also allows teams to question whether they are getting a "fair chance" to win matches.

To those who say that a biased match-scheduling system "doesn't matter", then if it doesn't matter why not just get rid of it and go back to random draws?

The fact is... it does matter, and it hasn't been "beat to death", because the biased scheduling system is not dead yet.

Jason

P.S. Although I argue this point strenuously, I'm not quite prepared to start marching to New Hampshire with an "Against Team-Number Oppression" banner or suggesting that FIRST isn't making an honest effort to do a good job... this is one of life's smaller annoyances, after all, but when a team does well... or poorly... in qualifying matches, those results should not be prejudiced by what date they first registered.

Richard Wallace 12-03-2007 16:54

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by dtengineering (Post 595956)
The fact is... it does matter, and it hasn't been "beat to death", because the biased scheduling system is not dead yet.

Not dead yet?

Nawaid Ladak 12-03-2007 17:04

Re: "New" 2nd Week Scheduling Algorithm
 
just in relitive terms, the amount of matches has decreased significantly in the past four years.

look at florida fro example

Year one 104 Matches
Year two 86 Matches
Year three 77 Matches
This year 68 Matches

Thank god super-regionals are starting to pop up, like GTR adn UCF next year. this way, there are more matches, and i doubt UCF will fill up to 80 teams, because GTR only had 71 out of 80 lasy tear.

With two fields, you can divide the teams in half, so it would end up being like 35 teams per field.

but the only drawback is the 12 alliances instead of 16. Were using human nature here, we're using something that will NEVER BE PERFECT

Wayne C. 12-03-2007 17:06

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by DeepWater (Post 595900)
As a spin off from this thread
Depending on whether you are a pessimist or an optimist you can look at it in one of two ways:

1) A veteran team always HAS to play with a rookie team.

2) A rookie team always GETS to play with a veteran team.

Maybe this is the best reason to make sure the rookies are trained and ready for competition. That is the responsibility of the veterans.

My gripe with the first week schedule was not playing with rookies. Rather it was playing the same teams 5 times over. We were all rookies once. Some of us still are.....

WC :cool:

Lil' Lavery 12-03-2007 17:46

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by FreedomForce (Post 595988)

Year one 104 Matches
Year two 86 Matches
Year three 77 Matches
This year 68 Matches

"Year 1" also was a 2v2 game, and would require more matches for each team to participate in the same amount. Matches haven't gone down per team in my experience.
2004- 7 (VCU), 7 (Chesapeake)
2005- 8 (Chesapeake) 7 (Championship)
2006- 9 (VCU, 1 as "surrogate"), 10 (Peachtree-smaller regional)
2007- 8 (VCU)

Billfred 12-03-2007 18:24

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Richard (Post 595915)
Hasn't this been beat to death yet? OK, so low numbered teams don't get to play with each other in qualifying. So what?

So what? Going by team number alone is a lousy way to go about it--the software is making assumptions about their on-field performance that are often erroneous. Consider a qualification match at UCF (Q2, for the record): 61/665/1557 vs. 69/744/1592. On paper, that seems like a pretty balanced match--until you actually consider the robots in question.
Spoiler for Billfred's Pseudo-scientific analysis of the contenders of Q2:
61 and 1557 both finished the event as boxes on wheels (1557 was the whole time; 61 had significant arm problems--like the arm popping off--nearly every match until they took it off), while 69 and 1592 were both pretty good arms with ramps. I don't remember enough of 665 or 744 on the field to make a good call about them, but I know that 665's arm ended up out of commission by Saturday morning because of a burned-out motor.
No good scout would pick an alliance partner for eliminations just because they have a low number, yet the current match pairing algorithm is doing just that. For my money, I'd rather have a mostly-random match pairing algorithm (with the obligatory concessions for some space between rounds) than one that tries to meddle with things. If I wind up with 71, 45, and 111 against my team and two flowerpots, so be it--at least I can write that off to bad luck and have a good laugh about that over dinner.

Jeffrafa 12-03-2007 19:17

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by FreedomForce (Post 595988)
just in relitive terms, the amount of matches has decreased significantly in the past four years.

look at florida fro example

Year one 104 Matches
Year two 86 Matches
Year three 77 Matches
This year 68 Matches

Quote:

Originally Posted by Lil' Lavery (Post 596032)
"Year 1" also was a 2v2 game, and would require more matches for each team to participate in the same amount. Matches haven't gone down per team in my experience.

I think what he's hitting at is not that each team is getting fewer matches than before (although this is the case in some situations), but that each regional is having notably fewer matches than in the past. Sure, 2004 was 2v2, and therefore required significantly more matches to grant teams as much time on the field. But just like 2004, the rounds are only 2 minutes and 15 seconds long. If Florida was able to manage 104 matches of that length in 2004, why are they doing only 65% as many rounds this year? Surely setting up two additional robots per match dosen't make a match cycle take 50% longer.

Since our team's rookie year in 2004 we have seen a significant decline in the number of matches we play per regional. I don't have any hard numbers for each event, but our rookie year we got between 10 and 12 matches at both the Pacific NW Regional and the Sacremento Regional. This year, despite being 3v3 now over the 2v2 the first year, we only got 8 matches - largely due to the regionals growing. Portland has gone from roughly 36 teams in 2004 to 54 teams this year.

- Jeff

Richard Wallace 12-03-2007 19:54

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Billfred (Post 596056)
So what? Going by team number alone is a lousy way to go about it--....

How is it lousy? Doesn't that judgement depend on what criteria you choose to emphasize in assigning qualification match partners and opponents? Why should the criteria for making qualifying matches be anything at all like those used by scouts to select elimination partners? What goal of FIRST would be served by that?

IMO the object of qualifying matches is to expose every team's strengths and weaknesses so that scouts get the clearest possible picture of the pool of talent from which elimination partners can be drafted. (Clearly this object was poorly served by the "pertpetual opponent" algorithm because that scheme failed to put teams on the field with the largest number of different alliance partners and against the largest number of different opponents.) Given that during 1-1/2 days of qualifying it is not possible to ally every robot with every other robot, nor oppose every robot with every other robot, would any good end be served by making qualifying alliances of low numbered teams?

Students of statistics will appreciate that "randomness" in assigning qualifying match alliance partners and opponents is not a realistic goal. There is simply not enough time available. The best we can do is agree on a set of realistic criteria for qualifying match assignments and devise an algorithm that comes as close as possible to satisfying those criteria. Tom Saxton has taken a crack at this; I think his effort is worth detailed study by FIRST engineering staff and such study may move this discussion in the right direction.

Fred Sayre 12-03-2007 19:58

Re: "New" 2nd Week Scheduling Algorithm
 
When more then a few teams show up in the top 8 who can not score points I think something is wrong. I think the new matching algorithm makes the top 8 too much about luck rather than skill.

I think after qualification matches, the best teams should have more or less the best records. I think it will be discouraging when teams have excellent machines that don't make the top 8, or get picked by a rookie that can't score who was paired with the same dominating veteran the whole competition.

Good design, build, strategy, and gameplay are not being adaquately rewarded.

Lil' Lavery 12-03-2007 20:50

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Jeffrafa (Post 596099)
I think what he's hitting at is not that each team is getting fewer matches than before (although this is the case in some situations), but that each regional is having notably fewer matches than in the past. Sure, 2004 was 2v2, and therefore required significantly more matches to grant teams as much time on the field. But just like 2004, the rounds are only 2 minutes and 15 seconds long. If Florida was able to manage 104 matches of that length in 2004, why are they doing only 65% as many rounds this year? Surely setting up two additional robots per match dosen't make a match cycle take 50% longer.

Since our team's rookie year in 2004 we have seen a significant decline in the number of matches we play per regional. I don't have any hard numbers for each event, but our rookie year we got between 10 and 12 matches at both the Pacific NW Regional and the Sacremento Regional. This year, despite being 3v3 now over the 2v2 the first year, we only got 8 matches - largely due to the regionals growing. Portland has gone from roughly 36 teams in 2004 to 54 teams this year.

The reason your team is getting matches is what you said, more teams. With more teams it is inevitable that you will have fewer matches. Every regional has given teams a minimum of 8 matches, and several have given more. If you try and cram more matches in, you give each team less time to make repairs between matches, less time for pre-match strategy, etc. There is no need to play more matches than you can comfortably fit into a schedule. Of all the competitions I listed with 116 earlier, 5 of the 7 had 56-66 teams (the VCUs and Chesapeakes), and each played 7 or 8 matches (9 for 2006 VCU because we played as a "surrogate"). I think that's a decent amount considering the number of teams participating.

Andrew Blair 12-03-2007 21:19

Re: "New" 2nd Week Scheduling Algorithm
 
I don't care what teams we get paired with, what number they are, what color their eyes are. I want, as accurately as the tournament's dynamics allow, seeding matches to represent the actual abilities of the teams who play them.

An example.

Looking through many regional standings and videos, and our own second week experience, I have seen far to many box bots in the top eight than either the game's dynamics or historical precedent allow for. There are always robots who get carried into the top eight, but they used to be relatively sparse. Teams might see one or two teams float up throughout a couple regionals.

This year is not the same. I have not analyzed the algorithm extensively, I have not interpreted the match results to find out why a disproportionate amount of non-scoring robots are making it into the top eight. I do however feel that there is a problem that needs to be fixed- and I do not necessarily fix the blame on the new algorithm. There is an issue that is allowing teams who are not leading their alliances to seed extremely high with regularity, and I'm not at all sure why. But regardless of why it's happening, in my opinion, many people of all team numbers are getting the short end of the stick.

Teams that are not especially skilled in game play or strategy, and their alliances as explained in my linked post, are at a direct disadvantage going into finals, and it's not beneficial to anyone caught up in it. Teams aren't losing because of bad planning, design, or strategy, but because of good luck. The situation just doesn't seem to sit well in my mind, but it doesn't seem impossible to remedy either.

Lil' Lavery 12-03-2007 21:32

Re: "New" 2nd Week Scheduling Algorithm
 
There are essentially three distinct schools of though I have noticed in this and the other threads dealing with this subject. Each of these has a different idea of the objective of the qualification matches:
*Seeding: that the qualification matches are for seeding teams based on ability, with the best teams emerging as the top seeds
*Performance: that the qualification matches are for determining which bots are the best by pitting them against each other and allowing them to showcase their abilities. The W/L/T records are not important, and it's the responsibility of scouts to make the determination of who is the best.
*Competition: that the qualification matches are to create dynamic and competitive matches with evenly matched alliances in every match to produce the most entertaining product possible.

Unfortunately it is probably impossible to create an algorithm and schedule that fulfills each of these criteria fully, and someone will always be unhappy with the results.

ChuckDickerson 12-03-2007 21:40

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Richard (Post 595915)
So what?

A few thoughts off the top of my head:

1) I don’t think that FIRST segregating team by purposely pooling low, medium and high teams is a good thing. By FIRST segregating teams into “pools” they are inherently saying that one group is different than the other. Whether they are “better” or “worse” or otherwise “special” I can only speculate but as a society I thought we learned somewhere back in the 60’s that segregating groups of people is never a good idea.

2) In a regional with a lot rookie teams (more than 1/3 of the teams in this case), many of the rookies don’t yet understand the importance of good scouting and tend to pick the next highest ranked team whether or not that team will be the best alliance partner for them or not.

3) With the current non-random scheduling algorithm rookies are artificially pushed up the rankings.

4) My contention is that is doesn’t make nearly as much of a difference to low number teams with a good arm design as it does to low number teams with a good ramp/lift design. Dave told everyone at Bayou that the GDC wants teams to learn how to work together. They intentionally made the lift part of the game this year to challenge us to think about all the different types of robots that we will compete with and against. They intentionally wanted us to “get in the heads” of the other teams that we have never met. We actually did this. We sat down and thought about how we could make our robot as compatible with as many different unknown robot designs as possible. Then we proceeded to do the best we could to accommodate these designs. Our robot has and effective arm to place ringers and dual lifts that are spring loaded and are actuated by our alliance partner going up the ramp and into position and then are released by the alliance partners robot. Thus we are using part of the energy in our alliance partner to make the actual lift. While it was not compatible with all robot drivetrains we looked at the list of teams that would be competing at our regional and assumed that we would be randomly partnered with veteran teams at least as often as rookie teams. This was a false assumption. Had the rules given to us at kickoff stated “veteran teams will always be paired with rookie teams” I would have read between the lines that the chances of all the rookie teams having solid reliable drivetrains capable of climbing any sort of incline is much lower than veteran teams with established drivetrains. Also, I would have figured in the likelihood that the rookie drivers (with or without as solid, reliable drivetrain) probably didn't have as much driver practice as the veteran drivers who more than likely had a robot from a previous season to train their drivers on. Factoring these in we would have determined that ramps would not be as effective with rookie partners as they would be with veteran partners thus we would have focused more of our resources on a more effective arm and less on our ramps. As it stands we abandoned our usual strategy of do one thing and do it well and tried to do both this year. Unfortunately, this was not the year to change our usual strategy. That was our fault, however, if we had known then what we know now about the scheduling algorithm we would not have made that decision. I guess what I am saying is if you are an arm bot and you show up to a match with out any alliance partners or neither of your alliance partners can even move then at least you are capable of scoring ringer points on your own. You may or may not win but at least you can play the game. If you are in the same situation but are a ramp/lift bot at best you can play defense but you can’t actually score any points unless you have alliance partners. FIRST wants us to play the partner game but then they fix the matches so the chances are much higher that we won’t have any partners to play with.

Quote:

Originally Posted by Richard (Post 596140)
Students of statistics will appreciate that "randomness" in assigning qualifying match alliance partners and opponents is not a realistic goal. There is simply not enough time available.

Why? There were 37 teams at Bayou. We played 12 qualifying matches. That is 3 teams per alliance x 12 matches = 36 teams. Why couldn’t we each play with AND against each team but one?

Quote:

Originally Posted by burkechrs1 (Post 595933)
My thoughts about the alliance picking algorithm... IT"S NEVER GOING TO BE PERFECT! There simply aren't enough matches in a regional for one team to play every team they want. A basically what people are saying are they don't want to play with rookie teams and only want to play with veteran teams. Well not everyone is going to get what they want. Is it fair? No. But it's a randomizer for a reason. I think one reason it is a randomizer is so there isn't an obvious bias. Of course your going to get the near geniuses that will find some bias in there but why anazlyze the randomizer software. FIRST worked on fixing it from week 1. They did, what more do you want?

I believe you must not have actually looked at my attached spreadsheet or you would see that the “randomizer” in question is, in fact, in no way random. This is the sum of the issue.

Quote:

Originally Posted by burkechrs1 (Post 595941)
Exactly what did you pay for? I keep hearing "not what we payed for," but what exactly is it that you didn't get that you payed for. You got a kit, you got a regional that there is no way you can admit wasn't fun, and you got an experience you'll remember forever. Seriously what more could you ask. If you think you got ripped off because you didnt get chosen for an alliance for the finals, i'm sorry to say and I don't mean to sound rude but a huge portion of that has absolutely nothing to do with your record! Good record or bad record it all comes down to how you perfrom as a robot.

We all paid for the same opportunity to play with and against the same teams not a subset. Yes it does come down to how you perform, unfortunately, in a game where partners are important your performance is directly effected by your partners.

Andrew Blair 12-03-2007 22:02

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Lil' Lavery (Post 596246)
There are essentially three distinct schools of though I have noticed in this and the other threads dealing with this subject. Each of these has a different idea of the objective of the qualification matches:
*Seeding: that the qualification matches are for seeding teams based on ability, with the best teams emerging as the top seeds
*Performance: that the qualification matches are for determining which bots are the best by pitting them against each other and allowing them to showcase their abilities. The W/L/T records are not important, and it's the responsibility of scouts to make the determination of who is the best.
*Competition: that the qualification matches are to create dynamic and competitive matches with evenly matched alliances in every match to produce the most entertaining product possible.

Unfortunately it is probably impossible to create an algorithm and schedule that fulfills each of these criteria fully, and someone will always be unhappy with the results.

I'm not asking to find a perfect algorithm that makes every team do awesome and every team feel warm and fuzzy. You're absolutely right- that is impossible. I'm simply saying that what has existed in the past did an adequate job of fair ranking, whereas this year, everything seems to be out of whack.

To add some weigh to the "seedings should reflect a teams ability", FIRST themselves has an award specifically to reward the rookie with the highest seed. I doubt they would give such an award if they didn't put faith in the fact that rookies would be awarded it for an excellent robot design, not simply serendipitous matches.

meaubry 12-03-2007 22:17

Re: "New" 2nd Week Scheduling Algorithm
 
Sean,
I think you missed a 4th school - "Variety".
Variety - the desire to mix things up and not repeatedly play with or against the same partner or opponent.

The other 3 you mentioned were spot on though.

The beauty is that there are probably many solutions - if there was only that one criteria.

Mike

GaryVoshol 12-03-2007 22:21

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by fsayre (Post 596147)
When more then a few teams show up in the top 8 who can not score points I think something is wrong. I think the new matching algorithm makes the top 8 too much about luck rather than skill.

Another factor that I noticed contributing to this was that fact that a 3rd alliance partner can coast to victory on their partners' abilities. This happened at GLR on Friday; one robot got out on the field for only 1 or 2 of their 6 matches. But because their human player was there, they got credit for their alliance's accomplishments. At the end of the day they were ranked just below the top 10.

I'm going to use this as an example at Detroit when I'm lead queuer, to emphasize the importance of getting at least your human player to the field.

Ben Mitchell 12-03-2007 22:57

Re: "New" 2nd Week Scheduling Algorithm
 
Personally I don't think that team pairings should be made with a particular algorithm - pure randomization works for me.

With regards to teams being categorized into different groups based on number - that's more than a little unfair to all teams. I would have loved to have the (purely random) opportunity to play against different teams, regardless of age or status. To stratify teams within a competition is outrageous.

Teams did not get what they paid for equally - a spot in a regional should be just that - a spot in a randomized list, the same as any other team. To be separated and segregated really is against the spirit of how the competitions have been arranged in the past - or at least, how I have seen them - a part of the competition is seeing how you due in random pairings against randomly paired opponents.

Putting an even mix of differently aged teams on the field accomplishes nothing. Rookie teams, as well as veteran teams, should be viewed as equal when on the field and when determining when and with whom they play.

dtengineering 12-03-2007 23:38

Re: "New" 2nd Week Scheduling Algorithm
 
1 Attachment(s)
I'll briefly state my support for the non-pooled match scheduling system employed in previous years. Sometimes we ended up going against a powerhouse alliance, sometimes we were part of one, but I at least had confidence that overall the alliances would end up balanced. After all, we were being treated the same as everyone else. Now we aren't -- we're being treated the same as teams with the same team number, which is quite a different thing.

In fact, had we (Team 1346) attended the Wisconsin Regional, we would have been grouped in with the "veteran" pool. If we had attended Great Lakes we would have been in the "rookie" pool. At most regionals we would have been pool "B", smack in the middle.

Looking at the overall results of the five second weekend regionals (I've left off Brazil, not because I don't like Brazil, but because it is listed as a "Pilot" regional) and sorting the teams into pools based on team numbers and looking at the top eight qualifiers, then comparing the numbers to similar regionals from last year you get:

Top Eight Qualifying Spots Sorted by "Pool" for Second Week Regionals
Pool.............................................. .......2007............2006
Veterans (lowest 1/3 of team numbers).......14................21
Mid Year Teams......................................16..... ...........10
Rookies (highest 1/3 of team numbers).........10.................9

This would indicate to me that (as one would suspect) the pooling scheme has resulted in a more even distribution of top eight finishes amongst the three pools. Is this good? Is this bad? Are the numbers even significant?

All I know, is that if they are signficant, then veteran teams are being denied positions as alliance captains because of their team numbers. If the numbers aren't significant then the pooling system wasn't needed in the first place, was it?

Jason

P.S. Apologies for not formatting the excel file a little neater, it was mostly for my work and the data is summarized here.

rachal 13-03-2007 20:55

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Wayne C. (Post 595991)
My gripe with the first week schedule was not playing with rookies. Rather it was playing the same teams 5 times over.

Here are my team's (1072) qualifying matches for LA (there were 52 teams present):
Allies
597, 1722
599, 1669
702, 1759
399, 2174
597, 1836
702, 2174
254, 1702
606, 1759
691, 1855

Opponents
599, 1138, 1759
687, 812, 1644
4, 1722, 1197
330, 1438, 2272
294, 1266, 2178
599, 1266, 1669
399, 980, 2029
330, 867, 1669
330, 968, 2178

We had nine matches and interacted with a total of 31 teams, many of them multiple times. Getting paired with a few teams multiple times, sure that happens. But we were allied with four teams twice, played against four teams twice, and played against 330 three times. I still had fun, but it seemed repetitive after a while.

Quote:

Originally Posted by Ben Mitchell (Post 596359)
Personally I don't think that team pairings should be made with a particular algorithm - pure randomization works for me.

Same here. Complete randomization will let teams demonstrate their capabilities much more accurately (at least relative to each other), since lucky and unlucky alliances will balance out on average.

Madison 28-03-2007 17:27

Re: "New" 2nd Week Scheduling Algorithm
 
So, did anyone notice if this algorithm has remained the same since week 2 or if it's been changed through the other events?

The scheduling at Silicon Valley left a lot to be desired during week 3. We played a more randomized set of opponents, but not notably so. What's worse than playing against the same opponent four times, however, was the scheduling of our matches themselves. We had two sets of nearly back-to-back matches separated by a gap of 3.5 hours.

Has anyone else had a longer gap between matches? :) The closest we've had two consecutive matches was three apart.

Jeff Rodriguez 28-03-2007 17:49

Re: "New" 2nd Week Scheduling Algorithm
 
Here's our matches from Boston, week 4:

2099, 1757, 173 vs 2103, 1761, 246
1973, 1568, 173 vs 1922, 1403, 155
2103, 1686, 173 vs 2125, 1761, 69
1975, 1768, 173 vs 2079, 1761, 529
1973, 1626, 173 vs 1975, 1909, 97
2099, 1757, 173 vs 2262, 1831, 125
2110, 1761, 173 vs 1965, 1768, 126
1965, 1350, 173 vs 2124, 1685, 97
2043, 1685, 173 vs 2103, 1735, 155

You can draw your own conclusions.

Travis Hoffman 28-03-2007 17:54

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by M. Krass (Post 606805)
So, did anyone notice if this algorithm has remained the same since week 2 or if it's been changed through the other events?

The scheduling at Silicon Valley left a lot to be desired during week 3. We played a more randomized set of opponents, but not notably so. What's worse than playing against the same opponent four times, however, was the scheduling of our matches themselves. We had two sets of nearly back-to-back matches separated by a gap of 3.5 hours.

Has anyone else had a longer gap between matches? :) The closest we've had two consecutive matches was three apart.

One thing that hasn't changed - you could still accurately predict the 1st round of qualifying matches at Buckeye in Week 4.

Tom Saxton 29-03-2007 00:41

Re: "New" 2nd Week Scheduling Algorithm
 
I've posted a white paper with an analysis of both scheduling algorithms, proposed metrics for measuring the desirable qualities of a match schedule, explained a superior solution to the program, and posted a program that generates much better schedules.

Bharat Nain 30-03-2007 09:41

Re: "New" 2nd Week Scheduling Algorithm
 
It looks like they tuned the system up for Week 5. They still have the teams divided into 3 parts, but you don't play with/against everyone a whole lot often. In fact, if the regional is big enough, I think it tries to pair you up only once with/against teams. Anyone have a match list to confirm this?

mathking 30-03-2007 11:11

Re: "New" 2nd Week Scheduling Algorithm
 
Well, the analysis from my stat class is not done yet (I decided to wait until data from all of the regional matches is available) but a preliminary look seems to confirm that FIRST did succeed in "levelling the playing field" in terms of rookies and veteran teams. Using this algorithm there is basically no correlation between number of years in FIRST and record after qualifying. It is a completely different question as to whether this is "better" or "worse" than in the past.

The data from the past two years shows that veteran teams do have a significant advantage over non-veterans, but that advantage largely disappears once you get to about 3-4 years in FIRST. This is about what you would expect because by that time you are turning over team members. The really big advantage goes for teams competing in multiple regional competitions. The more competitions, the bigger the advantage. (But the biggest jump comes from the second regional.) I am going to try to compile some data from the past couple of years and find out if the "veteran" correlation gets stronger by comparing the total number of competitions attended to qualifying record. My suspicion is that it will.

A couple of other things jump out at first glance. The algorithm seems to work quite differently for larger regionals (over 50 teams) and smaller regionals. At a small regional there are many more repeat partners and opponents. The teams in the top 8 tend to have played multiple matches with other teams in the top 8. It would have been very interesting to keep track of number of points scored (or at least ringers placed, ramps successfully deployed or climbed) by each team. I know we kept data at Pittsburgh and the correlation between points scored by a team and its record in qualifying is a lot weaker than one would expect.

Tom Saxton's algorithm looks good to me. I am thinking that something like Arrow's Impossibility Theorem will be in play here in that there can't be a system that does the best under all criteria. So there is a need to look at the problems perceived in the current system and try to find a way to change them without causing other problems. I am also wondering if it might not be advisable for FIRST to incorporate how many total competitions a team has attended rather than just number of years in FIRST and perhaps how many competitions a team has/will attend that year.

The real question becomes two part: Should FIRST change the algorithm and will FIRST change the algorithm? Neither answer is really obvious to me.

Travis Hoffman 01-04-2007 10:52

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Bharat Nain (Post 607607)
It looks like they tuned the system up for Week 5. They still have the teams divided into 3 parts, but you don't play with/against everyone a whole lot often. In fact, if the regional is big enough, I think it tries to pair you up only once with/against teams. Anyone have a match list to confirm this?

We were against 1114 TWICE IN A ROW and with them once during qualifying at GTR, a very large regional.

We had several other repeating partners/opponents.

Draw your own conclusions.

Billfred 01-04-2007 11:23

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Bharat Nain (Post 607607)
It looks like they tuned the system up for Week 5. They still have the teams divided into 3 parts, but you don't play with/against everyone a whole lot often. In fact, if the regional is big enough, I think it tries to pair you up only once with/against teams. Anyone have a match list to confirm this?

Teams went ten rounds at Palmetto, which had 45 teams. Of 1618's rounds:

Played With:
Three times each: 665 and 1251
Two times each: 34, 343, 1102, 1293
Once each: 337, 538, 587, 900, 1249, 1390

Played Against:
Three times each: 1051, 1522, 2237
Two times each: 337, 538, 1436
Once each: 135, 281, 342, 386, 804, 832, 845, 900, 1026, 1225, 1251, 1287, 1390, 1553, 2187

Take from that what you will.

Kevin Sevcik 01-04-2007 11:46

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Bharat Nain (Post 607607)
It looks like they tuned the system up for Week 5. They still have the teams divided into 3 parts, but you don't play with/against everyone a whole lot often. In fact, if the regional is big enough, I think it tries to pair you up only once with/against teams. Anyone have a match list to confirm this?

We were with 1255 3 times at LSR, and they didn't have a prayer of dribing up our ramps. In fact, we only had about 3 matches where teams could actually drive on our ramps. If we had known we'd be consistently paired with rookies and never with veterans, it would've seriously affected our strategy and robot design at the beginning of th season. The very least FIRST can do next year is let us know what the match algorithm will be so we can plan accordingly.

Bharat Nain 01-04-2007 21:52

Re: "New" 2nd Week Scheduling Algorithm
 
OK, I guess I was wrong. I was working at the LI regional and I "heard" that they improved the match pairing system. The only improvement was the digital clock in the scoring software. Sorry for the confusion.

The Lucas 01-04-2007 22:06

Re: "New" 2nd Week Scheduling Algorithm
 
Quote:

Originally Posted by Bharat Nain (Post 609232)
The only improvement was the digital clock in the scoring software.

They also improved the rankings display. It goes from page to page instead of poorly scrolling. It also has both RS and MP now so you can actually see how they break ties

JackN 01-04-2007 22:21

Re: "New" 2nd Week Scheduling Algorithm
 
Code:

Team 494 Partners          Opponents
71        494        1718        74        515        1783
68        494        1504    2054        910        111
494    288        1481    1940        71        703
2188        302        494        2075        818        71
27        494        2075    2145        904        47
111        494        1783        904        2188        226
1918        494        288        703        71        2054
494        1918        111    1504        27        326

111-2 with 1 against
71-1 with 3 against
703-2 against (Both with Hammond)
1918-2 with
288-2 with
27-1 with 1 against
2188- 1 with 1 against
1783-1 with 1 against
2054-2 against

The system really bugs me. Take 1718 for example, Yes they have on of the best machines this year, but with the scheduling system they also have easier matches. We love 71 alot, but playing them three times is ludicrous. Admit it FIRST, the system is partially broken, go back to last years system or risk having several teams really upset with you


All times are GMT -5. The time now is 08:38.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi