"New" 2nd Week Scheduling Algorithm

[ChuckDickerson]

As a spin off from this thread I started this thread to specifically discuss the "new" 2nd week scheduling algorithm. The other thread was getting too long and was more of a discussion of the 1st week algorithm.

Last night I made a spreadsheet to analyze the Bayou match schedule (see attached). There were 37 teams. All the "veteran" teams (7+ year old) are color coded blue (12 teams). All the "mid year" teams (2-4 year old teams) are color coded green (12 teams). All the "rookie" (1st year) teams are red (13 teams). The lookup tab has team #, location, year started, team age (1st, 2nd, 3rd, etc. year), and an order # by team number (1-37). The schedule tab is the actual match pairings we played in New Orleans ordered from low team number on the left to highest team number on the right for both the red and blue alliances. The other tabs replace the team # with the year the team was started, team age, or order. As you can see there is nothing random about the pairings at all except whether an alliance was red or blue. It is clear that FIRST divided the teams into 3 sub-groups and intentionally partnered a veteran, a mid-year, and a rookie team for every alliance and then only randomly selected whether they were red or blue. This is not at all what I call random. FIRST has effectively segregated the teams into groups based entirely on team number/age. The only reason to do this is that FIRST is now implying that there is some advantage or disadvantage to being in one group or another. FIRST is without doubt introducing a bias for or against one group or another.

Depending on whether you are a pessimist or an optimist you can look at it in one of two ways:

1) A veteran team always HAS to play with a rookie team.

2) A rookie team always GETS to play with a veteran team.

But either way, at no time does the algorithm allow two veteran teams on the same alliance during qualification rounds. This effectively stacks the deck in favor of the rookies and against the veterans.

My warning to all teams heading into the upcoming regionals: We all pay the same money to play in the same regionals and deserve the same opportunity to play with the same teams. FIRST is not giving you what you paid for.

[Richard Wallace]

Originally Posted by DeepWater

... Depending on whether you are a pessimist or an optimist you can look at it in one of two ways:

1) A veteran team always HAS to play with a rookie team.

2) A rookie team always GETS to play with a veteran team.

But either way, at no time does the algorithm allow two veteran teams on the same alliance during qualification rounds. This effectively stacks the deck in favor of the rookies and against the veterans. ...

Hasn't this been beat to death yet? OK, so low numbered teams don't get to play with each other in qualifying. So what?

Like almost everyone, I agreed that the "perpetual opponent" algorithm was flawed. But as long as you get a variety of opponents so you can show your stuff, the new "random" algorithm seems fair enough to me.

In St. Louis, 148 got one of the worst draws I can imagine: every other match had them against either 217 or 45. And how did they respond? Undefeated #1 seed. (Well, except for one surrogate match.)

And although 45 seeded near the bottom, they showed the scouts clear and convincing evidence of their capabilities, and were picked 2nd in the draft -- their alliance captain was rewarded for that pick with a regional win.

[dtengineering]

Originally Posted by Richard

Hasn't this been beat to death yet? OK, so low numbered teams don't get to play with each other in qualifying. So what?

I will first of all agree with the rest of your post, in which you point out that excellent teams can still make it to the top of the qualifying rounds, and that it is possible for a low-seeded team to be picked by a top eight-seeded team. I don't think anyone is arguing that, but I feel obliged to address the question above.

The "so what" is that by introducing team number as a factor in scheduling matches FIRST is intentionally attempting to bias the outcome of the qualifying matches and introducing a factor into the final seeding that is in no way related to how well a team works or how well designed their robot is. Perhaps this would be acceptable if it were stated in advance in the tournament rules that "your team number will be a factor in determining your opponents and alliance partners", or if a win-loss record of qualifying matches was not kept. (Maybe we should just vote for the top eight teams?)

By reducing the randomness of the qualifying match draws, FIRST is not only ending up with a qualification process that less adequately represents the relative abilities of the teams in the overall standings, but also allows teams to question whether they are getting a "fair chance" to win matches.

To those who say that a biased match-scheduling system "doesn't matter", then if it doesn't matter why not just get rid of it and go back to random draws?

The fact is... it does matter, and it hasn't been "beat to death", because the biased scheduling system is not dead yet.

Jason

P.S. Although I argue this point strenuously, I'm not quite prepared to start marching to New Hampshire with an "Against Team-Number Oppression" banner or suggesting that FIRST isn't making an honest effort to do a good job... this is one of life's smaller annoyances, after all, but when a team does well... or poorly... in qualifying matches, those results should not be prejudiced by what date they first registered.

[Richard Wallace]

Originally Posted by dtengineering

The fact is... it does matter, and it hasn't been "beat to death", because the biased scheduling system is not dead yet.

Not dead yet?

[Nawaid Ladak]

just in relitive terms, the amount of matches has decreased significantly in the past four years.

look at florida fro example

Year one 104 Matches
Year two 86 Matches
Year three 77 Matches
This year 68 Matches

Thank god super-regionals are starting to pop up, like GTR adn UCF next year. this way, there are more matches, and i doubt UCF will fill up to 80 teams, because GTR only had 71 out of 80 lasy tear.

With two fields, you can divide the teams in half, so it would end up being like 35 teams per field.

but the only drawback is the 12 alliances instead of 16. Were using human nature here, we're using something that will NEVER BE PERFECT

[Lil' Lavery]

Originally Posted by FreedomForce

Year one 104 Matches
Year two 86 Matches
Year three 77 Matches
This year 68 Matches

"Year 1" also was a 2v2 game, and would require more matches for each team to participate in the same amount. Matches haven't gone down per team in my experience.
2004- 7 (VCU), 7 (Chesapeake)
2005- 8 (Chesapeake) 7 (Championship)
2006- 9 (VCU, 1 as "surrogate"), 10 (Peachtree-smaller regional)
2007- 8 (VCU)

[Jeffrafa]

Originally Posted by FreedomForce

just in relitive terms, the amount of matches has decreased significantly in the past four years.

look at florida fro example

Year one 104 Matches
Year two 86 Matches
Year three 77 Matches
This year 68 Matches

Originally Posted by Lil' Lavery

"Year 1" also was a 2v2 game, and would require more matches for each team to participate in the same amount. Matches haven't gone down per team in my experience.

I think what he's hitting at is not that each team is getting fewer matches than before (although this is the case in some situations), but that each regional is having notably fewer matches than in the past. Sure, 2004 was 2v2, and therefore required significantly more matches to grant teams as much time on the field. But just like 2004, the rounds are only 2 minutes and 15 seconds long. If Florida was able to manage 104 matches of that length in 2004, why are they doing only 65% as many rounds this year? Surely setting up two additional robots per match dosen't make a match cycle take 50% longer.

Since our team's rookie year in 2004 we have seen a significant decline in the number of matches we play per regional. I don't have any hard numbers for each event, but our rookie year we got between 10 and 12 matches at both the Pacific NW Regional and the Sacremento Regional. This year, despite being 3v3 now over the 2v2 the first year, we only got 8 matches - largely due to the regionals growing. Portland has gone from roughly 36 teams in 2004 to 54 teams this year.

- Jeff

[Fred Sayre]

When more then a few teams show up in the top 8 who can not score points I think something is wrong. I think the new matching algorithm makes the top 8 too much about luck rather than skill.

I think after qualification matches, the best teams should have more or less the best records. I think it will be discouraging when teams have excellent machines that don't make the top 8, or get picked by a rookie that can't score who was paired with the same dominating veteran the whole competition.

Good design, build, strategy, and gameplay are not being adaquately rewarded.

[Andrew Blair]

I don't care what teams we get paired with, what number they are, what color their eyes are. I want, as accurately as the tournament's dynamics allow, seeding matches to represent the actual abilities of the teams who play them.

An example.

Looking through many regional standings and videos, and our own second week experience, I have seen far to many box bots in the top eight than either the game's dynamics or historical precedent allow for. There are always robots who get carried into the top eight, but they used to be relatively sparse. Teams might see one or two teams float up throughout a couple regionals.

This year is not the same. I have not analyzed the algorithm extensively, I have not interpreted the match results to find out why a disproportionate amount of non-scoring robots are making it into the top eight. I do however feel that there is a problem that needs to be fixed- and I do not necessarily fix the blame on the new algorithm. There is an issue that is allowing teams who are not leading their alliances to seed extremely high with regularity, and I'm not at all sure why. But regardless of why it's happening, in my opinion, many people of all team numbers are getting the short end of the stick.

Teams that are not especially skilled in game play or strategy, and their alliances as explained in my linked post, are at a direct disadvantage going into finals, and it's not beneficial to anyone caught up in it. Teams aren't losing because of bad planning, design, or strategy, but because of good luck. The situation just doesn't seem to sit well in my mind, but it doesn't seem impossible to remedy either.

[Lil' Lavery]

There are essentially three distinct schools of though I have noticed in this and the other threads dealing with this subject. Each of these has a different idea of the objective of the qualification matches:
*Seeding: that the qualification matches are for seeding teams based on ability, with the best teams emerging as the top seeds
*Performance: that the qualification matches are for determining which bots are the best by pitting them against each other and allowing them to showcase their abilities. The W/L/T records are not important, and it's the responsibility of scouts to make the determination of who is the best.
*Competition: that the qualification matches are to create dynamic and competitive matches with evenly matched alliances in every match to produce the most entertaining product possible.

Unfortunately it is probably impossible to create an algorithm and schedule that fulfills each of these criteria fully, and someone will always be unhappy with the results.

[Andrew Blair]

Originally Posted by Lil' Lavery

There are essentially three distinct schools of though I have noticed in this and the other threads dealing with this subject. Each of these has a different idea of the objective of the qualification matches:
*Seeding: that the qualification matches are for seeding teams based on ability, with the best teams emerging as the top seeds
*Performance: that the qualification matches are for determining which bots are the best by pitting them against each other and allowing them to showcase their abilities. The W/L/T records are not important, and it's the responsibility of scouts to make the determination of who is the best.
*Competition: that the qualification matches are to create dynamic and competitive matches with evenly matched alliances in every match to produce the most entertaining product possible.

Unfortunately it is probably impossible to create an algorithm and schedule that fulfills each of these criteria fully, and someone will always be unhappy with the results.

I'm not asking to find a perfect algorithm that makes every team do awesome and every team feel warm and fuzzy. You're absolutely right- that is impossible. I'm simply saying that what has existed in the past did an adequate job of fair ranking, whereas this year, everything seems to be out of whack.

To add some weigh to the "seedings should reflect a teams ability", FIRST themselves has an award specifically to reward the rookie with the highest seed. I doubt they would give such an award if they didn't put faith in the fact that rookies would be awarded it for an excellent robot design, not simply serendipitous matches.

[meaubry]

Sean,
I think you missed a 4th school - "Variety".
Variety - the desire to mix things up and not repeatedly play with or against the same partner or opponent.

The other 3 you mentioned were spot on though.

The beauty is that there are probably many solutions - if there was only that one criteria.

Mike

[ChuckDickerson]

Originally Posted by Richard

So what?

A few thoughts off the top of my head:

1) I don’t think that FIRST segregating team by purposely pooling low, medium and high teams is a good thing. By FIRST segregating teams into “pools” they are inherently saying that one group is different than the other. Whether they are “better” or “worse” or otherwise “special” I can only speculate but as a society I thought we learned somewhere back in the 60’s that segregating groups of people is never a good idea.

2) In a regional with a lot rookie teams (more than 1/3 of the teams in this case), many of the rookies don’t yet understand the importance of good scouting and tend to pick the next highest ranked team whether or not that team will be the best alliance partner for them or not.

3) With the current non-random scheduling algorithm rookies are artificially pushed up the rankings.

4) My contention is that is doesn’t make nearly as much of a difference to low number teams with a good arm design as it does to low number teams with a good ramp/lift design. Dave told everyone at Bayou that the GDC wants teams to learn how to work together. They intentionally made the lift part of the game this year to challenge us to think about all the different types of robots that we will compete with and against. They intentionally wanted us to “get in the heads” of the other teams that we have never met. We actually did this. We sat down and thought about how we could make our robot as compatible with as many different unknown robot designs as possible. Then we proceeded to do the best we could to accommodate these designs. Our robot has and effective arm to place ringers and dual lifts that are spring loaded and are actuated by our alliance partner going up the ramp and into position and then are released by the alliance partners robot. Thus we are using part of the energy in our alliance partner to make the actual lift. While it was not compatible with all robot drivetrains we looked at the list of teams that would be competing at our regional and assumed that we would be randomly partnered with veteran teams at least as often as rookie teams. This was a false assumption. Had the rules given to us at kickoff stated “veteran teams will always be paired with rookie teams” I would have read between the lines that the chances of all the rookie teams having solid reliable drivetrains capable of climbing any sort of incline is much lower than veteran teams with established drivetrains. Also, I would have figured in the likelihood that the rookie drivers (with or without as solid, reliable drivetrain) probably didn't have as much driver practice as the veteran drivers who more than likely had a robot from a previous season to train their drivers on. Factoring these in we would have determined that ramps would not be as effective with rookie partners as they would be with veteran partners thus we would have focused more of our resources on a more effective arm and less on our ramps. As it stands we abandoned our usual strategy of do one thing and do it well and tried to do both this year. Unfortunately, this was not the year to change our usual strategy. That was our fault, however, if we had known then what we know now about the scheduling algorithm we would not have made that decision. I guess what I am saying is if you are an arm bot and you show up to a match with out any alliance partners or neither of your alliance partners can even move then at least you are capable of scoring ringer points on your own. You may or may not win but at least you can play the game. If you are in the same situation but are a ramp/lift bot at best you can play defense but you can’t actually score any points unless you have alliance partners. FIRST wants us to play the partner game but then they fix the matches so the chances are much higher that we won’t have any partners to play with.

Originally Posted by Richard

Students of statistics will appreciate that "randomness" in assigning qualifying match alliance partners and opponents is not a realistic goal. There is simply not enough time available.

Why? There were 37 teams at Bayou. We played 12 qualifying matches. That is 3 teams per alliance x 12 matches = 36 teams. Why couldn’t we each play with AND against each team but one?

Originally Posted by burkechrs1

My thoughts about the alliance picking algorithm... IT"S NEVER GOING TO BE PERFECT! There simply aren't enough matches in a regional for one team to play every team they want. A basically what people are saying are they don't want to play with rookie teams and only want to play with veteran teams. Well not everyone is going to get what they want. Is it fair? No. But it's a randomizer for a reason. I think one reason it is a randomizer is so there isn't an obvious bias. Of course your going to get the near geniuses that will find some bias in there but why anazlyze the randomizer software. FIRST worked on fixing it from week 1. They did, what more do you want?

I believe you must not have actually looked at my attached spreadsheet or you would see that the “randomizer” in question is, in fact, in no way random. This is the sum of the issue.

Originally Posted by burkechrs1

Exactly what did you pay for? I keep hearing "not what we payed for," but what exactly is it that you didn't get that you payed for. You got a kit, you got a regional that there is no way you can admit wasn't fun, and you got an experience you'll remember forever. Seriously what more could you ask. If you think you got ripped off because you didnt get chosen for an alliance for the finals, i'm sorry to say and I don't mean to sound rude but a huge portion of that has absolutely nothing to do with your record! Good record or bad record it all comes down to how you perfrom as a robot.

We all paid for the same opportunity to play with and against the same teams not a subset. Yes it does come down to how you perform, unfortunately, in a game where partners are important your performance is directly effected by your partners.

[GaryVoshol]

Originally Posted by fsayre

When more then a few teams show up in the top 8 who can not score points I think something is wrong. I think the new matching algorithm makes the top 8 too much about luck rather than skill.

Another factor that I noticed contributing to this was that fact that a 3rd alliance partner can coast to victory on their partners' abilities. This happened at GLR on Friday; one robot got out on the field for only 1 or 2 of their 6 matches. But because their human player was there, they got credit for their alliance's accomplishments. At the end of the day they were ranked just below the top 10.

I'm going to use this as an example at Detroit when I'm lead queuer, to emphasize the importance of getting at least your human player to the field.

[dtengineering]

I'll briefly state my support for the non-pooled match scheduling system employed in previous years. Sometimes we ended up going against a powerhouse alliance, sometimes we were part of one, but I at least had confidence that overall the alliances would end up balanced. After all, we were being treated the same as everyone else. Now we aren't -- we're being treated the same as teams with the same team number, which is quite a different thing.

In fact, had we (Team 1346) attended the Wisconsin Regional, we would have been grouped in with the "veteran" pool. If we had attended Great Lakes we would have been in the "rookie" pool. At most regionals we would have been pool "B", smack in the middle.

Looking at the overall results of the five second weekend regionals (I've left off Brazil, not because I don't like Brazil, but because it is listed as a "Pilot" regional) and sorting the teams into pools based on team numbers and looking at the top eight qualifiers, then comparing the numbers to similar regionals from last year you get:

Top Eight Qualifying Spots Sorted by "Pool" for Second Week Regionals
Pool.............................................. .......2007............2006
Veterans (lowest 1/3 of team numbers).......14................21
Mid Year Teams......................................16..... ...........10
Rookies (highest 1/3 of team numbers).........10.................9

This would indicate to me that (as one would suspect) the pooling scheme has resulted in a more even distribution of top eight finishes amongst the three pools. Is this good? Is this bad? Are the numbers even significant?

All I know, is that if they are signficant, then veteran teams are being denied positions as alliance captains because of their team numbers. If the numbers aren't significant then the pooling system wasn't needed in the first place, was it?

Jason

P.S. Apologies for not formatting the excel file a little neater, it was mostly for my work and the data is summarized here.