Re: Auto Rank Vs. Human Rank
 

I agree it is an interesting problem. However, humans being what they are, I doubt if it is solvable. While you might be able to make a system that is resistant to those less laudable inputs, I doubt they could be eliminated entirely.

One of the basic assumptions of the current tournament structure seems to be that two properly combined teams working together can out perform a single very capable team with a mediocre partner, or one that isn't a good fit. Smart teams look for synergy in scouting for partners. Steve's team was probably chosen because they could do one aspect of the game reliably and well. The Alliance leader could then concentrate on other things because they were pretty sure Steve's team would handle their end.

Because different teams will have different strengths and weaknesses, they should look for different things in a partner. That means they should use different weighting factors in evaluating other teams. The team that is a perfect fit for the BeachBots might be a team that does exactly the same things as Chief Delphi (to pick a couple of random examples ) :D So they might be the right team for us and the wrong one for CD. There are many factors that go into such a decision, so picking the right partner can be a real challenge. Especially if there are other smart teams out there looking for the same kinds of robots you are.

So I don't think we will ever be able to come up with any uniform ranking structure. We might be able to come up with a relatively consistent database for teams to use for their own ranking purposes. But even that would be a challenge.

Now that the waters have been sufficiently muddied...

There have been some interesting questions raised by this discussion.

1. Is there a "true ranking" of teams?

2. Should the qualifying rounds be designed to produce this "true ranking"?

3. Since Eliminations are going to pit alliances (not teams) against each other, should the rankings incorporate the "functional class" into which a robot fits?

My answers are:
1. There are several "true rankings" of teams. The goal is to define a relevant ranking. Each ranking would be a scalar cost function which takes a set of inputs (the teams at a competition) and converges to the same values after a finite number of iterations (8 matches).

Since there are a finite number of teams and matches, this process should be deterministic rather than stochastic. Therefore, it should include the initial selection of teams for matches and should be invariant over the order in which teams are entered into the process.

2. I think that qualifying should produce true ranking. However, there seems to be some debate.

If some random component to seeding were desired, I would prefer to play a few qualifying matches and then draw seeds out of a hat, where the order of the draw is determined by your rank at the end of qualifying. With such a system, we could dispense with qualifying in half a day and go to a double elimination tournament for the next day and a half. (Note: there is heavy sarcasm in the above statement. Don't take it seriously.)

3. Each year, there have been generally about five practical "solutions" to the game. Each one of these solutions could be called a class (A, B, C, D, E). Robots generally fit into one or two classes.

This year the classes seem to be:
A = strong man, hill taker
B = autonomous stack attack
C = bin pusher, sweeper, collector
D = stacker
E = hill dominator

In a well ordered world (ie one in which rock-paper-scissors doesn't work), the combination of the best in class for two of the classes will always be the winning alliance.

In other words, best(A) + best(B) will always beat best(C)+best(D), best(C)+best(E), best(D)+best(E), second best(A) + second best(B), second best(A)+best(C,D,E), second best(B)+best(C,D,E).

The ideal ranking procedure should accomplish two things. It should establish the precedence (A > B > C > D > E) and it should establish ranking within the classes. In the case of multi-class robots, they would potentially fall in between. In other words, an A+B robot (which is not either best(A) or best(B)) should fall in between best(A) and best(B) in the rankings.

Seems to me that this issue is two-fold. First, does the strategy , effective robotics design and game play have anything to do with standings in all cases? An argument could be made that alliance and luck have something to do with it as well as the former. Thus the "standings" on the first day are arbitrary to some degree. Second, if a true pairing was to be made then the "practice day" would actually be a day for point allocation and a seeding process. In this case the "practice" sessions would be actual competition and teams would be "seeded" based upon their performance on this day. Doing this could create a tremendous amount of problems. Some teams are still on a "shakeout cruise" mode and need the day for trial and error. Some teams have already competed at one or more Regional events and are simply scoping out the competition and fine tuning their 'bot. Thus, some could argue that a true "seeding" is impossible. The only other means of a true seeding would be to take the first round scores of all teams, rank order and create a round-robin where the top and bottom teams would pair off in an elimination setup and only the strongest would survive into a pool where alliance teams could be managed. It would be a tremendous opportunity to really see who had what it takes but the administration of the "game" would be a logistical nightmare! I like the idea of fairness and ultimate selection of a "champion" but ultimately the goals of FIRST and a true, even and fair tournament may never be possible under the current system.
Happy 4th of July!
:cool:

paper scissors rock is common...
 

This brings up one problem I think with rankings. I think that Paper-Scissors-Rock situations happen all the time in FIRST competitions.

By this I mean that A can almost always beat B, B can almost always beat C, and C can almost always beat A.

Think back to your own experience. In the Elimination Rounds, how many times have you hoped that team XXX would come up against team YYY because you thought you would have difficulty with team XXX but you could beat YYY and that team YYY could probably beat XXX?

I can think of a number of times this has come up.

Joe J.

agree wrt rock-paper-scissors
 

I agree that rock-paper-scissors situations make it difficult to come up with a true ranking that is meaningful. Thats why it is so great to allow the teams to pick their partners for eliminations - they get to use their own strategy and opinion to decide on a team's rank.

Publishing the ally-opponent list is a great thing because it gets teams talking to each other. I strongly believe that we should not do away with this in any sort of seeding system.

Ken

The best thing about 2v2 and publishing matches in advance is that you are really incentivized to talk to other teams.

The worst thing about the 4v0 year was that you had twelve teams in a flight and you didn't know who you were going to be paired with until instants before the match. That was a bad ide-ar!

My team only has enough focus to talk strategy for one match in advance, so an hour before the match is plenty of time to find out who our partner is.

This year's lack of randomization was disheartening because we didn't get much variety in pairings. Another reason for a partially deterministic approach.

I agree that rankings are never going to be absolute. However, it would be nice if the number 1 ranked team were better than the number 20 ranked team which is better than the number 40 ranked team.

I would also like to see more "marquee matchups." A system which opposes teams which are competitive with each other would yield matches which would be more interesting for us (the players) and the spectators.

As for including subjective information in the rankings...
If the qualifying followed a deterministic approach (high seeds playing each other, low seeds playing each other) I suppose you could have a "coaches poll" (like the college football rankings) at the end of practice day and form an initial seeding. After that, you go up or down based on performance.

Ok as head of all that stuff on my team. I would actually liek to relax. I prob got no more than 5 min of actualy sitting down doing nothing time at regionals. Those 5 min would become 0 if we went back to not knowing before.

Please, I would like to see my friends too!!!

If one's robot is truly that good, then the help of others is not necissary. In 2002, through 5 competitions, we had appx. 8 matches where we were alone (with a dead bot or a no-show), and those 8 matches were some of our best matches. We never had a problem winning, and when we were alone.. we sorta shined because we didn't have to worry about getting our parner back home. We were multi-functional.. and could easily score 40 points by ourselves. We really didn't play well with others! ;)

All in all, I drove through the games of Ramp Riot, Zone Zeal, and Stack Attack. It is a complete nightmare when you have no idea who you will be paired with in your next match. The 2 vs. 2 is a good setup so is 1 vs. 1 vs. 1 from back in the old day of '98.

I would agree with you, as we won the play of the day in 2002 with a non functional partner. However this is not always the case as problems with your own robot can occur. Also I think the whole working together buisneuss thing is nice to have with the 2v2 matches.