Open Challenge: Make a better FRC ranking algorithm

[Billfred]

I've been involved with FIRST for three seasons now, and with the exception of some oddities in the Hatch software forcing a team update to fit it midway through 2005, there hasn't really been much change to how teams are ranked. Robot sizes have changed, games have changed, weight has changed, point systems have changed, technology has evolved, but at least for as long as I've been around, it's been QPs and RPs equal to the losing score.

So just for the fun of it, how would you rank teams if you ruled the world? Any method is fair game, but bonus points to those that are plausible enough that FIRST could actually use them. And assume the game will be similar in format to Aim High, to keep things simple. (I'll post mine later.)

[ewankoff]

winning team gets their own score if the margin of victory is less than 1/3 of the losing team's score. if not they get the losing teams score.

losing team gets either the margin or their own score whichever is higher.

i think this sounds good. this way blowouts favor the losing team and winning teams will try and keep the margins low.

edit:this is added to the normal ranking by record and is used only to break ties for teams with the same record.

[Jeff Rodriguez]

I don't think that the ranking system should change.
All the things you mentioned (game, weight, size, scoring) are part of the challenge. Determining which robot is the best should not be part of that challenge.
The ranking system is actually similar to many other sports or competitions. If you win all your games, you'll be ranked well.
The only small twist that FIRST has is the ranking point system and it main purpose is to distinguish between two team whose records are tied.

Whatever people come up with, the Win/Loss record should always be the biggest determining factor. Having a system that rewards close wins over dominant wins seems contradictory to me. If a robot can shutout its opponents each time it plays, why shouldn't it be ranked #1?

In the eliminations, a win is a win is a win. It doesn't matter if you win by 100 or tie and win by tiebreaker.

Also, if it ain't broke, don't fix it.

[KenWittlief]

I think FIRST should be more like the Olympics

teams are scored not only on technical aspects of the game (whether a ball is placed in a goal....)

but also on style and artistic expression!

At the end of each match the 10 judges along the sideline would hold up cards, with their scores for both the winning and the losing alliance.

This would eliminate the need to base scores on the losing alliance, because, while you can help your opponent fix a mechanical or electrical part before a match, there is no way you can teach a clutzy robot artistic expression in 1 day!

[Daniel_LaFleur]

My 2 Cents:

Losing team gets the winning teams score.

Winning team gets the winning teams score + 2x the losing teams score.

Rank is determined by win-loss record, with ties being broken by the team scores.


This would create an atmosphere of offensive scoring and helping the opponents to score.

[Jack Jones]

I think FIRST should be more like the Olympics

teams are scored not only on technical aspects of the game (whether a ball is placed in a goal....)

but also on style and artistic expression!

At the end of each match the 10 judges along the sideline would hold up cards, with their scores for both the winning and the losing alliance.

This would eliminate the need to base scores on the losing alliance, because, while you can help your opponent fix a mechanical or electrical part before a match, there is no way you can teach a clutzy robot artistic expression in 1 day!
Yuck! Judges? No thanks!

I think the rankings are fine as they are. However, I think the top eight should stay that way. That is, no building powerhouse aliances by picking within the seeded teams. Scouting would be much more important than it is now.

I realize that it may encourage the seventh or eighth seed to sandbag their last match in order to finish ninth and thereby have a chance to get selected by a top seed, but they also run the risk of not getting picked at all.

Maybe this, and having a 1..8 then 8..1 selection order, would make for better elimination matches than the blowouts we get now.

[Billfred]

Yuck! Judges? No thanks!

I think the rankings are fine as they are. However, I think the top eight should stay that way. That is, no building powerhouse aliances by picking within the seeded teams. Scouting would be much more important than it is now.

I realize that it may encourage the seventh or eighth seed to sandbag their last match in order to finish ninth and thereby have a chance to get selected by a top seed, but they also run the risk of not getting picked at all.

Maybe this, and having a 1..8 then 8..1 selection order, would make for better elimination matches than the blowouts we get now.
I'm not so quick to block picking within the top 8. At Duel, 25's first pick was MOE, the #9 seed. I'm not saying MOE was sandbagging by any stretch--25 just made the best decision they could under the rules. I'd imagine that you'd see a lot of that under a restricted picking system. (And besides, the other teams in the top 8 can always decline.)

If you ask me, I think the best way to go about it would be to go by QPs, then the average QPs of your opponents over the qualifying rounds. If you're beating other highly-seeded teams, which are usually harder opponents, you'll do better than if you're beating teams that are 0-8. Furthermore, it encourages teams to do everything they can to help the very teams they'll be going against--you want your opponents to win every match (except the ones you're in, naturally).

[Jack Jones]

.... (And besides, the other teams in the top 8 can always decline.)



I'm glad you brought that up. Because, it sometimes causes hard feelings. It's as hard to decline as it is to be snubbed. We've all seen times when a team seeds high due to the luck of the draw wrt/ alliance partners. Then we see team after team decline them. Now suppose they could only pick from number nine down, and the team they selected had sandbagged their way there. I call that justice. ;)

[Steve W]

Leave as is with win/loss but change the way RP points are tabulated. If you take the losers score, subtract from the winners score, take the remainder and subtract from 100 to give RP. The losing alliance gets 2/3 of the RP. Example:

Red 52 Blue 41
52 - 41 = 11
100 - 11 = 89 RP winning alliance
89/3 x 2 = 60 RP losing alliance

Red 52 Blue 10
52 - 10 = 42
100 - 42 = 58 RP winning alliance
58/3 x 2 = 38 RP losing alliance

Red 89 Blue 0
89 - 0 = 89
100 - 89 = 11 RP winning alliance
11/3 x 2 = 8 RP losing alliance

Red 98 Blue 97
98 - 97 = 1
100 - 1 = 99 RP winning alliance
99/3 x 2 = 66 RP losing alliance

Red 10 Blue 9
works out to the same as above

There would be a max amount of RP per game. This encourages close games which are exciting. Not sure of how to handle ties at this point but maybe 50 RPs for a tie. The closer the score, the higher the RPs for both winning and losing. Would also make strategy much tougher.

[Jeff Rodriguez]

Leave as is with win/loss but change the way RP points are tabulated. If you take the losers score, subtract from the winners score, take the remainder and subtract from 100 to give RP. The losing alliance gets 2/3 of the RP. Example:

clip...

Why 100? Would that number change from year to year? What happens if, in the future, a normal score is Red 445 v. Blue 290?

[BandChick]

I really like Steve's idea. But, Jeff brings up a good point, rather than 100, it should be the next highest hundred.

So, say the score is
Red 98 - Blue 90 --- use 100
Red 103 - Blue 95 --- use 200
Red 205- Blue 115 --- use 300
and so on...

That would account for scores over 100, and continued growth in scoring for the future.

[Steve W]

Why 100? Would that number change from year to year? What happens if, in the future, a normal score is Red 445 v. Blue 290?

I choose an arbitrary number. Your score would still work.
Red 445 Blue 290
445 - 290 = 155
100 - 155 = 0

Both teams get 0 RP

As was asked, I came up with an idea. Numbers can be changed to work with the scoring scheme. I like 100 as it stops blowouts as can be seen by your scores. If the score had been 445 - 444 then the winning alliance would get 99 RPs and the losing 66 RPs.

[Jherbie53]

Leave as is with win/loss but change the way RP points are tabulated. If you take the losers score, subtract from the winners score, take the remainder and subtract from 100 to give RP. The losing alliance gets 2/3 of the RP.
There would be a max amount of RP per game. This encourages close games which are exciting. Not sure of how to handle ties at this point but maybe 50 RPs for a tie. The closer the score, the higher the RPs for both winning and losing. Would also make strategy much tougher.
Another way for RP's would be take the losing teams points as RP's. Winner gets 2x the points, and 1pt off for every 5pts or 10pts (depends on the game) the winning alliance is higher than the losing team. This would also create close matches.

This year I saw some teams go and shoot balls in the higher goal of the their opponents, so they would have a higher RP. It makes sense, but you can really make some teams think that you think they can't do it. So you could subtract points from the teams that scores for the other. Example: Red scores 15pts on their own and Blue scores 50+pts in the match. Blue wants more RP's, and scores 15pts for Red making their total 30pts. All of the Red teams would get the 30pts and the Blue teams would get 15pts. This would mean that either Blue lets Red score more or they don't score for them. This sounds like its an OK solution but not a perfect one for stopping it.

Edit: RP=Ranking Points - instead of QP=Qualifying Points

[Dan Petrovic]

I have a problem with Steve W's idea.

I don't think QP should be directly related to the opponent's score like that.

Who's fault is it if a powerful team is set up against less powerful opponents?
I don't like the idea having the teams who worked hard to make a strong, competitive robot suffer because of a computer program that randomly generates alliances.

In 2003, I think the QP of the winner = 2x loser's score + winner's score. 166 suffered because of that. I wasn't on the team at that point, but they built a very good robot and would win matches with 90 points or something to 3 points. I think robots who can shut out teams like that deserve to be ranked higher.

That may sound like a biased opinion, but everyone has to have their own biased opinions at some point in their lives.

[Donut]

If you ask me, I think the best way to go about it would be to go by QPs, then the average QPs of your opponents over the qualifying rounds. If you're beating other highly-seeded teams, which are usually harder opponents, you'll do better than if you're beating teams that are 0-8. Furthermore, it encourages teams to do everything they can to help the very teams they'll be going against--you want your opponents to win every match (except the ones you're in, naturally).

If FIRST ever wants a system that factors in the difficulty of your opponents, like this, then you can copy many of the sports ranking systems.

Arizona Football does a "power-point" system for seeding, with your team receiving 50 points for every win and 5 points for each of your opponent's wins. You divide all the points from your wins by the total games you played.

They play a 10 game schedule, and the theory behind the system is that beating a team with no wins (resulting in 50 points for your win) should be equivalent to losing to an undefeated team (resulting in 50 points for their 10 wins).

There are huge flaws in applying this to a system with multiple team alliances and opponents though, and I think I've figured out a system but am not sure of how accurate it would really be.

[Billfred]

There are huge flaws in applying this to a system with multiple team alliances and opponents though, and I think I've figured out a system but am not sure of how accurate it would really be.
What flaws would there be? The only one that comes to mind (and I haven't tested it yet) is that everyone would have the same opponent-win percentage if everyone played everyone else once at a regional--but when does that happen?

[Tom Bottiglieri]

The system works. No need for change. If a team doesn't end up a good position, that means they weren't good enough.

[ewankoff]

possibly use a combined record to gauge an alliances record to fit into the above power-point system

[Donut]

What flaws would there be? The only one that comes to mind (and I haven't tested it yet) is that everyone would have the same opponent-win percentage if everyone played everyone else once at a regional--but when does that happen?

I meant just using a straight copy of it would be flawed. If instead you factor in each opponents' wins and also each of your alliance partners' losses, you could make a better system (the idea then would be a win over 3 winless opponents with 2 undefeated partners would be equal to a loss to 3 undefeated opponents with 2 winless partners).

The bigger problem is I'm not sure how accurate it would be in a system where your opponents can also be your partners, as the original system wasn't designed for this type of scenario.

I like the current system (especially when compared to the old one from 2003 and earlier), and whether it should be changed really depends on what FIRST wants to promote in matches and design.

If they want high scoring matches, they should keep the current system, which rewards an offensive strategy more than a defensive one.
If they want to account more for opponents' strength, they need a system like Billfred's or many high school sports.
If they want close games, they should probably go back to the old system or get one like Steve's.
If they want something else... well, adjust accordingly.

Regardless of how FIRST does rankings, I will still use something more like my idea when trying to figure out alliance selections, because when I'm doing that I care far more about how good they are performing than how close their matches have been and whether they're an offensive or defensive robot. FIRST would want to promote exciting matches, but from a success viewpoint I would not want to be part of one because that usually means one team barely wins a shootout or that they come from behind for a close victory.

[Imajie]

Another way for QP's would be take the losing teams points as QP's. Winner gets 2x the points, and 1pt off for every 5pts or 10pts (depends on the game) the winning alliance is higher than the losing team. This would also create close matches.

This year I saw some teams go and shoot balls in the higher goal of the their opponents, so they would have a higher QP. It makes sense, but you can really make some teams think that you think they can't do it. So you could subtract points from the teams that scores for the other. Example: Red scores 15pts on their own and Blue scores 50+pts in the match. Blue wants more QP's, and scores 15pts for Red making their total 30pts. All of the Red teams would get the 30pts and the Blue teams would get 15pts. This would mean that either Blue lets Red score more or they don't score for them. This sounds like its an OK solution but not a perfect one for stoping it.
Instead of not giving the winning alliance the points they scored, take those away from the points the other alliance actually scored. So, using your example the Blue alliance would get 0 points. Reds 15 - their 15 they scored for Red = 0. This would greatly discourage scoring for your opponent.

[Chris Marra]

The only issue I can see with the current system is when one team on an alliance is liable for the loss of the alliance. All penalties, and especsially back-bot penalties, are applied to the alliance, even though its possible one team was designed responsible for being backbot before the match. In this event, a team might lose a close match by another drive team's mistake, and then two teams are penalized for one teams actions, and that 2-point sway on a record can make a big difference at a regional with few qualifiers, or at Nationals.

Otherwise, the present system works fine and there is no reason for a change. A team's record demonstrates typically demonstrates how well and their robot having been performing at a competition.

[Lil' Lavery]

Quick note of clarification first off. There has been a HUGE discussion over how to determine QPs, when in reality you were discussing RPs. Qualification Points (QPs) were determined in 2006 by 2 points for a win, 1 for a tie, and 0 for a loss. RANKING Points (RPs) were the adverage score of the losing alliance in matches you participated in.
I don't really think this system is broken. I think it works very well, in fact. W/L/T should be the primary factor. RP serves both as a measure of "schedule strength" and GP.
The problem with using your opponent's W/L/T as a Strength of Schedule is that it doesn't rank how strong they were in the particular match you faced them in, but rather how strong they were over the entire competition. For example, say Team 9999 has a 9-1 record, but you faced them in their only loss, when they had a malfunction and their robot never moved. With a "power ranking" system, it would reward you for beating a team that was a top notch team, but in reality you beat a team that never even moved. With the current RP system, you would likley receive a lower RP because they didn't actually score points. Another situation may be when you face excellent teams that don't work well together as an alliance. Like a combination of 3 herders (where you could simply block the corner goals all match), or (in 2004) two cappers with not so hot human players.
In short, if you wanted to modify the system to have more emphasis on schedule strength, you have to use a system that uses strength in the matches you competed in, not overall competition strength.

[Donut]

Quick note of clarification first off. There has been a HUGE discussion over how to determine QPs, when in reality you were discussing RPs. Qualification Points (QPs) were determined in 2006 by 2 points for a win, 1 for a tie, and 0 for a loss. RANKING Points (RPs) were the adverage score of the losing alliance in matches you participated in.
I don't really think this system is broken. I think it works very well, in fact. W/L/T should be the primary factor. RP serves both as a measure of "schedule strength" and GP.
The problem with using your opponent's W/L/T as a Strength of Schedule is that it doesn't rank how strong they were in the particular match you faced them in, but rather how strong they were over the entire competition. For example, say Team 9999 has a 9-1 record, but you faced them in their only loss, when they had a malfunction and their robot never moved. With a "power ranking" system, it would reward you for beating a team that was a top notch team, but in reality you beat a team that never even moved. With the current RP system, you would likley receive a lower RP because they didn't actually score points. Another situation may be when you face excellent teams that don't work well together as an alliance. Like a combination of 3 herders (where you could simply block the corner goals all match), or (in 2004) two cappers with not so hot human players.
In short, if you wanted to modify the system to have more emphasis on schedule strength, you have to use a system that uses strength in the matches you competed in, not overall competition strength.

The herder scenario would be another problem with the power-point rankings. They weren't designed for alliance competitions, and so probably wouldn't be as effective.

The current system has little problems. It does discourage defense to an extent; but this keeps teams from putting the same robot on the field year after year that is designed simply to push people around and get in their way. Since the FIRST rankings only determine who are alliance captains, not who is ultimately in the playoffs, they should be designed to promote what FIRST wants us to do, and it should be up to the teams to figure out their way to determine who are the best teams to pick.

Since this thread is about designing a better system for truly ranking teams, I will continue to develop my idea, though I hope FIRST won't actually use it.

[Rick]

Using a system where the rank is decided by the difference of the winner and loser score is horrible from a strategy point of view. Any ranking system in which a team needs to "hold back" scoring to seed higher will not be received well after the current system. Wins and loses need to continue to be a factor for ranking.

A team that wins all their matches should be number 1. Two teams that win all thier matches are decided by the higher RP MEANING they have played the "tougher" opponents. Tougher meaning the opponents scored more points.

This system has been succesful since 2004. Compared to 2003's ranking system its night and day. Lets keep this simple. We want to make this easy for spectators right? Gracious professional refers to making your grandmother proud. If I can't explain the game and its ranking to my grandmother, then its too complicated.

[Noah Kleinberg]

We want to make this easy for spectators right? Gracious professional refers to making your grandmother proud. If I can't explain the game and its ranking to my grandmother, then its too complicated.

Exactly what I thought when I read this thread...

The system now works well, and is easily understood by spectators. They shouldn't have to do any math, or try to think of how the other robots have been doing throughout the day in order to know who "won" the match in terms of ranking.

Now the serpentine draft on the other hand...

[Steve W]

Quick note of clarification first off. There has been a HUGE discussion over how to determine QPs, when in reality you were discussing RPs. Qualification Points (QPs) were determined in 2006 by 2 points for a win, 1 for a tie, and 0 for a loss. RANKING Points (RPs) were the adverage score of the losing alliance in matches you participated in.
I don't really think this system is broken. I think it works very well, in fact. W/L/T should be the primary factor. RP serves both as a measure of "schedule strength" and GP.


You are correct with the term RP so I have edited all posts that I have posted wrong or was quoted. Thanks for the correction.

[KenWittlief]

We recongnize that rookie teams have a difficult challenge, starting from scratch and competing against experienced teams

and we also keep our eyes on previous regional and championship winners, expecting a lot from them

so instead of having a regional by regional 'toughness' rating system, we could have something like a handicap system (used in golf for example).

Your teams handicap rating would be based, not on the event you are attending, but on past performances.

They use a system like this in sailboat racing. Different boat designs can be raced in an open class event, because you are not really racing against the other boats, you are racing against the average or typical performance numbers for your boat. If you sail your boat very well that day, and someone with a faster design sails poorly, you can win the race, even though they crossed the finish line before you did.

Likewise, if three rookie teams are allied against three championship winners, the rookie teams would have a large handicap rating, and they could win the match if they play well, even if they finish with fewer points.

You could also have ratings based on the robot design. A purely defensive robot would have a different handicap rating than a robot designed to score points (because it is much more difficult to score points with a purely defensive design).

[Rick]

...we could have something like a handicap system (used in golf for example).

Your teams handicap rating would be based, not on the event you are attending, but on past performances.

Anyone who has played a sport with handicap like golf or bowling, will understand the term "sandbagging". Sandbagging is when you perform badly on purpose to gain an advantage later in the tournament or season. Handicap systems lend themselves easily to this kind of strategy.


Likewise, if three rookie teams are allied against three championship winners, the rookie teams would have a large handicap rating, and they could win the match if they play well, even if they finish with fewer points.

Rookies have to learn to compete somehow. How can you compute how much a team from 1996 will score compared to a team from 2006?


You could also have ratings based on the robot design. A purely defensive robot would have a different handicap rating than a robot designed to score points (because it is much more difficult to score points with a purely defensive design).

The best rules in FIRST are rules that are black and white. What kind of commitee decides what is defensive and what is offensive? This is definatley a gray area. FIRST is great because it lets kids make ideas into reality! We have rules for robot safety and identification, size and wieght, we don't need rules telling us what to build (with the exception of bumpers which are great for so many reasons).

[Cory]

The system works. No need for change. If a team doesn't end up a good position, that means they weren't good enough.

This may be true. But the reverse happens all the time. Teams who do end up in a good position who aren't good enough.

[KenWittlief]

Anyone who has played a sport with handicap like golf or bowling, will understand the term "sandbagging". Sandbagging is when you perform badly on purpose to gain an advantage later in the tournament or season. Handicap systems lend themselves easily to this kind of strategy.
....

I believe this can be generalized as poor-sportmanship. No matter what the rules and scoring algorythm is, its always possible to play the rules, instead of playing the game.

In some games it becomes the default, like football teams snapping the ball and taking a knee when they are way ahead, instead of playing the game and trying to score.

Sportmanship is important, no matter how the scores are tallyed.

[groves]

I had the chance to see the 1189 (gearheads) Gearhead Crossing (http://blarg.moonoo.com/crossing/) Scouting system a couple weeks ago at the FORD FIRST invitational. I thought it was a neat way to rank teams individually. The system has a fantasy robotics feel to it. I think it would be neat to be able to rank robots on the multi-national level all throughout the season. The major paradigm shift would be that defensive robots would tend to score low (because they do not score in the conventional method). Six or officials would be in charge of watching all of the matches each watching a robot. Filling out a performance report each match for your robot.

[KenWittlief]

The best rules in FIRST are rules that are black and white. What kind of commitee decides what is defensive and what is offensive? This is definatley a gray area. ...

If the rules were black and white the manual would be one page long.

I think it would be fairly easy to classify a robot. Using last years game as an example: if your robot has a mechanism that can launch the ball into the air, it is a centergoal shooter.

If it has a way to capture, hold or store balls, and dump them into the side goals, it is a dumper.

If all your robot can do at best if push balls around with a flat surface, with no captive edges on the 'plow', then it is purely defensive.

[Jack Jones]

After reading through some incredibly bad ideas - mine included - I think we should leave well enough alone.

[artdutra04]

I got it! How about an algorithim that allocates for cross space-time continuum flucuations as a function of seismic activity in China, over the squareroot of the temperature in Kelvin of the ambient temperature of the playing field plus two times the fourth devivitive of two times pi times one over x to the nth power, where n is the number of years that the team has existed, and x is the team number. To this, we subtract four times the third root of the loser's score while adding one fourth of the winner's score. To account for experience differences, we divide this by the combined age of the alliance drive team times minus the number of matches they have played. From all of this, we add this onto unix epoch and divide by the combined gracious professionalism quotient of the participating team, to get the new SUCK ranking score - Scoring that's Uberly Complicated Karma. :rolleyes:


While discussing hypothetical changes to the rules is never a bad idea, let's not make everything more complicated than it needs be. Some things in life are best left plain and simple, like vanilla ice cream, and the FIRST scoring system should be one of them.

Life is not fair, and Dean always reminds us of that every chance he gets. Sure rookies and even veteran teams may get shafted occasionally by the random matches or the scoring algorithim, but that's life. Sometimes in life you will get the short end of the stick, no matter how much you try to prevent it. What better way is there to learn life lessons than in FIRST?

[Lil' Lavery]

If the rules were black and white the manual would be one page long.

I think it would be fairly easy to classify a robot. Using last years game as an example: if your robot has a mechanism that can launch the ball into the air, it is a centergoal shooter.

If it has a way to capture, hold or store balls, and dump them into the side goals, it is a dumper.

If all your robot can do at best if push balls around with a flat surface, with no captive edges on the 'plow', then it is purely defensive.
That isn't true. 116 HAD a shooter, yet we never used it. 56 used their shooter, yet they still typically would score as many or more points in the corner goals. 1139 had a shooter, yet only scored 6 (or maybe 9) total points in two matches on archimedes, as they spent almost all their time defending. 48 used their shooter for a majority of the Championship event, yet during the eliminations, they replaced it with a hunk of metal and become a defensive bot. Welcome to the fuzzy grey area.
And regarding an earlier point about rookies you made. While a majority of rookies are not extremely competative, there are several notable exceptions each year. 1731 built a fully functional camera-guided turret and shooter this year, finished as the #1 seed at the largest regional in America (VCU), and if 414 doesn't DQ their alliance in SF1-2, then they would have advanced AT LEAST to the VCU Finals. 1902 was the #1 seed at Lonestar, a finalist at UCF, and a finalist on Archimedes. 1816 won Midwest and was the #6 alliance captain on Archimedes. Etc.

[KenWittlief]

That isn't true. ....

The purpose of a handicap/rating system isnt to makes everyones final scores all come out exactly even

the purpose would be to take a step towards leveling the playing field.

On average rookie teams do not field very competitive robots. If a rookie team does expectionally well, they deserve to win.

If a team won a regional last year, we expect them to do even better this year, and be a championship contender.

If you put a shooter on your robot, and you do a poor job on the design, so it cannot score many points, then you will be one of the poorly designed robots in the shooter class, and your score will reflect that.

Having a handicap system will allow HS-only teams with little funding and no mentors to focus on a simple defensive robot, and at least have some chance of making it to the finals.

Look at it this way: you are seated in the top 8, and you need to pick two robots. You want one shooter and one defensive bot. With the handicap system you have more incentive to pick a good purely defensive bot, than to pick a team that tried to do everything and does nothing well, just to play defense. You would have more incentive to pick a rookie team, if that means your alliance has a better handicap rating.

Leveled playing field.

[Tom Bottiglieri]

This may be true. But the reverse happens all the time. Teams who do end up in a good position who aren't good enough.
That's life?
Yeah, sure it would be great if we could have some greater being rank and order teams in magnitude of "how good they really are," but when was the last time a national championship was won by someone who didn't deserve it? Remember, strategy and luck are part of the game just as robot design is.

[Lil' Lavery]

The purpose of a handicap/rating system isnt to makes everyones final scores all come out exactly even

the purpose would be to take a step towards leveling the playing field.

On average rookie teams do not field very competitive robots. If a rookie team does expectionally well, they deserve to win.
And a veteran with an even better robot doesn't?

If a team won a regional last year, we expect them to do even better this year, and be a championship contender.

If you put a shooter on your robot, and you do a poor job on the design, so it cannot score many points, then you will be one of the poorly designed robots in the shooter class, and your score will reflect that.
But what about a team like 56 in 2006? They had a well above average shooter, but they decided to play as a "hybrid" and both shoot and dump. What "category" would they fall under? Or a team like 522 who had an excellent shooter, but played even better defense and ended up playing primarily defense? Because defense typically only requires a strong drivetrain, moderately sized frame, and a robust robot, it is often very easy to combine with any design, which creates problems with your system.

Having a handicap system will allow HS-only teams with little funding and no mentors to focus on a simple defensive robot, and at least have some chance of making it to the finals.
1139 made it to Einstein, that was the first award they ever won. 1731 made one of the best rookie bots ever in a barn. The playing field is plenty level. Many of the most creative and dangerous bots come out of these lower budget teams.


Look at it this way: you are seated in the top 8, and you need to pick two robots. You want one shooter and one defensive bot. With the handicap system you have more incentive to pick a good purely defensive bot, than to pick a team that tried to do everything and does nothing well, just to play defense. You would have more incentive to pick a rookie team, if that means your alliance has a better handicap rating.

Leveled playing field.
Or pick a robot that doesn't even move to get a ton of extra points....

[Rick]

EVERYTHING Lil' Lavery has been posting

Lil' lavery and tom, you guys are right on. I was in class and could not defend myself...

But think about it this way Ken. This might be too logical for some people.

Your on a rookie team that just defeated 71 or 111 or 469 or (other great team with a history of doing well). How great would you feel if you won that in a fair match? Pretty good huh. But what if you win since you get X amount of handicap. You would like the trophy, but wouldn't feel like you earned it.

You make a robot, you analyze the weaknesses, you make it better. Adding handicap to the mix would make it so innovation and change would stop. Teams with subpar robots just think "handicap will take care of our faulty X". You strive to be better every year. Goals are usually to improve on the knowledge from last year. What's next, everyone gets the same amount of trophies?

If your robot is not performing as well as other teams, instead of adding to scores, teams should realize they need to innovate and change, not default to a ranking system.

WIth handicap teams have no reason to improve since it would not matter. This is the basis of my argument.

[KenWittlief]

And a veteran with an even better robot doesn't?...

part of the idea of a handicap system is that you are playing against yourself, your teams past performance, just as much as you are playing against other teams.

If veteran team has done very well in the past, and they dont do even better this year, then they are sitting on their lauraels. Thats not what engineering is about.

If a team trys to design a robot that does everything, and they dont have the experience and resources to pull it off, then they have made a bad engineering decision.

Science and engineering - thats what we need to keep the focus on. If Im working alone as an engineer, and I decide Im going to put Microsoft out of business by writing a better operating system, then I have set my sites on an unreachable goal. No matter how good of an engineer I am I will never be able to take down Microsoft by myself.

But I might design a simpler product, or take on a smaller project, and create something that is pure genius in design and execution. Who would you give an engineering award to? A company that turns out a so-so operating system year after year

or an individual who creates something that is perfect in its function and form and design for its intended purpose?

I know we already have engineering design awards, and quality awards, but with some type of team handicap system we could open up the competition itself and make the contest more level for all the teams.

WIth handicap teams have no reason to improve since it would not matter.

maybe I need to create an example of how a handicap system would work. Your handicap would be based on your actual scores in last years events. The handicap would be based on the average of all the teams in your class (rookie, intermediate, veteran...) and the level you chose to play at (defensive, offensive, auton modes...). If you are a rookie team this year, your handicap would be based on the average of all the rookie teams last year.

a handicap would bring the average team in each class up to an equal footing for this year. That means, if you were average last year, and you are average this year, you will rank somewhere around 50% at an event. If you were below average your handicap would be higher, but if you play below average this year you will not win.

But if you were average last year, and you go home with a drive for improvement, and your performace this year is twice as good as before, you have a good shot at ranking high in the competition.

[Steve W]

What happened to the algorithms. I thought that this was going to be an idea thread not what it has become.

[Tom Bottiglieri]

What happened to the algorithms. I thought that this was going to be an idea thread not what it has become.
Let X be equal to how teams have always been ranked. Let Y be equal to their new ranking.

Y = X

:)

[Heretic121]

Im gonna go soccer/footbal style...

Win's - Lose record = most important... win = 3 pts, 2(or 1) for a tie... 0 for a loss...

Ties are determined by the average points your that team has scored over thier qualifaction matches. Highest average = 1st place... if they happen to be tied... use either points scored against that alliance or head to head wins/loses...

easy... simple... not much change...

no more QP/RP stuff...

[Lil' Lavery]

part of the idea of a handicap system is that you are playing against yourself, your teams past performance, just as much as you are playing against other teams.

If veteran team has done very well in the past, and they dont do even better this year, then they are sitting on their lauraels. Thats not what engineering is about.

If a team trys to design a robot that does everything, and they dont have the experience and resources to pull it off, then they have made a bad engineering decision.

Science and engineering - thats what we need to keep the focus on. If Im working alone as an engineer, and I decide Im going to put Microsoft out of business by writing a better operating system, then I have set my sites on an unreachable goal. No matter how good of an engineer I am I will never be able to take down Microsoft by myself.

But I might design a simpler product, or take on a smaller project, and create something that is pure genius in design and execution. Who would you give an engineering award to? A company that turns out a so-so operating system year after year

or an individual who creates something that is perfect in its function and form and design for its intended purpose?

I know we already have engineering design awards, and quality awards, but with some type of team handicap system we could open up the competition itself and make the contest more level for all the teams.

I don't beleive there is a FIRST team that doesn't already strive to get better and better each year though. Sometimes you don't accomplish that, and that's part of life. It's quite hard to follow up a Regional win with a Division win, and a Division win with a Championship win, etc. That's why you see so few teams do it, which is why the few that can are so special. The fact is, you HAVE to be better to stay at the same level of success in FIRST without a system to force it. Because FIRST is growing, in order to win the same awards and get the same ranking etc, you have to beat more teams, which means being even better. A system like this isn't "leveling the playing field", it's rigging it against the historically successful teams.


I don't particularly think that the current system is broken. If I was going to change it, the only thing I'd do is add some more tie-breakers (I never like "coin flip" scenarios, as are always used as the eventual tie-breakers in every major sport and FIRST). Maybe Head-to-head record, or average points scored as additional tie-breakers after RP and Max Score. Although, I don't beleive that anyone has ever been tied after Max Score anyway...

[Andrew Blair]

Not to 'jack this thread, but the scoring system is quite fair, simple, and well practiced. What causes far more upsets and hard feelings is, as was said before, the alliance algorithm.

Our team has seen both ends of the problem. We've had phenomenal matching with 90% of matches with at least one strong robot, placed high accordingly, but perhaps not deserved the position.

Inversly, we've had regionals where every match seems like a disadvantage- and often is.

We've also been in one small regional where we played with 433 three matches in a row, and yet only played in matches- allied or against- with 3/4 of the teams there. If alliances are distributed evenly, consistently, a team will recieve a much more accurate ranking. There should be no "free rides" or teams who must serve as the primary machine every match.

[KenWittlief]

I don't believe there is a FIRST team that doesn't already strive to get better and better each year though. ...

A system like this isn't "leveling the playing field", it's rigging it against the historically successful teams.


I don't particularly think that the current system is broken. ...

have you ever been on a team, or worked with a team that had six HS students and only one teacher for a mentor? A team that had nothing but a couple handtools, and ended up building their robot out of PVC pipes and plywood?

Do they have any chance against teams with $100k in funding, an huge corporate sponsor, and ten engineer mentors? Seriously? No. At best they end up trying not to be in the bottom 5 of the ranking at a regional.

We keep telling students that engineering is all about what happens between your ears, how well you following the design cycle / process, how well you think things through

but the competition is already rigged, its rigged in favor of the big veteran teams that have lots of resources

that doenst mean that little 7 person team didnt try just as hard as everyone else. They may have pulled 48 hour weekends, and worked all year to raise funds. They may have been excellent in their design process, but you can only do so much with PVC and plywood against teams with CNC machines, a full scale machine shop, and a team of mentors at your disposal.

Go back to my example of sailing. I may never be able to buy an F40 catamaran (40 foot hulls), but I do have a 16 foot hobie cat. If I can sail my boat to its absolute limit, then why shouldnt I be able to compete in an open class event, against the 40 foot cats. I will never be able to sail faster then they can, but that doenst mean I am not the best sailer.

Sailing commitees came up with a rating /handicap system that allows exactly that - open class competition, where each skipper has a chance to win an event if they are the best sailor, with the boat they have.

We already have open class competition, but the way things are now the little guy has no real chance.

So why cant we come up with a scoring / ranking system for events in which all teams have some hope of placing well? Is it too hard for us to figure out?

What causes far more upsets and hard feelings is, as was said before, the alliance algorithm.

Our team has seen both ends of the problem. We've had phenomenal matching with 90% of matches with at least one strong robot, placed high accordingly, but perhaps not deserved the position.

Inversly, we've had regionals where every match seems like a disadvantage- and often is...

this is another problem that a well devised handicap system would fix. If you are allied with a small team that historically has not done well, or with a small rookie team, they would have a higher handicap rating, that would tend to even out the match.

[Tom Bottiglieri]

have you ever been on a team, or worked with a team that had six HS students and only one teacher for a mentor? A team that had nothing but a couple handtools, and ended up building their robot out of PVC pipes and plywood?

Yes.

Do they have any chance against teams with $100k in funding, an huge corporate sponsor, and ten engineer mentors? Seriously? No. At best they end up trying not to be in the bottom 5 of the ranking at a regional.

If you call the #4 alliance captain at the NJ regional "in the bottom 5," then yes.

Maybe the big powerhouse teams are inspiration to everyone?! I know I sure was motivated to do better after seeing 45, 217, 111, and 25 in 2003 at nationals. Would that have happened with the communist ranking system? No.

[KenWittlief]

Yes.

If you call the #4 alliance captain at the NJ regional "in the bottom 5," then yes.
...

if a team accomplished that then they are truely exceptional.

If you average the rankings of all the very small teams with limited resources, I dont think that winning a top 8 seat will be anywhere close to typical.

Inspiration is a separate issue. We are discussing ways to possibly make the scoring and ranking system better.

For years we have all accepted that FIRST is not fair. Big teams have a clear advantage. Are we unwilling to even consider a way to level the playing field?

What is everyone afraid of?

[Tom Bottiglieri]

For years we have all accepted that FIRST is not fair. Big teams have a clear advantage. Are we unwilling to even consider a way to level the playing field?

What is everyone afraid of?
Its not so much that we dont want things to change or feel that it would be bad, we just dont see a point in it. Yes, some teams are successful from year to year, but the high school kids are constantly changing. How can you explain to new team members that they just lost a match on the book which they rightfully won? Why is right to "take from the rich and give to the poor"? Some teams are much better, but FIRST needs that. Even if there was a handicap, some teams would find ways to exploit it. So no, I don't think there should be a handicap, nor should we consider it.

[Cory]

That's life?
Yeah, sure it would be great if we could have some greater being rank and order teams in magnitude of "how good they really are," but when was the last time a national championship was won by someone who didn't deserve it? Remember, strategy and luck are part of the game just as robot design is.

The rankings don't have anything to do with this. The issue is number of matches

The real question here should be "How do we get teams more matches at nationals?" (Most regionals already get enough matches to have the cream of the crop at the top). There's absolutely nothing wrong with the ranking system. If it aint broke don't fix it. Another more worthwhile issue is improving the random match generator, cuz it's never worked so well.

[Tom Bottiglieri]

The rankings don't have anything to do with this. The issue is number of matches

The real question here should be "How do we get teams more matches at nationals?" (Most regionals already get enough matches to have the cream of the crop at the top). There's absolutely nothing wrong with the ranking system. If it aint broke don't fix it. Another more worthwhile issue is improving the random match generator, cuz it's never worked so well.
I totally agree with both of those point. And no I'm not being a hypocrite! :cool:

[efoote868]

perhaps these arguements would be under a different category like "year to year" team rankings, and shouldn't make much of a difference to the "here and now", competition rankings. Please don't take this the wrong way, but I wouldn't want to see a completely uneven match in any of the qualifying rounds, because of a robot that isn't exceptional, put there because of its special circumstances.

Besides, the rating system does what it is supposed to do, and that is to seperate the robots from "best" to "worst". further more, if that doesn't do the trick, then the alliance selection does.

oh, high school students only stick around for 4ish years, so the "powerhouse teams" are really just the mentors

[Rick]

Maybe the big powerhouse teams are inspiration to everyone?! I know I sure was motivated to do better after seeing 45, 217, 111, and 25 in 2003 at nationals. Would that have happened with the communist ranking system? No.

Tom your the man.

But anyways...
The competition is the end of the journey. The journey is where the learning happens, and seeing others at competition inspire innovation for others and new thinking. Keep it simple, lets have some fun, and lets not make competition stressful with complicated ranking.

[Donut]

oh, high school students only stick around for 4ish years, so the "powerhouse teams" are really just the mentors

This would be the killer to any kind of handicap system, to me at least. If you're going to tell me I'm going to be penalized because last year's group of kids did really well (and since our team tends to have a rather large student and teacher turnover every year or 2, we wouldn't have any of the "same resources"), I would personally not want to compete that year.

We all know the competitions are not about winning it. There is, what, three awards based on competitions results? How many are there that aren't? I'm personally very happy to get ANY kind of award, so I don't see a reason for making the "victory" award any easier to get. If I win it yet I didn't have to perform nearly as well as *insert-really-good-team-here*, I wouldn't feel like I've earned it much.

[JaneYoung]

oh, high school students only stick around for 4ish years, so the "powerhouse teams" are really just the mentors
Apologies in advance for going off topic -

The powerhouse teams have many things going for them -
stability can be a key factor regarding location for build, mentors that commit their time and energy, and students who develop an organized system of training and being trained in an on-going manner. Off seasons are excellent tools for developing in all the areas the teams need to prepare for competition season. I do agree that the mentors on the powerhouse teams are important - they are incredible leaders. They are also wise ones in that they understand the process of developing new leadership, new growth, new ideas in order to continue to improve as teams competing in FIRST. Remarkable things can occur in 4 years - and with thought and preparation, beyond.

[MikeDubreuil]

If I were a game designer…

My goal would be to have great offensive matches. I would award winning teams. I also think teams should be rewarded for consistently scoring well despite some lost matches.

I would keep the Ranking Points system mostly intact:
2 points for a Win
1 point for a Tie
0 points for a Loss

Your RP score would be summed the entire day and divided by matches played. This is an important scoring metric because at the end of the day you must be able to beat your opponents.

You would receive an “RP Ranking” based solely on your RP score against the rest of the field. Ties would be allowed.

There would be a second ranking list of teams sorted by average match score. This is simply a running average of their average match score. Ties would be allowed.

Finally, I would average a team’s rank in the RP Ranking list with their rank in the Average Match Score list to create there Overall Rank.


RP Rank Team Name RP Score
1 Team M 8
2 Team B 6
3 Team D 2
4 Team Z 0



MS Rank Team Name Match Score
1 Team M 53
2 Team B 47
3 Team Z 45
4 Team D 20


MS Rank Team Name
1 Team M
2 Team B
3 Team Z
4 Team D


As you can see from the "Overall Rank" Team Z benefited most from my method. They scored high points but just couldn't seem to win matches. This probably had something to do with not so great alliance partners. Team D did not benefit at all. Their low match score points and decent RP points indicate that they were probably carried by their alliance partners.

I think this way is exciting because each match you're not just playing your opposing alliance you're against the whole regional.

EDIT: Tie breakers for computing overall rank are by priority:average match score, # of penalties, electronic coin flip.

[Daniel_LaFleur]

if a team accomplished that then they are truely exceptional.

If you average the rankings of all the very small teams with limited resources, I dont think that winning a top 8 seat will be anywhere close to typical.

Inspiration is a separate issue. We are discussing ways to possibly make the scoring and ranking system better.

For years we have all accepted that FIRST is not fair. Big teams have a clear advantage. Are we unwilling to even consider a way to level the playing field?

What is everyone afraid of?

Level the playing field??? That is nothing but a euphemism for telling someone that they are not good enough so we're going to give you some points.

I have been on a small, underfunded, under mentored team for the last 3 years. I am now switching to another small, underfunded team. Do I want your leveled playing field? NO! It would be an empty victory if I won by less than the "bonus" points I got because I wasn't "good enough".

FIRST already has a level playing field. There are no advantages on the court, except what you have built into your robot. And this is how it sould be.

What is needed is for the veteran teams to reach out to the newer teams and offer the support that the newer teams need (now, before the regionals). Most new teams don't know that there is support out there, or where to ask. THIS is where FIRST should work to 'level the playing field', not on the court.

I'm sorry Ken, but while I feel your heart is in the right place, your conclusion is wrong.

The above is, as always, IMHO.

[Nawaid Ladak]

hey guys, it's been a LONG time seince ive been on this website.

but yea, now to the algerathums

the point system should work like this.

as a ranking system

ie

Match 1
Team A: 12
Team B: 13
Team C: 9
Bonus Points: 10
Red Total Score: 44

Team D: 15
Team E: 18
Team F: 3
Bonus Points: 5
Blue Total Score: 41

(one robot for the blue, and two robots for the red made it up to the ramp in time)

Match 2

Team A: 33
Team C: 6
Team E: 12
Bonus Points: 10
Red Total Score: 61

Team B: 20
Team D: 3
Team F: 12
Bonus Points: 25
Blue Total Score: 60

(All three robots for blue made it up to the ramp, while two robots made it up to the ramp in time.

HERE WE GO.

Team A scores 33+12+5pts(ramp Match 1)+5pts(ramp Match 2)=55pts

Team A's alliance total score was 105 so...
55/105 x100= 52.381

Team A's Opponants total score was 101 so...
52.381/101 x100= 51.862

Total Rating: 51.862

Team B scores 13+20+5(ramp Match 1)+8.334(ramp Match 2)=46.334pts

Team B's alliance total score was 104 so...
46.334/104 x100= 44.552

Team B's Opponants average score was 102 so...
44.552/102 x100= 43.678

Total Ranking: 43.678

Team C scores 9+6=15pts

Team C's alliance total score was 105 so...
15/105 x100= 14.286

Team C's opponants total score was 101 so...
14.286/101 x100= 14.145

Total Ranking 14.145

Team D scores 15+3+8.334(ramp Match 2)=26.334pts

Team D's alliance total score was 101 so...
26.334/101 x100= 26.073

Team D's Opponents total score was 105 so...
26.073/105 x100= 24.831

Total Ranking 24.831

Team E scores 18+12+5(ramp, Match 2)=35

Team E's alliance total score was 102 so...
35/102 x100= 34.314

Team E's Opponants total score was 104 so...
34.314/104 x1000= 32.994

Total Ranking 32.994

Team F scores 3+12+5(ramp, Match 1)+8.334(ramp, Match 2)= 28.334

Team F's alliance total score was 101 so...
28.334/101 x100= 28.053

Team F's Opponents total score was 105 so...
28.053/105 x100= 26.717

Total Ranking: 26.717

Standings
1 Team A (2-0-0) 51.862
2 Team B (1-1-0) 43.678
3 Team E (1-1-0) 32.994
4 Team F (0-2-0) 26.717
5 Team D (0-2-0) 24.831
6 Team C (2-0-0) 14.145

How do you guys like that

[Donut]

Point based ranking systems are very dangerous to have. Any kind of system that is based off of how much you score typically promotes "running up the score" on your opponents, and with your system it would also promote inter-alliance competition (because you would be ranked higher if the rest of your alliance scored less). This is why most high school sports leave margin of victory out of ranking systems, because it promotes the teams winning 65-0 to run the score up to 100 instead of putting in their 3rd string and having some kind of mercy.

I'd hope this wouldn't be a problem in FIRST with GP and whatnot it supports, but I could still see this brewing into an issue.

[Cory]

Point based ranking systems are very dangerous to have. Any kind of system that is based off of how much you score typically promotes "running up the score" on your opponents, and with your system it would also promote inter-alliance competition (because you would be ranked higher if the rest of your alliance scored less). This is why most high school sports leave margin of victory out of ranking systems, because it promotes the teams winning 65-0 to run the score up to 100 instead of putting in their 3rd string and having some kind of mercy.

I'd hope this wouldn't be a problem in FIRST with GP and whatnot it supports, but I could still see this brewing into an issue.

Not to mention the fact that it'd be impossible to keep track of, unless the scoring is not real-time, and the scoring items are color coded by team.

[Donut]

One interesting thing with Freedom's ranking method: it would encourage teams to let other teams score at least a little. If you shutout an opponent, the second part of the ranking equation (involving the other team's score) would result in a Team 229 error, and so you'd be penalized and given a ranking score of 0 for that match.

[Nawaid Ladak]

You are right. but if you add it into multiple matches, then it doesn't mean anything, if there only was a way to modify my system to strenghen my system. if i could think of something, maybe you guys can ehlp, PM me if you have any ideas

this system would also help to rank teams OVERALL as well if FIRST would give us that information, CD could have their own Top 25 (Just like in College Football.)

This would also tell us which regionals are weak in strength (or strong in Defense.)

Speaking of Defense, this system really messes them up as well. lets work together to make this work.

Thanks

[Alan Anderson]

Define "better". What do you want to reward? Only after you decide what behavior you like can you figure out how to make that behavior rank high.

In a competition, it's obvious to me that winning the game is the primary goal. Ranking must be based on that, and breaking ties among teams with the same win/loss record ought to be based on the ranking of the teams you played against. The problem with that is the need to continuously reassess rankings as all the other teams play their matches.

The present scheme uses the losing alliance's score as a proxy for the strength of the alliance. It ends up rewarding offensive play over defensive. That seems reasonable to me.

[Joe Johnson]

Wild idea.

Premise:
People are better at ranking teams than any system that purely looks at the numbers (for example, I would trust Bill Beatty's or Karthik's ranking of teams as closer to "reality" than anything we could come up with based strictly on the numbers)

Problem:
Individual ranking are not available and may be destroyed by the knowledge of the use of their ranking (for example, Bill Beatty & Karthik may not willingly share their ranking with us and even if they would, the problem is that they are humans and cannot isolate their rankings from influence by the knowledge that they hold the ranking of other teams in their hands.)

Proposal:
Get people an incentive to share their honest opinion of teams' relative strength by having them predict the outcomes of matches (for example, win/loss & final score), use predictions to infer an underlying ranking system, weight each individual predictor's ranking system by its success rate, over time, the weighting system would lean heavier and heavier on the better predictor's predictions.

How to actually implement this? It is going to take a lot of money ;-)

1) Establish a significant team scholarship at each competition (say $5,000) that is won by the team that makes the best predictions of match outcomes.
2) At each regional, set up a wifi system where a each team gets to register computer to make their predictions up to X minutes before the start of the match.
3) The team that is the best predictor (some TBD scheme that combines the win/loss/score predictions) gets an award at the Regional and college scholarship cash for one of their team members.
4) Hire an professor of math/probability to do 2 things:




Develop a real time tool to infer from the predictions what the most likely underlying rankings of the predictors are (To give a simple example, if I consistently predict that alliances with Team X will win by a higher margin than I predict for similar alliances with Team Y, then I must think that Team X is better than Team Y)
Develop a real time tool that weights the predictions from good predictors higher than those from lousy predictors -- i.e. this is the rabbit out of the hat: The Ranking System that incorporates the knowledge of the Bill Beattys and the Karthiks of the FIRST community
5) The system only applies to the seeding rounds.
6) Teams can compete as a predictor in any/all regionals, even those they are not competing in.
7) Perhaps only predictors of quality X are allowed to predict at the Championships.
8) Perhaps the best Y predictors at the Championships (the best from each Div?) get to a "pre game show" before the Finals on Einstein handicapping how it is all going to go down.


There are a lot of reasons to hate this system, but there are some reasons to like it too. Corruption is one of the possible down sides beyond money. We'd have to make sure that teams are not taking a dive to improve their prediction ability.

As I said, I think it would take a lot of money to make it happen, but I think that such a system could make a better ranking system while at the same time make the matches more exciting for all the teams, especially those that are trying to predict each outcome.

Joe J.

[Donut]

As you noted, defensive play is at a major disadvantage in your ranking system (in fact the you can win every match and be ranked in last place due to points being the decider). If you want to use a system that uses points, you must take into account the margin of victory, since this is the only way a defensive style of play can rank well in a point based system. Of course this automatically means you can't use it for any kind of "official" system either, because it will promote blow-outs, as I said before.

I think a system that uses a combination of your record, your opponents' and alliance partners' records, and your margin of victory in matches is probably the best factors you could use for a true ranking system. Any other data (such as how much each individual robot scored or received in penalties) becomes hard to keep track of, and things like "how well they play defense" are opinions and shouldn't be trusted in an actual ranking system (BCS take note). How much of a factor each of these things should be is the really hard thing to decide (your record should be the most important thing, but how important and what order after that is where everything gets skewed).

Now, this would probably make for the best "true" ranking system, with no promotion of any style of play or type of robot. Probably a good scouting tool; but once again not something FIRST should be using.

One note: although record should be the most important factor in a rank, I don't believe it should be the end-all-be-all for your rank. It is possible for a 7-3 team to be better than a 9-1 team, so ranking by record and then using other things for tie-breakers would not be advised in a true ranking system.

Was the point of this thread initially to design a new system for FIRST to use, or just the best true system possible? I'm assuming the latter from the first post, but I don't really know what we're looking for here.

[Tom Bottiglieri]

While I don't agree that rookies should get freebies, I do agree the current ranking system could be refined as it sometimes places GOOD teams lower than they should be based off the fact they play most of their matches with rookies.

Here is my solution.

1) Each team has their match points averaged across the event.
2) Each match is picked apart.. Teams receive a ranking score based on the points scored in each match in comparison to their alliance partners average score.
Ranking Score for a Match of Team A = Score Of Alliance - [(Alliance Partners 1 Avg + Alliance Partner 2 Avg) / 2]
3) Ranking scores are summed across all matches. Ranking scores may be positive or negative. Mean of data should be 0. Standard deviation tells "how good" regional actually was.

For example:
-Teams A, B, and C are allianced together.
-Team B has an average of 10 points per match.
-Team C has an average of 20 points per match
-The alliance of A, B, and C score a total of 60 points in their match together.
-Therefore, we can make the assumption A was the benefactor to this alliance, an A gets the Ranking Score of :
RS = 60 - [(10 + 20) / 2] = 45

This is the method I based my nationals scouting data off of. I ranked each team in their division with this algorithm. The results were VERY close to what the actual standings were! Check it out.. http://www.chiefdelphi.com/media/papers/1830

[efoote868]

What if (biggest what if) instead of the ranking system being changed, the "random sort" was changed. Instead of having a "truely random" sort for matches, the powerhouse teams were paired (well, moreso than they are now) with the underfunded, unmentored, rookie teams. If we kept the ranking system the same, then this would definitely "jumble" the results... I would like to see how hammond, wild stang, and the pink team would then fair.
(besides, if anyone complained about this, then are they really understanding what F.I.R.S.T. is?)

This would not only jumble the results, but then the teams surfacing to the top 8 are more likely to be mediocre teams, or the truely exceptional ones, that can beat the bias.

Then scouting would also be a bigger deal...

[Nawaid Ladak]

What about this...

I know they do this in my fantasy football leauge.

you set up a rankiing system.. simular to this

Teams A-F play in one match

Team A 1-9
Team B 7-3
Team C 4-6

Team D 5-5
Team E 10-0
Team F 2-8

Team A-C are playing a allince with a combined record of 17-13

Part A of the rankings is based on your opponents records.
Part B is based on how many matches you won
Part C is based on How Many Points You can Rack up
Part D can be used for margin of victory (somehow: the closer the better.)

Now that would be a great idea

Combine scores form 1-10 in each part, and rate teams on a 40 point system.

Therefore, a team playing Defense can create some noise in the rankings insted of being dead last.

And we could use it for a system to create a Top 25 involving first teams.

imagine
1: 254 (your USC)... any more ideas...