Open Challenge: Make a better FRC ranking algorithm

[Billfred]

Originally Posted by Donut

There are huge flaws in applying this to a system with multiple team alliances and opponents though, and I think I've figured out a system but am not sure of how accurate it would really be.

What flaws would there be? The only one that comes to mind (and I haven't tested it yet) is that everyone would have the same opponent-win percentage if everyone played everyone else once at a regional--but when does that happen?

[Tom Bottiglieri]

The system works. No need for change. If a team doesn't end up a good position, that means they weren't good enough.

[ewankoff]

possibly use a combined record to gauge an alliances record to fit into the above power-point system

[Donut]

Originally Posted by Billfred

What flaws would there be? The only one that comes to mind (and I haven't tested it yet) is that everyone would have the same opponent-win percentage if everyone played everyone else once at a regional--but when does that happen?

I meant just using a straight copy of it would be flawed. If instead you factor in each opponents' wins and also each of your alliance partners' losses, you could make a better system (the idea then would be a win over 3 winless opponents with 2 undefeated partners would be equal to a loss to 3 undefeated opponents with 2 winless partners).

The bigger problem is I'm not sure how accurate it would be in a system where your opponents can also be your partners, as the original system wasn't designed for this type of scenario.

I like the current system (especially when compared to the old one from 2003 and earlier), and whether it should be changed really depends on what FIRST wants to promote in matches and design.

If they want high scoring matches, they should keep the current system, which rewards an offensive strategy more than a defensive one.
If they want to account more for opponents' strength, they need a system like Billfred's or many high school sports.
If they want close games, they should probably go back to the old system or get one like Steve's.
If they want something else... well, adjust accordingly.

Regardless of how FIRST does rankings, I will still use something more like my idea when trying to figure out alliance selections, because when I'm doing that I care far more about how good they are performing than how close their matches have been and whether they're an offensive or defensive robot. FIRST would want to promote exciting matches, but from a success viewpoint I would not want to be part of one because that usually means one team barely wins a shootout or that they come from behind for a close victory.

[Imajie]

Originally Posted by Jherbie53

Another way for QP's would be take the losing teams points as QP's. Winner gets 2x the points, and 1pt off for every 5pts or 10pts (depends on the game) the winning alliance is higher than the losing team. This would also create close matches.

This year I saw some teams go and shoot balls in the higher goal of the their opponents, so they would have a higher QP. It makes sense, but you can really make some teams think that you think they can't do it. So you could subtract points from the teams that scores for the other. Example: Red scores 15pts on their own and Blue scores 50+pts in the match. Blue wants more QP's, and scores 15pts for Red making their total 30pts. All of the Red teams would get the 30pts and the Blue teams would get 15pts. This would mean that either Blue lets Red score more or they don't score for them. This sounds like its an OK solution but not a perfect one for stoping it.

Instead of not giving the winning alliance the points they scored, take those away from the points the other alliance actually scored. So, using your example the Blue alliance would get 0 points. Reds 15 - their 15 they scored for Red = 0. This would greatly discourage scoring for your opponent.

[Chris Marra]

The only issue I can see with the current system is when one team on an alliance is liable for the loss of the alliance. All penalties, and especsially back-bot penalties, are applied to the alliance, even though its possible one team was designed responsible for being backbot before the match. In this event, a team might lose a close match by another drive team's mistake, and then two teams are penalized for one teams actions, and that 2-point sway on a record can make a big difference at a regional with few qualifiers, or at Nationals.

Otherwise, the present system works fine and there is no reason for a change. A team's record demonstrates typically demonstrates how well and their robot having been performing at a competition.

[Lil' Lavery]

Quick note of clarification first off. There has been a HUGE discussion over how to determine QPs, when in reality you were discussing RPs. Qualification Points (QPs) were determined in 2006 by 2 points for a win, 1 for a tie, and 0 for a loss. RANKING Points (RPs) were the adverage score of the losing alliance in matches you participated in.
I don't really think this system is broken. I think it works very well, in fact. W/L/T should be the primary factor. RP serves both as a measure of "schedule strength" and GP.
The problem with using your opponent's W/L/T as a Strength of Schedule is that it doesn't rank how strong they were in the particular match you faced them in, but rather how strong they were over the entire competition. For example, say Team 9999 has a 9-1 record, but you faced them in their only loss, when they had a malfunction and their robot never moved. With a "power ranking" system, it would reward you for beating a team that was a top notch team, but in reality you beat a team that never even moved. With the current RP system, you would likley receive a lower RP because they didn't actually score points. Another situation may be when you face excellent teams that don't work well together as an alliance. Like a combination of 3 herders (where you could simply block the corner goals all match), or (in 2004) two cappers with not so hot human players.
In short, if you wanted to modify the system to have more emphasis on schedule strength, you have to use a system that uses strength in the matches you competed in, not overall competition strength.

[Donut]

Originally Posted by Lil' Lavery

Quick note of clarification first off. There has been a HUGE discussion over how to determine QPs, when in reality you were discussing RPs. Qualification Points (QPs) were determined in 2006 by 2 points for a win, 1 for a tie, and 0 for a loss. RANKING Points (RPs) were the adverage score of the losing alliance in matches you participated in.
I don't really think this system is broken. I think it works very well, in fact. W/L/T should be the primary factor. RP serves both as a measure of "schedule strength" and GP.
The problem with using your opponent's W/L/T as a Strength of Schedule is that it doesn't rank how strong they were in the particular match you faced them in, but rather how strong they were over the entire competition. For example, say Team 9999 has a 9-1 record, but you faced them in their only loss, when they had a malfunction and their robot never moved. With a "power ranking" system, it would reward you for beating a team that was a top notch team, but in reality you beat a team that never even moved. With the current RP system, you would likley receive a lower RP because they didn't actually score points. Another situation may be when you face excellent teams that don't work well together as an alliance. Like a combination of 3 herders (where you could simply block the corner goals all match), or (in 2004) two cappers with not so hot human players.
In short, if you wanted to modify the system to have more emphasis on schedule strength, you have to use a system that uses strength in the matches you competed in, not overall competition strength.

The herder scenario would be another problem with the power-point rankings. They weren't designed for alliance competitions, and so probably wouldn't be as effective.

The current system has little problems. It does discourage defense to an extent; but this keeps teams from putting the same robot on the field year after year that is designed simply to push people around and get in their way. Since the FIRST rankings only determine who are alliance captains, not who is ultimately in the playoffs, they should be designed to promote what FIRST wants us to do, and it should be up to the teams to figure out their way to determine who are the best teams to pick.

Since this thread is about designing a better system for truly ranking teams, I will continue to develop my idea, though I hope FIRST won't actually use it.

[Rick]

Using a system where the rank is decided by the difference of the winner and loser score is horrible from a strategy point of view. Any ranking system in which a team needs to "hold back" scoring to seed higher will not be received well after the current system. Wins and loses need to continue to be a factor for ranking.

A team that wins all their matches should be number 1. Two teams that win all thier matches are decided by the higher RP MEANING they have played the "tougher" opponents. Tougher meaning the opponents scored more points.

This system has been succesful since 2004. Compared to 2003's ranking system its night and day. Lets keep this simple. We want to make this easy for spectators right? Gracious professional refers to making your grandmother proud. If I can't explain the game and its ranking to my grandmother, then its too complicated.

[Noah Kleinberg]

Originally Posted by Ricksta121

We want to make this easy for spectators right? Gracious professional refers to making your grandmother proud. If I can't explain the game and its ranking to my grandmother, then its too complicated.

Exactly what I thought when I read this thread...

The system now works well, and is easily understood by spectators. They shouldn't have to do any math, or try to think of how the other robots have been doing throughout the day in order to know who "won" the match in terms of ranking.

Now the serpentine draft on the other hand...

[Steve W]

Originally Posted by Lil' Lavery

Quick note of clarification first off. There has been a HUGE discussion over how to determine QPs, when in reality you were discussing RPs. Qualification Points (QPs) were determined in 2006 by 2 points for a win, 1 for a tie, and 0 for a loss. RANKING Points (RPs) were the adverage score of the losing alliance in matches you participated in.
I don't really think this system is broken. I think it works very well, in fact. W/L/T should be the primary factor. RP serves both as a measure of "schedule strength" and GP.

You are correct with the term RP so I have edited all posts that I have posted wrong or was quoted. Thanks for the correction.

[KenWittlief]

We recongnize that rookie teams have a difficult challenge, starting from scratch and competing against experienced teams

and we also keep our eyes on previous regional and championship winners, expecting a lot from them

so instead of having a regional by regional 'toughness' rating system, we could have something like a handicap system (used in golf for example).

Your teams handicap rating would be based, not on the event you are attending, but on past performances.

They use a system like this in sailboat racing. Different boat designs can be raced in an open class event, because you are not really racing against the other boats, you are racing against the average or typical performance numbers for your boat. If you sail your boat very well that day, and someone with a faster design sails poorly, you can win the race, even though they crossed the finish line before you did.

Likewise, if three rookie teams are allied against three championship winners, the rookie teams would have a large handicap rating, and they could win the match if they play well, even if they finish with fewer points.

You could also have ratings based on the robot design. A purely defensive robot would have a different handicap rating than a robot designed to score points (because it is much more difficult to score points with a purely defensive design).

[Rick]

Originally Posted by KenWittlief

...we could have something like a handicap system (used in golf for example).

Your teams handicap rating would be based, not on the event you are attending, but on past performances.

Anyone who has played a sport with handicap like golf or bowling, will understand the term "sandbagging". Sandbagging is when you perform badly on purpose to gain an advantage later in the tournament or season. Handicap systems lend themselves easily to this kind of strategy.

Originally Posted by KenWittlief

Likewise, if three rookie teams are allied against three championship winners, the rookie teams would have a large handicap rating, and they could win the match if they play well, even if they finish with fewer points.

Rookies have to learn to compete somehow. How can you compute how much a team from 1996 will score compared to a team from 2006?

Originally Posted by KenWittlief

You could also have ratings based on the robot design. A purely defensive robot would have a different handicap rating than a robot designed to score points (because it is much more difficult to score points with a purely defensive design).

The best rules in FIRST are rules that are black and white. What kind of commitee decides what is defensive and what is offensive? This is definatley a gray area. FIRST is great because it lets kids make ideas into reality! We have rules for robot safety and identification, size and wieght, we don't need rules telling us what to build (with the exception of bumpers which are great for so many reasons).

[Cory]

Originally Posted by Tom Bottiglieri

The system works. No need for change. If a team doesn't end up a good position, that means they weren't good enough.

This may be true. But the reverse happens all the time. Teams who do end up in a good position who aren't good enough.

[KenWittlief]

Originally Posted by Ricksta121

Anyone who has played a sport with handicap like golf or bowling, will understand the term "sandbagging". Sandbagging is when you perform badly on purpose to gain an advantage later in the tournament or season. Handicap systems lend themselves easily to this kind of strategy.
....

I believe this can be generalized as poor-sportmanship. No matter what the rules and scoring algorythm is, its always possible to play the rules, instead of playing the game.

In some games it becomes the default, like football teams snapping the ball and taking a knee when they are way ahead, instead of playing the game and trying to score.

Sportmanship is important, no matter how the scores are tallyed.