Open Challenge: Make a better FRC ranking algorithm

[Billfred]

I've been involved with FIRST for three seasons now, and with the exception of some oddities in the Hatch software forcing a team update to fit it midway through 2005, there hasn't really been much change to how teams are ranked. Robot sizes have changed, games have changed, weight has changed, point systems have changed, technology has evolved, but at least for as long as I've been around, it's been QPs and RPs equal to the losing score.

So just for the fun of it, how would you rank teams if you ruled the world? Any method is fair game, but bonus points to those that are plausible enough that FIRST could actually use them. And assume the game will be similar in format to Aim High, to keep things simple. (I'll post mine later.)

[ewankoff]

winning team gets their own score if the margin of victory is less than 1/3 of the losing team's score. if not they get the losing teams score.

losing team gets either the margin or their own score whichever is higher.

i think this sounds good. this way blowouts favor the losing team and winning teams will try and keep the margins low.

edit:this is added to the normal ranking by record and is used only to break ties for teams with the same record.

[KenWittlief]

I think FIRST should be more like the Olympics

teams are scored not only on technical aspects of the game (whether a ball is placed in a goal....)

but also on style and artistic expression!

At the end of each match the 10 judges along the sideline would hold up cards, with their scores for both the winning and the losing alliance.

This would eliminate the need to base scores on the losing alliance, because, while you can help your opponent fix a mechanical or electrical part before a match, there is no way you can teach a clutzy robot artistic expression in 1 day!

[Jack Jones]

Originally Posted by KenWittlief

I think FIRST should be more like the Olympics

teams are scored not only on technical aspects of the game (whether a ball is placed in a goal....)

but also on style and artistic expression!

At the end of each match the 10 judges along the sideline would hold up cards, with their scores for both the winning and the losing alliance.

This would eliminate the need to base scores on the losing alliance, because, while you can help your opponent fix a mechanical or electrical part before a match, there is no way you can teach a clutzy robot artistic expression in 1 day!

Yuck! Judges? No thanks!

I think the rankings are fine as they are. However, I think the top eight should stay that way. That is, no building powerhouse aliances by picking within the seeded teams. Scouting would be much more important than it is now.

I realize that it may encourage the seventh or eighth seed to sandbag their last match in order to finish ninth and thereby have a chance to get selected by a top seed, but they also run the risk of not getting picked at all.

Maybe this, and having a 1..8 then 8..1 selection order, would make for better elimination matches than the blowouts we get now.

[Billfred]

Originally Posted by Jack Jones

Yuck! Judges? No thanks!

I think the rankings are fine as they are. However, I think the top eight should stay that way. That is, no building powerhouse aliances by picking within the seeded teams. Scouting would be much more important than it is now.

I realize that it may encourage the seventh or eighth seed to sandbag their last match in order to finish ninth and thereby have a chance to get selected by a top seed, but they also run the risk of not getting picked at all.

Maybe this, and having a 1..8 then 8..1 selection order, would make for better elimination matches than the blowouts we get now.

I'm not so quick to block picking within the top 8. At Duel, 25's first pick was MOE, the #9 seed. I'm not saying MOE was sandbagging by any stretch--25 just made the best decision they could under the rules. I'd imagine that you'd see a lot of that under a restricted picking system. (And besides, the other teams in the top 8 can always decline.)

If you ask me, I think the best way to go about it would be to go by QPs, then the average QPs of your opponents over the qualifying rounds. If you're beating other highly-seeded teams, which are usually harder opponents, you'll do better than if you're beating teams that are 0-8. Furthermore, it encourages teams to do everything they can to help the very teams they'll be going against--you want your opponents to win every match (except the ones you're in, naturally).

[Jack Jones]

Originally Posted by Billfred

.... (And besides, the other teams in the top 8 can always decline.)

I'm glad you brought that up. Because, it sometimes causes hard feelings. It's as hard to decline as it is to be snubbed. We've all seen times when a team seeds high due to the luck of the draw wrt/ alliance partners. Then we see team after team decline them. Now suppose they could only pick from number nine down, and the team they selected had sandbagged their way there. I call that justice.

[Donut]

Originally Posted by Billfred

If you ask me, I think the best way to go about it would be to go by QPs, then the average QPs of your opponents over the qualifying rounds. If you're beating other highly-seeded teams, which are usually harder opponents, you'll do better than if you're beating teams that are 0-8. Furthermore, it encourages teams to do everything they can to help the very teams they'll be going against--you want your opponents to win every match (except the ones you're in, naturally).

If FIRST ever wants a system that factors in the difficulty of your opponents, like this, then you can copy many of the sports ranking systems.

Arizona Football does a "power-point" system for seeding, with your team receiving 50 points for every win and 5 points for each of your opponent's wins. You divide all the points from your wins by the total games you played.

They play a 10 game schedule, and the theory behind the system is that beating a team with no wins (resulting in 50 points for your win) should be equivalent to losing to an undefeated team (resulting in 50 points for their 10 wins).

There are huge flaws in applying this to a system with multiple team alliances and opponents though, and I think I've figured out a system but am not sure of how accurate it would really be.

[Billfred]

Originally Posted by Donut

There are huge flaws in applying this to a system with multiple team alliances and opponents though, and I think I've figured out a system but am not sure of how accurate it would really be.

What flaws would there be? The only one that comes to mind (and I haven't tested it yet) is that everyone would have the same opponent-win percentage if everyone played everyone else once at a regional--but when does that happen?

[Tom Bottiglieri]

The system works. No need for change. If a team doesn't end up a good position, that means they weren't good enough.

[Cory]

Originally Posted by Tom Bottiglieri

The system works. No need for change. If a team doesn't end up a good position, that means they weren't good enough.

This may be true. But the reverse happens all the time. Teams who do end up in a good position who aren't good enough.

[Tom Bottiglieri]

Originally Posted by Cory

This may be true. But the reverse happens all the time. Teams who do end up in a good position who aren't good enough.

That's life?
Yeah, sure it would be great if we could have some greater being rank and order teams in magnitude of "how good they really are," but when was the last time a national championship was won by someone who didn't deserve it? Remember, strategy and luck are part of the game just as robot design is.

[Cory]

Originally Posted by Tom Bottiglieri

That's life?
Yeah, sure it would be great if we could have some greater being rank and order teams in magnitude of "how good they really are," but when was the last time a national championship was won by someone who didn't deserve it? Remember, strategy and luck are part of the game just as robot design is.

The rankings don't have anything to do with this. The issue is number of matches

The real question here should be "How do we get teams more matches at nationals?" (Most regionals already get enough matches to have the cream of the crop at the top). There's absolutely nothing wrong with the ranking system. If it aint broke don't fix it. Another more worthwhile issue is improving the random match generator, cuz it's never worked so well.

[ewankoff]

possibly use a combined record to gauge an alliances record to fit into the above power-point system

[Donut]

Originally Posted by Billfred

What flaws would there be? The only one that comes to mind (and I haven't tested it yet) is that everyone would have the same opponent-win percentage if everyone played everyone else once at a regional--but when does that happen?

I meant just using a straight copy of it would be flawed. If instead you factor in each opponents' wins and also each of your alliance partners' losses, you could make a better system (the idea then would be a win over 3 winless opponents with 2 undefeated partners would be equal to a loss to 3 undefeated opponents with 2 winless partners).

The bigger problem is I'm not sure how accurate it would be in a system where your opponents can also be your partners, as the original system wasn't designed for this type of scenario.

I like the current system (especially when compared to the old one from 2003 and earlier), and whether it should be changed really depends on what FIRST wants to promote in matches and design.

If they want high scoring matches, they should keep the current system, which rewards an offensive strategy more than a defensive one.
If they want to account more for opponents' strength, they need a system like Billfred's or many high school sports.
If they want close games, they should probably go back to the old system or get one like Steve's.
If they want something else... well, adjust accordingly.

Regardless of how FIRST does rankings, I will still use something more like my idea when trying to figure out alliance selections, because when I'm doing that I care far more about how good they are performing than how close their matches have been and whether they're an offensive or defensive robot. FIRST would want to promote exciting matches, but from a success viewpoint I would not want to be part of one because that usually means one team barely wins a shootout or that they come from behind for a close victory.

[Steve W]

Leave as is with win/loss but change the way RP points are tabulated. If you take the losers score, subtract from the winners score, take the remainder and subtract from 100 to give RP. The losing alliance gets 2/3 of the RP. Example:

Red 52 Blue 41
52 - 41 = 11
100 - 11 = 89 RP winning alliance
89/3 x 2 = 60 RP losing alliance

Red 52 Blue 10
52 - 10 = 42
100 - 42 = 58 RP winning alliance
58/3 x 2 = 38 RP losing alliance

Red 89 Blue 0
89 - 0 = 89
100 - 89 = 11 RP winning alliance
11/3 x 2 = 8 RP losing alliance

Red 98 Blue 97
98 - 97 = 1
100 - 1 = 99 RP winning alliance
99/3 x 2 = 66 RP losing alliance

Red 10 Blue 9
works out to the same as above

There would be a max amount of RP per game. This encourages close games which are exciting. Not sure of how to handle ties at this point but maybe 50 RPs for a tie. The closer the score, the higher the RPs for both winning and losing. Would also make strategy much tougher.