Worst Scoring System in Years

[EddieMcD]

Leon Machado did this spreadsheet attached of the BAE regional. A few things I noticed off the bat:

• RPs are still necessary for a higher seed. The catch is, this year you actually have to win rounds.
  
• On the topic of RPs, the ninth seed had the highest, 17th had second, 32nd was third, and the 2 seed was fourth. So again, you need high QPs. Imagine the shakeup that would've happened had we used a scoring system based on the RPs.
  
• Most of the higher seeds pulled lower RP scores. In fact, seeds 1, 2, 3, 4, 5, 6, 7, & 8 in RPs placed 5, 4, 21, 9, 16, 25, 30, & 44, respectively.
  
• This is the first year in the Alliance Era QPs were based on Win/Loss. If they had been based on previous formulas, we'd be seeing teams shooting for the opposing goals, and maybe less fighting (for lack of a better word) on the platform. Take that any way you want. We'd also see the #9 seed placed first. Again, take that as you will.

[Aidan F. Browne]

I think you might be missing the big picture...

Who ever said that the scoring system was designed to be fair? Who ever said that its purpose was to identify the best robot? Perhaps the scoring system each year pefectly meets FIRST's purpose...

Team Update 15 stated it in black-and-white: "Remember, our organization is not one that centers on winning a robotics competition, but rather one that is focused on transforming lives."



Aidan

[MisterX]

I think this years set up was nice. It awarded a good desin that was capable to win which was the main ranking tool. But then it had the opponents score as the second which gave an oppurtunity to show gracious profeesionalism in not beating a team exubriantly like removing a multiplier even if you knew you were winning

[piotrm]

Out of the few years I knew of FIRST, I think this year has the best ranking system (just MY opinion). The main reason has already been mentioned: it rewards both good offensive as well as defensive robots. This vastly increases the variety of strategies that teams have and makes the game more interesting.

[EvanG]

I also agree that the system is fine because its based on wins and not points.

If the system was based on points, that meant that the top seeded team must have allowed their opponents to score a lot of points, almost as many as them. While it means that the winning team scored a lot of points, it means they didn't play defense against the other team. I'm not saying that its not good that both teams scored a lot of points, I just feel that ranking by score removes extra variables that are necessary to make an educated guess about how a team performs.

[Eugene]

In my analysis I used the match points, not the ranking points (RP) since FIRST did not make the average match points available on their website, I actually had to calculate them myself.

Lastly, with exception of the few posts arguing that the newer scoring levels the playing field for offensive and defensive robots I did not see many other good arguments in support of the newer system. Most degenerated into repetitive “that is the way it is and deal with it” or “I just FEEL that it is good.” Please include some stats to support your arguments.

To clarify my argument once again, I dislike the current scoring system because it produces results that are far more random than the results produced by other system. Eight games just are not enough for this to give consistent results. (Speaking scientifically it is too small of a sample.) There are hundreds of little things that can sway the game. However, the team with the best strategy and the robot will consistently score better than a team with the worse robot and strategy.

To those of you nicely point out that I should shut up simply because the poll is stating that I’m a minority let me remind you of a few other minorities that changed the world.

Galileo and Copernicus – Changed the view of the solar system
Martin Luther – Forever changed Christianity
Mark Twain – One of the strongest speakers against slavery
Emerson – Writer of “Self-Reliance”


Eugene

[Marc P.]

Originally Posted by RoteAugen

why you anti-utilitarian you!

to paraphrase their main point:
"The wants of the many outweigh the wants of the few, or the one"

Oddly enough, I was watching Star Trek II, and that quote from Spock came up literally 5 minutes before I read that post...

But back on topic, I do very much like this year's scoring compared to previous years. In 2002 (Zone Zeal), during our team's first match at nationals, we had an incredible match. With 10 seconds left, the score was something like 54 to 45. With that 10 seconds left, the opposing alliance moved their goal full of balls out of the scoring zone. The resulting score for the match- 54 - 0.

Rank was determined by average qualification points. Qualification points were equal to three times the loosing alliance's score. Three times zero averaged into 7 matches doesn't work out very well, despite a very good match, and a relatively strong showing the other 6 matches. Needless to say, I didn't think it was too fair, but didn't mind. I had a great time at nationals that year, and accepted it as part of the game.

This year, the primary method of rank based on wins is a tremendous breath of fresh air to anyone who ailed under the previous ranking systems. The inclusion of the Ranking Points system further increases the fairness in the ranks, by including the all important factor of specific performance in addition to a simple win/loss.

Originally Posted by Eugene

To clarify my argument once again, I dislike the current scoring system because it produces results that are far more random than the results produced by other system. Eight games just are not enough for this to give consistent results. (Speaking scientifically it is too small of a sample.) There are hundreds of little things that can sway the game. However, the team with the best strategy and the robot will consistently score better than a team with the worse robot and strategy.

What it comes down to is this: robots that do well deserve to win. The argument is how well they win. In a game focused more on strategy (as inferred by the quote from the manual earlier in the discussion), does a robot not deserve to be ranked by merit of win/loss than to points? Does a robot who hangs on the bar deserve to be ranked higher than a ball gatherer/capper? Each potentially yield different point values yet will succeed if well built and capable. In the current system, robots who do well win, and rank accordingly. However, as you say, 8 matches is not enough to accurately rank based on that alone. Hence the ranking point system. To quote the FIRST manual-

All four teams will receive a number of Ranking Points equal to the Match Score of the
losing alliance or their alliance score in the case of a tie.

In order to gain qualification points, a team must win the match, implying they have a score greater than the loosing alliance. However, in the spirit of FIRST, shutouts are discouraged, and will reflect accordingly in the rankings. Therein lies what I believe to be the primary concept behind the current scoring system: Robots must be capable of controlling the situation of each match to the point where they win by a minimalistic margin while attaining the highest possible score. So essentially, a quarter of what you wish to be in the scoring system already is, in that score has something to do with it. But, in all fairness and the interest of FIRST, it's not the winners score which is important, but the loosing alliance's, which adds a very interesting dynamic to the way the game is played. It makes the competition that much more exciting, particularly from a strategy perspective.

[Mike Schroeder]

While i have never had the pleasure of meeting him in person, i would have to say a pretty wise man said last year :

Originally Posted by Paul Copioli

A stack is a stack is a stack.
-Paul

and to morph that into a term we can use this year, "A WIN IS A WIN IS A WIN"

i think Dean Kamen said it best at kick off STOP BEING LAWYERS

the scoring is one of the most simple yet, you win, you win you loose, you loose. the tie breaker is your match scores

Competitor Friendly: Yes
Spectator Friendly: Yes
Gives us a new way to strategize: Yes
Gave us a headache at NJ as we tried to figure out who was ranked where: Yes

to quote another wise man: Thank you FIRST this really is the best yet

i wouldnt have found that quote if it werent for the spotlight system thanks brandon

[Gabriel]

I agree with the original poster, this years scoring system was a step backwards.

First, the stated goal was to make the system simpler, they succeeded in making it more complex. Now, instead of having to keep track of one number (avg. opponents score) you have to keep track of two (# of wins and avg opponents score). Also, winning is important but so is the "cooperation" aspect, I think that last years scoring system balanced both of them quite well. Unfortunately, this years system didn't - there was too much emphasis on winning at all costs, if you were in the middle of the pack winning or losing a game would cause you to jump a whole tier (maybe as many as 10 slots) so it wasn't worth the risk of losing by a small margin. Last year, losing by a small margin wasn't bad at all so there was more of an incentive to try for really interesting matches with high scores on both sides.

Here's an example of how the scoring system seems to punish losing a little too much: I remember that one of the last matches at UTC was between four excellent teams (181, 782, 195 and 236 IIRC). I think that 181 and 782 won by a score of something like 115 to 110 - a margin of 1 ball and as a result 195 dropped from 6th to about 12th and didn't make the finals. Last year a high losing score at the end wouldn't have hurt nearly as much.

I don't know, maybe this system will grow on me but I think its overly convoluted and a step backwards.

[Billfred]

Originally Posted by nuggetsyl

It is going to be something to say but i love this years scoring system. It simple and lets the stragties stay simlar in the finals. Not saying it the best first can do but it is a huge step foward.

shaun

Actually, I'd imagine that the strategies would shift somewhat. In qualifying, you want to win with your opponent having a high score (to get the win and the QPs). However, as eliminations come, QPs go out the window and the ability to win (within the rules and GP, of course) becomes more important. While it still isn't cool to utterly crush a team, you still want to come out ahead.

Maybe the folks that have been to regionals can prove me wrong, but that's how I'm reading the tourney setup.

[MikeDubreuil]

Originally Posted by Eugene

Lastly, with exception of the few posts arguing that the newer scoring levels the playing field for offensive and defensive robots I did not see many other good arguments in support of the newer system. Most degenerated into repetitive “that is the way it is and deal with it” or “I just FEEL that it is good.” Please include some stats to support your arguments.

To clarify my argument once again, I dislike the current scoring system because it produces results that are far more random than the results produced by other system. Eight games just are not enough for this to give consistent results. (Speaking scientifically it is too small of a sample.) There are hundreds of little things that can sway the game. However, the team with the best strategy and the robot will consistently score better than a team with the worse robot and strategy.

I'm not sure anyone should address your complaints unless you reccomend a better scoring system.
Most of your issue is with the randomness that results in playing a low number of seeding matches (8). That's not going to change any time soon- there's just not enough time in the day.

[jpsaul7usa]

Originally Posted by Cory

In previous years, a team could win every single match, but not score a ton of points, but some not so good robots would get paired with real good teams, and then theyd get a ton of points, when they didnt really deserve them, or just score lots of points but not win. Guess who ended up on top? Team #2. Now THAT is unfair.

Cory

I agree with Cory. Last year was my team's rookie year and our drive system was absolutely horrible (trust me, I built it and drove the bot). We were incredibly lucky during the qualifying rounds at had excellent allies and placed 5th out of 37 teams at the AZ Regionals by the time the elimation rounds began. I want to call it a fluke because we hardly could do anything on the field because our bot could barely drive anywhere. We never even made it on top of the ramp or stacked boxes when we needed to. In my opinion we coasted by last year because of some lucky rounds (77-76 point win) in which our allies almost completely won by themselves. When it came time for finals we got creamed by out higher-quality opponents.

This year, our bot is so much better than last. Even though it still doesn't look like much, it gets the job done. For the first two qualifying rounds our winch was malfunctioning, but after it was fixed we won every single round following in the qualifying rounds and only lost one match in the elimination rounds. Some wins were big, some were small. I think this year's scoring system is much more fair because our team made it to the 4th position (of 36) at the beginning of elimination rounds and I believe we deserved it.

I'm also a huge baseball fan, and in baseball scores don't matter, wins do. Like in the 2001 World Series, when the Diamond Backs (Go DBacks!) went up against the Yankees: In the first 6 games the DBacks outscored the Yankees 32-9, but despite this huge difference it still came down to the bottom of the ninth inning of game 7 to determine the winner. It was much more exciting to see who won than who scored the most runs of the series.

As in baseball, some robots cam win with good defense in low scoring games. Some bots loose with a great offense in high scoring games. By counting wins and not points, a greater variety of strategies are allowed and a greater variety of robots have a chance to go onto elimination rounds. I think this year's scoring system, simpler than last year's, was great, hands down.

[Chris Hibner]

I want to make a simple point here:

Comparing the actual rankings from this year to some other ranking system is pointless and not at all accurate. Why? Because teams play with a strategy that will earn them their highest possible ranking in the current system. In other words, they're not playing to rank highly in your hypothetical system, they're playing to rank highly according to wins and losses.

Therefore, whatever results you calculate will be highly skewed by the fact that teams are playing toward a particular ranking system. Recaculating in another system is simply inaccurate and doesn't prove anything.

[Eugene]

Chris, you definitely have a valid point that I did not consider. At least some one finally said something insightful.

I decided to make this post because I was rather surprised by the fact that my team ranked much higher than I would have expected by my prior experience, so I decided to analyze the situation. The day before, we were in the high 20s. The day after, we placed 14th. The first match of the second day we were paired against a really weak opponent and we beat them 35 to 30. The following match we were paired with an extremely strong opponent and we won again. This propelled us some 13 ranking places. I’m not complaining about our luck but after doing some statistical analysis I noticed that the current scoring system is far susceptible to be influenced by the factors outside of the team’s control than by their prior preparation.

To all who seem to be so crazy about winning, let me once again remind you that we are not about that. It is far better to have a great game were both sides end up benefiting in their rankings that a lousy game in which one side benefits tremendously. The current system, as Chris correctly pointed out encourages wining at all costs. Whatever Ranking Points (you opponent’s scores) you have accumulated in reality make very little difference in your ranking compared to wins and loses.

What I did like however is that in certain cases this system allowed for more creative teams by using strategies that did not score many points. If we could use some combination of this year’s system (to allow for more creative game play) and last year’s system (to eliminate randomness and shear luck) I think we would have a better scoring system.

Eugene

[jpsaul7usa]

Originally Posted by Eugene

If we could use some combination of this year’s system (to allow for more creative game play) and last year’s system (to eliminate randomness and shear luck) I think we would have a better scoring system.

Eugene

I think sheer luck will always be a gigantic part of the competition. If random and unexpected events never happened then a match wouldn't be all that exciting, i.e. team A always gets the ball release, always scores 5 balls in a basket, and always makes it onto the bar with 20 seconds left, while team B, their opponent, does the same except only gets 4 balls, meaning the team B wins before the match even starts. Boring. There is definitely a range between completely random and completely calculated (best antonym I can come up with for random) matches that is optimum, and I currently think it's at the right level.

However, I am puzzled this year why there are fewer matches. The field takes less time to reset (I think), at my regional there were fewer teams than last year, and the matches are the same length. Why are there fewer matches? Down to 9 from 11? Not to mention the even more limited number of teams you do play in those matches. If it's because there are more teams overall and there aren't enough new competitions to meet the increased demand then what's going to happen in a few years as the growth rate continues to increase and there isn't enough money to have more regionals? Sorry to get a bit off topic but in the future if the number of matches continues to decrease, teams will be less able to operate their bots effectively and random occurences will have a much greater impact than they do now. If randomness is to be lessoned then there need to be more matches. Perhaps longer days?