Win-Loss, Average, RP?

[Kartoffee]

Just wondering, since we've seen there different scoring systems in the last three games, what seems like the best system that rewards commitment, yet still encourages underfunded teams?

[Kevin Leonard]

I really loved the 2016 ranking system. It made it such that even if your alliance was over matched, you still had goals to try to achieve during a match. It also created a ton of interesting strategic choices about how to play each match.

[Jay O'Donnell]

I agree that 2016 was a great ranking system in that you always had a way to keep your ranking from completely tanking, but still needed to win matches to rank the highest. It took away defense from Quals more than it would've been otherwise, but it was for strategic benefit. It created a slight difference from Quals to playoffs which I enjoyed.

[logank013]

Statistically speaking, I think 2015 was the most accurate. No matter what ranking system you have, having a good schedule can skew your rank but having the averages were great for ranking because no defense means no low score matches due to the other alliance (except at worlds where can burglars decided the whole match). Now nobody wants to really play with no defense again, but I think that the 2015 ranking system is the most accurate for most non-defense games in the future (please, don't do another non-defense game...).

[TheBoulderite]

I actually started going through regionals/championship divisions in 2015 to see how teams would have been ranked if they had used the 2014 ranking system. I'll post a link once I'm done with the spreadsheet.

[Caleb Sykes]

Originally Posted by Kartoffee

Just wondering, since we've seen there different scoring systems in the last three games, what seems like the best system that rewards commitment, yet still encourages underfunded teams?

Rank in 2015 was much more highly correlated with ability than in any other year, and I value this correlation pretty highly with any ranking system.

My problem with systems like those of 2012, 2015 or 2016 is that the game becomes fundamentally different in the playoff matches from what it was in qualification matches. I don't think it is fair that abilities which are important to ranking high (see, making coopertition stacks) will not necessarily be as useful in the playoffs, and that an ability that is an invaluable asset in the playoffs (see, fast canburglars) can be essentially useless during qualifications. This forces teams to make a choice, either explicitly or implicitly, between focusing on seeding high or contributing to the maximum in the playoffs. That is just silly, other sports' playoff matches are essentially the same as qualification matches, just with better teams, let's have that again please, like we did in 2013 and 2014.

[Chris is me]

2016 was the best system to play under. The three methods of getting RP made the game more interesting and always gave an alliance a reason to try.

2010 produced the most accurate results, but I think the issues many had with the ranking system probably mean we'll never see anything like it again. I personally thought it was great as long as teams were willing to score in the proper goal...

[Rangel(kf7fdb)]

I really liked 2016 in that it rewarded an alliance for performing very well regardless if they won or lost but still made wins and losses important to the rankings. Though it might be an unpopular opinion, I absolutely hate when coopertition is a very important part of the ranking system. I think extra ranking for having good teamwork within your alliance should be rewarded vs teamwork with the opposing alliance(2016 vs 2012). Despite co-op trying to promote positive interactions with opponents, I feel as if it does the exact opposite. Especially when one alliance doesn't want to co-op for legitimate competitive reasons.

[EricH]

Originally Posted by Rangel(kf7fdb)

I really liked 2016 in that it rewarded an alliance for performing very well regardless if they won or lost but still made wins and losses important to the rankings. Though it might be an unpopular opinion, I absolutely hate when coopertition is a very important part of the ranking system. I think extra ranking for having good teamwork within your alliance should be rewarded vs teamwork with the opposing alliance(2016 vs 2012). Despite co-op trying to promote positive interactions with opponents, I feel as if it does the exact opposite. Especially when one alliance doesn't want to co-op for legitimate competitive reasons.

THIS.

2016 allowed you to get a "win" in rankings if you played well, regardless of whether you won or lost. 2012 required you to work with your opponent, and is the reason that every time I discuss the "rankings game", I make very clear that if a team makes an agreement with their opponents to do something, they need to KEEP it. There was quite a bit of "bad blood" that year, whether intentional or not I can't say. 2015 was OK... but required you to design to do it, and work with your opponent.

[asid61]

2016 was probably the best system IMO because it balanced multiple tasks, and the tasks that would net you RPs would also net a considerable number of points in eliminations, keeping you competitive (unlike 2015 as Caleb mentioned). The dynamic feel of the game also helped to keep Elims exciting even if the teams were not evenly matched.

[Knufire]

Note that the 2016 ranking system was very much tied to the 2016 game. If you want to apply the same ranking system to future games, you need a game with several different major activities. Applying the same concept to a similar game, such as 2014, might prove difficult (RP for assists, trusses, and winning?).

These are what I look for in a ranking system:
1. Produces rankings that correlate closely to individual robot ability.
2. Simple enough to explain to a general audience who may not be invested in FRC.
3. Universally applicable to different game types, such that FIRST could just use the same ranking system year after year.


Given these criteria, I am a very big fan of the 2015 ranking system. 2016 was definitely fun to participate in but not easy to explain to a more general audience.

I also agree with Caleb on "changing the game" between qualifications and eliminations; however, I see this as a game design issue and not a ranking system issue.

[Peyton Yeung]

And here I am still liking the 2010 system

[Caleb Sykes]

Originally Posted by Knufire

I also agree with Caleb on "changing the game" between qualifications and eliminations; however, I see this as a game design issue and not a ranking system issue.

I really have trouble imagining any ranking system that isn't WLT which won't cause qualification gameplay to differ from playoff gameplay. Take this year as an example, the ranking system discouraged many teams from playing defense in the quals matches where they would have if we had a WLT ranking system or if they were playing in the playoffs.

[Taylor]

2016>2012>2010>everything else>2015.

[Knufire]

Originally Posted by Caleb Sykes

I really have trouble imagining any ranking system that isn't WLT which won't cause qualification gameplay to differ from playoff gameplay. Take this year as an example, the ranking system discouraged many teams from playing defense in the quals matches where they would have if we had a WLT ranking system or if they were playing in the playoffs.

I think i see where you're coming from. Generally speaking, any ranking system that factors in performance relative to a standard outside the scope of that single match (i.e. bonus RP for secondary objectives, tournament average, etc) leaves room for teams in a given match to collude in order to maximize their own gains. I think this is a necessary evil, as purely looking at each match in a vacuum (which is basically what WLT does) leaves you more at the mercy of the schedule gods. I don't think this really happens enough in FRC to be a major concern, and with the 2016 manual, is explicitly discouraged by FiRST.

The differences between qualifications and playoff rounds that concern me were more related to game design, such as tasks being worth RP in quals and points in eliminations (2016), disappear completely in playoffs (co-op 2015, co-op 2012), or don't exist in the qualification rounds but are important to playoff rounds (triple-balancing 2012). These are almost all related to game design and not the ranking system. IMO, make everything worth points and let the ranking system do the work of filtering teams into rank.