Re: Open Challenge: Make a better FRC ranking algorithm
 

One interesting thing with Freedom's ranking method: it would encourage teams to let other teams score at least a little. If you shutout an opponent, the second part of the ranking equation (involving the other team's score) would result in a Team 229 error, and so you'd be penalized and given a ranking score of 0 for that match.

Re: Open Challenge: Make a better FRC ranking algorithm
 

You are right. but if you add it into multiple matches, then it doesn't mean anything, if there only was a way to modify my system to strenghen my system. if i could think of something, maybe you guys can ehlp, PM me if you have any ideas

this system would also help to rank teams OVERALL as well if FIRST would give us that information, CD could have their own Top 25 (Just like in College Football.)

This would also tell us which regionals are weak in strength (or strong in Defense.)

Speaking of Defense, this system really messes them up as well. lets work together to make this work.

Thanks

Re: Open Challenge: Make a better FRC ranking algorithm
 

Define "better". What do you want to reward? Only after you decide what behavior you like can you figure out how to make that behavior rank high.

In a competition, it's obvious to me that winning the game is the primary goal. Ranking must be based on that, and breaking ties among teams with the same win/loss record ought to be based on the ranking of the teams you played against. The problem with that is the need to continuously reassess rankings as all the other teams play their matches.

The present scheme uses the losing alliance's score as a proxy for the strength of the alliance. It ends up rewarding offensive play over defensive. That seems reasonable to me.

Re: Open Challenge: Make a better FRC ranking algorithm
 

Wild idea.

Premise:
People are better at ranking teams than any system that purely looks at the numbers (for example, I would trust Bill Beatty's or Karthik's ranking of teams as closer to "reality" than anything we could come up with based strictly on the numbers)

Problem:
Individual ranking are not available and may be destroyed by the knowledge of the use of their ranking (for example, Bill Beatty & Karthik may not willingly share their ranking with us and even if they would, the problem is that they are humans and cannot isolate their rankings from influence by the knowledge that they hold the ranking of other teams in their hands.)

Proposal:
Get people an incentive to share their honest opinion of teams' relative strength by having them predict the outcomes of matches (for example, win/loss & final score), use predictions to infer an underlying ranking system, weight each individual predictor's ranking system by its success rate, over time, the weighting system would lean heavier and heavier on the better predictor's predictions.

How to actually implement this? It is going to take a lot of money ;-)

1) Establish a significant team scholarship at each competition (say $5,000) that is won by the team that makes the best predictions of match outcomes.
2) At each regional, set up a wifi system where a each team gets to register computer to make their predictions up to X minutes before the start of the match.
3) The team that is the best predictor (some TBD scheme that combines the win/loss/score predictions) gets an award at the Regional and college scholarship cash for one of their team members.
4) Hire an professor of math/probability to do 2 things:

1. Develop a real time tool to infer from the predictions what the most likely underlying rankings of the predictors are (To give a simple example, if I consistently predict that alliances with Team X will win by a higher margin than I predict for similar alliances with Team Y, then I must think that Team X is better than Team Y)
2. Develop a real time tool that weights the predictions from good predictors higher than those from lousy predictors -- i.e. this is the rabbit out of the hat: The Ranking System that incorporates the knowledge of the Bill Beattys and the Karthiks of the FIRST community

5) The system only applies to the seeding rounds.
6) Teams can compete as a predictor in any/all regionals, even those they are not competing in.
7) Perhaps only predictors of quality X are allowed to predict at the Championships.
8) Perhaps the best Y predictors at the Championships (the best from each Div?) get to a "pre game show" before the Finals on Einstein handicapping how it is all going to go down.


There are a lot of reasons to hate this system, but there are some reasons to like it too. Corruption is one of the possible down sides beyond money. We'd have to make sure that teams are not taking a dive to improve their prediction ability.

As I said, I think it would take a lot of money to make it happen, but I think that such a system could make a better ranking system while at the same time make the matches more exciting for all the teams, especially those that are trying to predict each outcome.

Joe J.

Re: Open Challenge: Make a better FRC ranking algorithm
 

As you noted, defensive play is at a major disadvantage in your ranking system (in fact the you can win every match and be ranked in last place due to points being the decider). If you want to use a system that uses points, you must take into account the margin of victory, since this is the only way a defensive style of play can rank well in a point based system. Of course this automatically means you can't use it for any kind of "official" system either, because it will promote blow-outs, as I said before.

I think a system that uses a combination of your record, your opponents' and alliance partners' records, and your margin of victory in matches is probably the best factors you could use for a true ranking system. Any other data (such as how much each individual robot scored or received in penalties) becomes hard to keep track of, and things like "how well they play defense" are opinions and shouldn't be trusted in an actual ranking system (BCS take note). How much of a factor each of these things should be is the really hard thing to decide (your record should be the most important thing, but how important and what order after that is where everything gets skewed).

Now, this would probably make for the best "true" ranking system, with no promotion of any style of play or type of robot. Probably a good scouting tool; but once again not something FIRST should be using.

One note: although record should be the most important factor in a rank, I don't believe it should be the end-all-be-all for your rank. It is possible for a 7-3 team to be better than a 9-1 team, so ranking by record and then using other things for tie-breakers would not be advised in a true ranking system.

Was the point of this thread initially to design a new system for FIRST to use, or just the best true system possible? I'm assuming the latter from the first post, but I don't really know what we're looking for here.

Re: Open Challenge: Make a better FRC ranking algorithm
 

While I don't agree that rookies should get freebies, I do agree the current ranking system could be refined as it sometimes places GOOD teams lower than they should be based off the fact they play most of their matches with rookies.

Here is my solution.

1) Each team has their match points averaged across the event.
2) Each match is picked apart.. Teams receive a ranking score based on the points scored in each match in comparison to their alliance partners average score.

Ranking Score for a Match of Team A = Score Of Alliance - [(Alliance Partners 1 Avg + Alliance Partner 2 Avg) / 2]

3) Ranking scores are summed across all matches. Ranking scores may be positive or negative. Mean of data should be 0. Standard deviation tells "how good" regional actually was.

For example:
-Teams A, B, and C are allianced together.
-Team B has an average of 10 points per match.
-Team C has an average of 20 points per match
-The alliance of A, B, and C score a total of 60 points in their match together.
-Therefore, we can make the assumption A was the benefactor to this alliance, an A gets the Ranking Score of :
RS = 60 - [(10 + 20) / 2] = 45

This is the method I based my nationals scouting data off of. I ranked each team in their division with this algorithm. The results were VERY close to what the actual standings were! Check it out.. http://www.chiefdelphi.com/media/papers/1830

Re: Open Challenge: Make a better FRC ranking algorithm
 

What if (biggest what if) instead of the ranking system being changed, the "random sort" was changed. Instead of having a "truely random" sort for matches, the powerhouse teams were paired (well, moreso than they are now) with the underfunded, unmentored, rookie teams. If we kept the ranking system the same, then this would definitely "jumble" the results... I would like to see how hammond, wild stang, and the pink team would then fair.
(besides, if anyone complained about this, then are they really understanding what F.I.R.S.T. is?)

This would not only jumble the results, but then the teams surfacing to the top 8 are more likely to be mediocre teams, or the truely exceptional ones, that can beat the bias.

Then scouting would also be a bigger deal...

Re: Open Challenge: Make a better FRC ranking algorithm
 

What about this...

I know they do this in my fantasy football leauge.

you set up a rankiing system.. simular to this

Teams A-F play in one match

Team A 1-9
Team B 7-3
Team C 4-6

Team D 5-5
Team E 10-0
Team F 2-8

Team A-C are playing a allince with a combined record of 17-13

Part A of the rankings is based on your opponents records.
Part B is based on how many matches you won
Part C is based on How Many Points You can Rack up
Part D can be used for margin of victory (somehow: the closer the better.)

Now that would be a great idea

Combine scores form 1-10 in each part, and rate teams on a 40 point system.

Therefore, a team playing Defense can create some noise in the rankings insted of being dead last.

And we could use it for a system to create a Top 25 involving first teams.

imagine
1: 254 (your USC)... any more ideas...