[FRC Blog] "Standard District Points System for 2015"

[Christopher149]

Blog Date: Tuesday, February 3, 2015 - 15:01

As I’ve said a few times, the Standard District Points System will be made an official part of the FRC Manual this season, and in future seasons. It will come as a new Section 7 of the FRC Administrative Manual that deals broadly with advancement, not just for Districts, but for Regionals as well, covering such topics as Wild Cards. That new section is under final review. I apologize for the delay. In the meantime, folks are understandably anxious about what the points system will look like for this season, with our move away from Win/Loss/Tie in Recycle Rush, so I want to provide you with those details even though the manual section itself isn’t ready yet.

The updated system was developed by a team of two volunteer representatives each from all five Districts, along with some FRC staff members. We started work a few weeks before the game was released, so all volunteers from the Districts were asked to sign Non-Disclosure Agreements (NDAs). They never saw the full details of the game in advance of Kickoff. They knew that ranking would be based on average points, not Win/Loss/Tie, and that it would not be a low-scoring game, but knew very little else about the game itself. We asked them to sign NDAs because we considered even the fact that we would be basing rank on average points to be very confidential information.

The folks below were members of the team:

• Indiana District representatives – Chris Fultz, Liz Smith
• Mid Atlantic Robotics District representatives – Ben Ng, Ed Petrillo
• Michigan District representatives – Dan Kimura, Jim Zondag
• New England District representatives – Bruce Linton, Jamee Luce
• Pacific Northwest District representatives - Leo Conniff, Kevin Ross
• FRC Staff representatives – Lori Burkhamer, Danny Diaz, Miriam Somero, me

I want to thank everyone for their work on the team. This, in my mind, was a ‘dream team’ – everyone came to the table with an interest in doing the best job possible for the community as a whole, and our discussions were open, honest, friendly, and productive. Best of all, I think we ended up in a really good place! We recognized that the current points system works, and we were determined to make minimal changes in adapting it for a game not dependent on Win/Loss/Tie.

You can find the details of the 2015 FRC Standard District Points Ranking system here. Note that Michigan will be following a slightly modified points system at their Championship, because of the large number of teams they will be inviting. This modified system has not yet been finalized. FRC has agreed to Michigan’s verbal description of the system, but we are awaiting a written summary so we can review carefully before giving final approval. I would expect we’ll be able to release the system that will be in use at the Michigan District Championship within a few weeks.

The rather complicated-looking formula showing in the summary document was developed initially by FRC’s own Danny Diaz, FRC Systems Engineer, and modified with the support of the more mathematically-minded volunteers in the group. The intent was to come close to emulating the points distribution at events we had seen with the traditional Win/Loss/Tie system formulaically. Danny has put together a summary of how this formula works, along with representative points values for different size events, here.

Hope everyone is having a great build season so far!

[Ty Tremblay]

Here's the equation for input into Wolfram:

erf^-1((N-2(R)+2)/(1.07*N))*10/erf^-1(1/1.07)+12

[Lil' Lavery]

So much for something uniform and easy to understand.

[PVCpirate]

While this is pretty confusing at first glance(and second and third probably), for a given event size the point values are predetermined for each rank. So, the equation can be used to generate all the values for each event, and everyone can just look at a nice simple chart.

[dag0620]

It's a little confusing at first, but it's really not that complicated when you understand what is happening.

Kinda best way to simplify it down is it's like when a professor decides to grade on a bell curve. Same concept, just a little more complicated equation to work with the data parameters FRC requires.

[Zebra_Fact_Man]

First takeaway: Why was the QS capped at 22 pts instead of the previous 24? Also, why is the minimum 4 instead of 0?

Not that it matters, but it does strike me as odd.


So max pts this year is now 83 w/ Chairman's, 73 w/out at District events, instead of 85/75.

[artK]

Originally Posted by Zebra_Fact_Man

First takeaway: Why was the QS capped at 22 pts instead of the previous 24? Also, why is the minimum 4 instead of 0?

The answer to the first question is because, even in a WLT system, perfect 24s are pretty rare. In MAR, according to this, 23's have only been done twice, and in theory a 24 should have only happened once at this point. (Though it would be interesting to see what a difference a WLT would have on this game on the rankings). More matches per team make it harder to score a perfect record.

And regardless of the reason for the upper cap, the lower cap probably exists to keep the average score at 12. If the average score from qualifying becomes less than 12, teams that win judged awards, eliminations, or are a new to gain a slightly greater advantage in their 5 or 10 points, and re-adjusting the point values in that respect would make the system even more complicated.

[Ty Tremblay]

Here's a spreadsheet that uses an approximation method of InvERF(). It's not 100%, kind of a hack actually, so let me know if you have any improvements.

[Rachel Lim]

This Wolfram Alpha Widget: http://www.wolframalpha.com/widgets/...2fb3bb85e071b4 should calculate it if I put in all the equations properly (I'm not sure if I completely understand how the district ranking works, so there may be errors). It doesn't require any other programs to run (I know some students who don't have Excel so I decided not to try and figure out the equations in it). Please let me know if you catch any problems in it and I'll try to fix them.

[MasterMentor]

Originally Posted by Ty Tremblay

Here's a spreadsheet that uses an approximation method of InvERF(). It's not 100%, kind of a hack actually, so let me know if you have any improvements.

Originally Posted by Rachel Lim

Please let me know if you catch any problems in it and I'll try to fix them.

You both forgot the Ceiling Function.

[Thad House]

So I just did this for us, but it seems like this would be a small net increase for the teams at the top of the rankings. We went 11-1-0, and 10-2-0, and 10-2-0, with a 1 seed, 1 seed, and 3 seed. So we got 22, 20, and 60, for a total 102. With this structure, we would have gotten 22, 22, and 63, for a total of 107. So if you were constantly at the top seeds, this most likely will be a net increase in points, especially at tougher events, where the 1 seed might only have had 10 wins. However, the top teams are not usually the ones where 5 points makes a difference, so it most likely would not have much of an actual effect on the top.

[MikeE]

Originally Posted by Thad House

However, the top teams are not usually the ones where 5 points makes a difference, so it most likely would not have much of an actual effect on the top.

This is the key point (pun intended) in the discussion^*
The small difference in points for teams at the top and bottom of the distribution make little impact on progression through the season. For example, in New England last year every team with a record of better than 10-2-0 at either of their scored events ranked in the top 10 teams going into the District Championship.

However running the 2015 rank-based points model on 2014 New England results shows that there would have been only 1 change to the teams invited to the District Championship. So it doesn't seem to make much difference in the middle of the pack either.

One small effect of the change is that teams are no longer penalized directly for disqualification or not being inspected in time for the qualifications matches. The indirect penalty of a reduction in average score may be larger.

Finally the use of the ceiling function means smaller events have a slightly higher expected points total per team, but this is minuscule in comparison to the huge seeding/playoff points advantage at smaller events.

^*although I personally enjoy any work incorporating a statistical model, so props to Danny!

[Rachel Lim]

Originally Posted by MasterMentor

You both forgot the Ceiling Function.

Thanks for the catch--I believe it's fixed now, but I don't know what the results should look like, so I can't really check. Please let me know if you catch anything else.

The same link should work: http://www.wolframalpha.com/widgets/...2fb3bb85e071b4

[AGPapa]

Originally Posted by artK

The answer to the first question is because, even in a WLT system, perfect 24s are pretty rare. In MAR, according to this, 22's have only been done twice, and in theory a 24 should have only happened once at this point. (Though it would be interesting to see what a difference a WLT would have on this game on the rankings). More matches per team make it harder to score a perfect record.

To clarify, 23's have only been done twice. 22's are a little more common.


Oh, and I made a spreadsheet showing how the QS points will be distributed by rank for every MAR event. The MAR events are all similar in size this year, so it doesn't change much event by event.

[MasterMentor]

Originally Posted by Zebra_Fact_Man

First takeaway: Why was the QS capped at 22 pts instead of the previous 24?

In the GameSense interview, Frank mentioned that only 0.6% of all teams scored 24 points last year. I guess it would have been nice to have a perfect comparison to last year, but ignoring the 0.6% doesn't seem devastating to me.

Originally Posted by Zebra_Fact_Man

Also, why is the minimum 4 instead of 0?

In the summary document, it says "typically-sized district events" would have a minimum of 4 points. Though, in the explanation document the tables show that at about an event size of 55 the minimum goes from 4 to 3. Using Rachel Lim's district points widget, it looks like 3 is the minimum going all the way up to a tournament size of 1,000. If you look at the explanation document, the range scale in the formula is 10, but the gain factor at the end is 12; that says to me the formula is built to never go below 2 points, and because of the nature of the Inverse Error Function (going to infinity at the limits and all) and the use of the Ceiling function, you will never go below 3 points period.

Though, I must admit, it would be a crushing blow for a team to show up at an event and get no points whatsoever. Even if they don't have aspirations of making it to the District Championships, getting a goose-egg is deflating.