Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   General Forum (http://www.chiefdelphi.com/forums/forumdisplay.php?f=16)
-   -   [FRC Blog] Frank Answers Friday: Championship Slots (http://www.chiefdelphi.com/forums/showthread.php?t=138045)

orangemoore 28-08-2015 20:50
[FRC Blog] Frank Answers Friday: Championship Slots
 
Posted on the FRC Blog, 8/28/15: http://www.usfirst.org/roboticsprogr...pionship-Slots
Quote:
Frank Answers Friday: Championship Slots

Blog Date: Friday, August 28, 2015 - 16:37


Today’s good question comes from Jon Stratis, a mentor on FRC Team 2177, The Robettes, from Mendota, Minnesota, USA:

Question:

Frank -

With the way championship slots are allocated to District teams versus regionals, we're seeing areas that are still doing regionals vastly under represented at champs when compared to District areas. For example, Minnesota in 2015 had 192 teams, 6.63% of all FRC teams, but only sent 24 teams to champs - 4% of teams at champs. That's 16 teams less than the state would have sent if it was a district! What plans does FIRST have for future District expansion or to remedy this situation and provide regional areas with more equal representative attendance at champs?

Thanks,

~Jon Stratis

Answer:

Hi Jon. Thanks for the question.

One of the advantages of the District system is the ease with which we can proportionally allocate Championship slots to teams. The standard points model, used for ranking, makes this a breeze, and if a District team that earns its way to Championship on points declines the invitation, we’ve got the next ranked team that has not yet received an invitation to make the offer to, and so on down the line, until all allocated slots are full. This is why we call these slots guaranteed.

While the District model has this and even more impressive advantages going for it, it has its downsides also, a major one being the amount of work and planning it takes at the local level to stand one up and keep it running. While FIRST HQ and the local organizations work together to get a District up and going, by far the greatest amount of work is done at the local level. So, while there are a few good candidates for 2017, and we’re certainly encouraging District transition for those areas that are ready, I don’t feel comfortable listing who those candidates are, as we are still uncertain they will become a reality at this point for 2017.

For Regionals, we will be staying with awarding slots based on performance at events themselves. I understand that this does not proportionally award Championship slots based on geographic representation. However, I think to award based on geographic representation under the Regional system we would need to take some type of pseudo-District ranking approach, which I believe is not a good fit for a competition model in which teams may be attending one, two, or more events, and there is no competition level between Regionals and the FIRST Championship. I envision such a system as being complex and getting us away from the immediate satisfaction and excitement teams can get at Regionals of learning they’ve got their golden ticket. However, I’m not closing the door on this forever. I’m willing to take a look at any specific proposal anyone has on this, or any other concern.

Thanks again!

Frank

Frank Answers Fridays is a weekly-ish blog feature where I’ll be answering ‘good questions’ from the FRC community. You can e-mail your questions to goodfrcquestion@usfirst.org. Please include your name, team number and where you’re from, which will be shared, if selected.

Jacob Bendicksen 29-08-2015 01:09
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Not a terribly satisfying answer, but a reasonable one.

Does anyone have a proposal that could be debated/sent to Frank?

Greg Needel 29-08-2015 08:48
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
The only solution to this that I can think of is to offer waitlist teams based on geography and performance in the season, instead of first cone first serve or time since last championship.

There would need to be some sort of ranking system for performance that normalizes the performance within a region ( maybe a modified version of the district point system). There would definitely be some issues as the algorithm would need to take into account quite a few different things. For instince there would always be a debate about which regions get priority as it would be nearly impossible to perfectly balance the regions. It would also be quite a bit of work for FIRST to inform teams and make these decisions, which for the most part only affect a few teams.

Another method of implementation might be to modify the wildcard system so that it continues to pass down until it's used. This would be problematic when you get to the semi finalists as then you would have to bring scores and rank( pre elims) into the picture and this get equally as muddy. For example should the 2nd round pick on the #3 alliance who was a semifinalist advance before the #5 alliance captain that got put out in the quarter finals? Wildcard right now is a good thing that does help cover some inequalities at the region but it also does influence alliance selection as teams who thin they can't win will consider which situation best earns them a wildcard spot, I am not sure what an expanded version of this system would do.

Though question that would take some serious thought to figure out a reasonable solution, if in fact it is worth solving at all because of the small number of teams this effects each year.

MikLast 29-08-2015 10:16
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Jacob Bendicksen (Post 1494527)
Not a terribly satisfying answer, but a reasonable one.

Does anyone have a proposal that could be debated/sent to Frank?
His thoughts on the PNW's new funding model?

Brian Maher 29-08-2015 12:19
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
My proposed solution is three-pronged:
  • Replace Finalist alliance position with district points for Wildcard distribution. The Stand Points Model was created to be an ideal assessment of team performance, so it seems perfect for this. This creates a system that can scale to include more than three teams, including those not on the Finalist alliance.
  • If a team declines a CMP spot, pass a wildcard down the district points rankings until all spots are filled.
  • Distribute extra Wildcard slots, which I call Bonus Slots, to large regionals. It seems silly to me that the 31-team Waterloo Regional and the 66-team Palmetto Regional qualify the same number of teams for CMP. This can be done by awarding the remaining "waitlist" slots to regionals based on the number of teams attending minus a certain value, which I used 40 for. While this doesn't create a strictly geographic distribution, it would be a much better approximation than the current system.

I put together a Google Spreadsheet demonstrating my proposal.

Mr V 29-08-2015 14:31
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Greg Needel (Post 1494532)
The only solution to this that I can think of is to offer waitlist teams based on geography and performance in the season, instead of first cone first serve or time since last championship.
The current method of allocating the number of extra spots at CMP is best described as a lottery. It is not first come first served. If you do however choose to put your hat in the ring the number of lottery tickets you get is based on the number of years since your last CMP attendance. Overall I think it does a pretty good job of allocating spots and achieving the goal of teams making it to CMP on a periodic basis.

Quote:
Originally Posted by BMSOTM (Post 1494542)
Distribute extra Wildcard slots, which I call Bonus Slots, to large regionals. It seems silly to me that the 31-team Waterloo regional and the 66-team Palmetto regional qualify the same number of teams for CMP. This can be done by awarding the remaining "waitlist" slots to regionals based on the number of teams attending minus a certain value, which I used 40 for.
I think that this represents the biggest issue. The spots a district has is proportional to their size but it is fixed across a large range of Regional sizes. As nice as it would be I just don't see a good way to have all Regionals have a similar number of teams. So a way to give more spots to larger Regionals is not a bad idea.

Another issue is the fact that a district gets a number of spots proportional to the size of the district and teams in the district are still eligible to enter the lottery. So in at least a couple of instances last season "extra" teams from a district were awarded spots at CMP. Lottery spots were also awarded before the end of the regular season. Now I understand that it was done to ensure that declines were available to be passed down and that teams had more time to make arrangements but it did have some interesting effects.

It made some Districts send more teams than their allotment in addition to those that were pre-qualified. (PNW had two teams that won the lottery, and they earned a spot pushing their earned spot down to the next teams in the rankings) It also may have resulted in more wildcard spots being generated at week 6 Regionals, though I do not know that for a fact. IF that did happen then teams attending week 6 events had an even greater chance at earning a wild card spot than they already had.

One thing that I think might be a good option is to reserve a number of lottery spots and remove them from the available spots when calculating the number of spots for a district. FIRST does want teams to be able to attend CMP occasionally even if they are not able to meet the performance standards necessary. (I know that some people don't like that idea but FIRST has determined that best suits their goals, so it is what it is, and I'm not trying to restart that debate.)


So how about something along these lines, now the numbers are just pulled out of a hat because I really don't know the real numbers for the upcoming year with all the new districts which are causing some Regionals to go away and a couple of new Regionals that are supposed to happen.

Say there are 250 spots that could potentially be earned at Regionals and teams are 50% Regional and 50% District. So that means that there would be 250 spots to divide by the districts based on their relative size. That would mean there would be 500 spots that could be earned subtract that and the number of prequalified teams which I'll call 25 for the sake of this discussion and you have 75 spots reserved for the lottery.

That would mean that the proportion of District teams vs Regional teams attending CMP would be relative to the respective proportion of all teams, less the fudge factor caused by pre-qualified teams and the inevitable randomness of the lottery winners. Of course that would still not make it so individual states got proportional representation due to the different size of Regionals and the fact that sometimes teams earn spots outside of their home state. Of course the fact is that not all states have events and for some locations an out of state event is more convenient as a team's first or second play.

The ultimate solution of course is for all teams to participate in the District system but the District system is more work than the Regional system and some areas are a long way from having the number of teams to make the District system viable let alone having the ability to put the infrastructure in place to make it happen.

Ginger Power 29-08-2015 15:15
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
One thing I'd really like to see happen is a way for the best individual team at a regional to get to champs. The way things currently stand, districts do a much better job of getting the top robots to champs than do regionals. Whether that happens through an existing slot like Engineering Inspiration, or some other method I don't care.

Doug Frisk 29-08-2015 16:45
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Ginger Power (Post 1494547)
One thing I'd really like to see happen is a way for the best individual team at a regional to get to champs. The way things currently stand, districts do a much better job of getting the top robots to champs than do regionals. Whether that happens through an existing slot like Engineering Inspiration, or some other method I don't care.
Best robot or best team, and how do you define either of those other than as the winner of the regional or Chairman's/EI?

ATannahill 29-08-2015 18:03
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
I find it interesting that according to the survey about two championships the most important element of the championship is 'Seeing and competing with the teams with the best robots in FRC' yet we are currently discussing how to distribute the championship slots fairly among district regions and regional regions?

Do people feel that we can have both of these? Can we properly represent all regions while still having the experience of seeing and facing the best robots in FRC?

Please don't turn this into a discussion about two championships. That horse has been beaten twice already.

EricH 29-08-2015 18:10
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by rtfgnow (Post 1494557)
Do people feel that we can have both of these? Can we properly represent all regions while still having the experience of seeing and facing the best robots in FRC?
I think so. The big issue is getting the "representational adjustment" slots TO the best robots in FRC in any given year. For example, getting MN's adjustment slots to the best MN teams.

Of course, solving that issue is a fairly "interesting" "little" problem...

Richard Wallace 29-08-2015 18:42
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
I wonder if the FRC community should change our expectation of how CMP slots ought to be allocated?

For example, would it be more effective* if CMP slots were allocated to states, or district systems that combine states, in proportion to the total number of official event matches played in each state or district before CMP, rather than to the number of teams?

-------
*I expect that reasonable people will disagree on how "effectiveness" ought to be defined and measured.

dodar 29-08-2015 18:51
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Richard Wallace (Post 1494559)
I wonder if the FRC community should change our expectation of how CMP slots ought to be allocated?

For example, would it be more effective* if CMP slots were allocated to states, or district systems that combine states, in proportion to the total number of official event matches played in each state or district before CMP, rather than to the number of teams?

-------
*I expect that reasonable people will disagree on how "effectiveness" ought to be defined and measured.
So then you would unjustly give more spots to district areas. Districts, on average, play more matches than regionals.

Until all teams are under the District System, the regional slot allocation should be what we stay with.

Ginger Power 29-08-2015 20:41
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by DareDad (Post 1494555)
Best robot or best team, and how do you define either of those other than as the winner of the regional or Chairman's/EI?
Make it another judge's award. "Best on Field Robot Performance" or something along those lines. Or give EI to the best engineered robot, which is likely one of the be one of the best on the field. There isn't a simple solution like there is with Districts.

AdamHeard 29-08-2015 20:48
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Ginger Power (Post 1494565)
Make it another judge's award. "Best on Field Robot Performance" or something along those lines. Or give EI to the best engineered robot, which is likely one of the be one of the best on the field. There isn't a simple solution like there is with Districts.
I've judged at an event, and I wouldn't trust Judges to pick the "best robot".

Ginger Power 29-08-2015 20:51
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by AdamHeard (Post 1494566)
I've judged at an event, and I wouldn't trust Judges to pick the "best robot".
My thing isn't that we need to get the absolute best robot out of regionals. Any robot good enough to be in the conversation is better than the status quo.

jajabinx124 29-08-2015 21:13
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Ginger Power (Post 1494565)
Or give EI to the best engineered robot, which is likely one of the be one of the best on the field. There isn't a simple solution like there is with Districts.
If they were to give EI to the best engineered robot, they would have to change the definition of the award Engineering Inspiration. Right now it celebrates outstanding success in advancing respect and appreciation for engineering within a team’s school and community.

Maybe your saying make the Excellence In Engineering award qualify a robot to worlds too?

Jon Stratis 29-08-2015 21:13
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by BMSOTM (Post 1494542)
My proposed solution is three-pronged:
  • Replace Finalist alliance position with district points for Wildcard distribution. This creates a system that can scale to include more than three teams, including those not on the Finalist alliance.
  • If a team declines a CMP spot, pass a wildcard down the district points rankings until all spots are filled. The Stand Points Model was created to be an ideal assessment of team performance, so it seems perfect for this.
  • Distribute extra Wildcard slots, which I call Bonus Slots, to large regionals. It seems silly to me that the 31-team Waterloo Regional and the 66-team Palmetto Regional qualify the same number of teams for CMP. This can be done by awarding the remaining "waitlist" slots to regionals based on the number of teams attending minus a certain value, which I used 40 for. While this doesn't create a strictly geographic distribution, it would be a much better approximation than the current system.

I put together a Google Spreadsheet demonstrating my proposal.
Personally, I really like this idea. You can use the district point system to rank teams at a regional event based on their performance (at that event only). Give the initial 6 slots to the winners, RA, CA, and EI, then any additional slots to the top X teams at the event, based on the event size. So a 30 team regional might just get those first 6 slots, while a 60 team regional would get 12. Obviously, the numbers would need to be tweaked accordingly each year, as they are now for districts, but it would completely solve the problem we currently have in MN - 4 60-team events at 12 teams per event would be 48 teams, and some of those would go to Wisconsin, Iowa, the Dakota's, or whoever else comes here to play.

Quote:
Originally Posted by Ginger Power (Post 1494565)
Make it another judge's award. "Best on Field Robot Performance" or something along those lines. Or give EI to the best engineered robot, which is likely one of the be one of the best on the field. There isn't a simple solution like there is with Districts.
Why isn't there? The district point system can be adopted to a single event format easily, in order to give points to teams at that event based on robot and award performance.

Knufire 30-08-2015 03:31
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Jon Stratis (Post 1494570)
Why isn't there? The district point system can be adopted to a single event format easily, in order to give points to teams at that event based on robot and award performance.
In a fairly large amount of cases, this would just qualify the first two robots on the winning and finalist alliances. Right now, a massive chunk of points come from getting picked high and going far. Culture changing awards points would need to be rebalanced to work in a single event format. Someone could run numbers and see how things would pan out.

Nemo 30-08-2015 07:06
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Knufire (Post 1494599)
In a fairly large amount of cases, this would just qualify the first two robots on the winning and finalist alliances. Right now, a massive chunk of points come from getting picked high and going far. Culture changing awards points would need to be rebalanced to work in a single event format. Someone could run numbers and see how things would pan out.
Those are usually the four robots that I'm most interested in seeing compete at the Championship level. I'm a fan of any system that sends those four.

Andrew Schreiber 30-08-2015 10:19
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by AdamHeard (Post 1494566)
I've judged at an event, and I wouldn't trust Judges to pick the "best robot".
They aren't supposed to.

And most of the times judges don't get to watch many of the matches.

Brian Maher 30-08-2015 10:36
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Knufire (Post 1494599)
In a fairly large amount of cases, this would just qualify the first two robots on the winning and finalist alliances. Right now, a massive chunk of points come from getting picked high and going far. Culture changing awards points would need to be rebalanced to work in a single event format. Someone could run numbers and see how things would pan out.
I took a look at District Rankings for the North Brunswick District Event:
Code:
Points        Team        Selection        Finish        Awards
73        303        1 Capt                W        Winner, Inn. in Cont.
72        2590        1 First                W        Winner, Entr.
60        11        2 Capt                F        Finalist, Judges'
51        193        2 First                F        Finalist
48        4285        3 Capt                SF        Exc in Eng
46        1923        5 First                SF        Chairman's
46        25        3 First                SF        Creativity
46        3340        1 Second        W        Winner
44        869        5 Capt                SF        Quality
43        1257        2 Second        F        Finalist, Spirit, Safety
36        4954        4 Capt                QF        Ind Des
33        5666        5 Second        SF        RAS
33        3314        7 Capt                SF        EI
31        219        4 First                QF
...
If District Events were regionals, CMP qualification would look something like this:

Winners: 303 (prequalified, Mount Olive Chairman's), 2590 (Montreal Winner), 3340
CA: 1923 (prequalified, TVR EI)
EI: 3314
RAS: 5666

Generating 3 wild cards:
Current System: 11, 193, 1257
District points: 11, 193, 4285

Looking at the OPR for that event, 4285 averaged ~43 points per match, and 1257 ~21 points per match. Using district points for wildcards sends a slightly different but more competitive set of teams to CMP.

AdamHeard 30-08-2015 10:38
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Andrew Schreiber (Post 1494605)
They aren't supposed to.

And most of the times judges don't get to watch many of the matches.
Agreed. I wasn't criticizing the people, it's just that the process isn't currently setup to identify the best robots (as that's not the current goal of the process). Identifying the best robot would require a large shift, or addition of more judges, to allow substantial match view time by more than a single judge or two.

Ginger Power 30-08-2015 10:43
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Nemo (Post 1494602)
Those are usually the four robots that I'm most interested in seeing compete at the Championship level. I'm a fan of any system that sends those four.
Agreed. My only concern with any such system is that you take all the competition out of the finals. If everybody is guaranteed to go to champs you're basically playing an exhibition match. If you weighted the points such that only the winning alliance's 3rd robot also got to go that would be way more intriguing.

Kevin Leonard 30-08-2015 10:53
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Andrew Schreiber (Post 1494605)
They aren't supposed to.

And most of the times judges don't get to watch many of the matches.
I had a student complain to me once that since most judges weren't FRC people, they generally didn't know how to objectively judge for awards.

And I just said "That's the point." Having judges do what they do isn't about objectively determining who the "Innovation in Controls" award winner should be- it's about exposing industry leaders who are often judges to these amazing students and exposing these students to industry leaders.

Ex: I talk to Team A and Team B, and Team A's students tell me in detail about their sloppy control system for their mechanism, while Team B wins the regional with their tightly and highly controlled mechanism but can't discuss it with the judges. I, as an FRC person who understands what these students are doing, might still award the award to the winning team, whereas an outsider judge will award it to the team who can talk about what they built better. That is (I think) an intentional part of the system. I place more focus on results, whereas a non-FRC person will place more emphasis on the attempt and the innovation than the results, while also learning about what FIRST-er's do.

I think the main problem with the regional system is when finalists aren't invited to championships or when the second best robot at an event loses in the semifinals because they were on the wrong side of the bracket or something.

What if (bold idea), before week 7, FIRST polls a number of experts on who the best 20 or so teams to not make championships are (kind of like this, but two weeks earlier) and invites them to the championship event.

Brian Maher 30-08-2015 10:57
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Ginger Power (Post 1494609)
Agreed. My only concern with any such system is that you take all the competition out of the finals. If everybody is guaranteed to go to champs you're basically playing an exhibition match. If you weighted the points such that only the winning alliance's 3rd robot also got to go that would be way more intriguing.
From my experience in a district, I've noticed that there tends to be one or two captain/first pick semifinalists that outrank the second pick winner, and a few more semifinalists that outrank the second pick finalist.

Ginger Power 30-08-2015 11:14
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by BMSOTM (Post 1494611)
From my experience in a district, I've noticed that there tends to be one or two captain/first pick semifinalists that outrank the second pick winner, and a few more semifinalists that outrank the second pick finalist.
I'm not as familiar with the district system, but I thought this would be the case. That's why I suggested manipulating the points so that winning the event is more valuable. Although the system works well as is, and just adds more incentive to strive for safety, and spirit awards to push your team over the top.

EricH 30-08-2015 11:47
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Kevin Leonard (Post 1494610)

What if (bold idea), before week 7, FIRST polls a number of experts on who the best 20 or so teams to not make championships are (kind of like this, but two weeks earlier) and invites them to the championship event.
That, I think, would be one of the best ideas. Let's expand towards original intent of the question.

1) FIRST polls experts on best 50 teams to not CURRENTLY make championships. (Trust me, there's a reason this number is so high.) This happens sometime around Week 6.
2) FIRST looks at who still has a chance to qualify (playing Week 7, maybe playing Week 6), or is near the top of the waitlist. These teams are set aside.
3) FIRST looks at geography and representation. Teams from underrepresented areas get higher priority than teams that are from fully represented areas.
4) The combined "best teams" and "underrepresented areas' teams" sorts determine the order of offering--but only to the top 25 on the combined list.
5) After Week 7 plays, the list is revisited. The bottom 25 and the teams that were set aside are now eligible for invites.

dodar 30-08-2015 12:13
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by EricH (Post 1494616)
That, I think, would be one of the best ideas. Let's expand towards original intent of the question.

1) FIRST polls experts on best 50 teams to not CURRENTLY make championships. (Trust me, there's a reason this number is so high.) This happens sometime around Week 6.
2) FIRST looks at who still has a chance to qualify (playing Week 7, maybe playing Week 6), or is near the top of the waitlist. These teams are set aside.
3) FIRST looks at geography and representation. Teams from underrepresented areas get higher priority than teams that are from fully represented areas.
4) The combined "best teams" and "underrepresented areas' teams" sorts determine the order of offering--but only to the top 25 on the combined list.
5) After Week 7 plays, the list is revisited. The bottom 25 and the teams that were set aside are now eligible for invites.
If 2 teams are on equal standing, why should where they are from set them apart? If a team is from an area that has more "better" teams, why should that hurt their chances of making it?

Brian Maher 30-08-2015 12:17
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by BMSOTM (Post 1494607)
I took a look at District Rankings for the North Brunswick District Event:
...
Winners: 303 (prequalified, Mount Olive Chairman's), 2590 (Montreal Winner), 3340
CA: 1923 (prequalified, TVR EI)
EI: 3314
RAS: 5666

Generating 3 wild cards:
Current System: 11, 193, 1257
District points: 11, 193, 4285
I just remembered that 3340 declined their DCMP invite. In our hypothetical regional, let's assume they'd decline a CMP invite. Using the district point wildcard system, this would qualify 25. Without this system, the extra slot would go unused, with three wildcards generated already.

Kevin Leonard 30-08-2015 12:19
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by dodar (Post 1494617)
If 2 teams are on equal standing, why should where they are from set them apart? If a team is from an area that has more "better" teams, why should that hurt their chances of making it?
To be fair, the district system does this already. Lets say district A had 250 teams and 50 of them are really good, while district B also has 250 teams, but only 20 of them are really good. District A might not be able to send all of their good teams, while district B gets to send a number of not as good teams to championships anyway.

Any sort of equal representation system based on region will do this, and that's (I think) an unavoidable problem, albeit a lesser problem compared to how championship slots are currently allocated to regionals.

EricH 30-08-2015 12:21
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by dodar (Post 1494617)
If 2 teams are on equal standing, why should where they are from set them apart? If a team is from an area that has more "better" teams, why should that hurt their chances of making it?
I'm referring to the original question. For those that have forgotten, the question was, essentially, "Why are REGIONAL areas underrepresented at Championship?" Not about an area having "more better" teams that don't get to go.

So what I'm looking at is that you take the top X that aren't going, and those that are from an UNDERrepresented area get first crack at going, all other things being equal. It's a hybrid system: top X teams that aren't currently going, ranked 1-X, combined in some way with the underrepresentation status of their respective areas to produce the final priority list on who gets asked.

It's not about being from an area with good teams killing your chances. It's about answering the original question (not the somewhat-related but tangential current main focus of the thread, how do we get all of the best teams to the Championship).

Mr V 30-08-2015 12:35
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by EricH (Post 1494616)
That, I think, would be one of the best ideas. Let's expand towards original intent of the question.

1) FIRST polls experts on best 50 teams to not CURRENTLY make championships. (Trust me, there's a reason this number is so high.) This happens sometime around Week 6.
2) FIRST looks at who still has a chance to qualify (playing Week 7, maybe playing Week 6), or is near the top of the waitlist. These teams are set aside.
3) FIRST looks at geography and representation. Teams from underrepresented areas get higher priority than teams that are from fully represented areas.
4) The combined "best teams" and "underrepresented areas' teams" sorts determine the order of offering--but only to the top 25 on the combined list.
5) After Week 7 plays, the list is revisited. The bottom 25 and the teams that were set aside are now eligible for invites.
There is no waitlist to be "near the top" it is a lottery and they already start pulling names out of the hat before week 6 because the teams that won lottery spots in the PNW last season were notified before the end of our week 6 DCMP.

Gregor 30-08-2015 12:36
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by EricH (Post 1494620)
"Why are REGIONAL areas underrepresented at Championship?"
Not all regional areas are underrepresented.

EricH 30-08-2015 12:43
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Mr V (Post 1494621)
There is no waitlist to be "near the top" it is a lottery and they already start pulling names out of the hat before week 6 because the teams that won lottery spots in the PNW last season were notified before the end of our week 6 DCMP.
Please read the thread before commenting. We aren't discussing the waitlist lottery. We're discussing alternatives that will try to balance out representation of areas to be more proportional.



Gregor, I'm aware of that. But there are a lot of areas that are under-represented, and they're all regional areas. (And there are over-represented areas, too. Not all of those are necessarily regional areas. But that's another discussion altogether.)

Ian Curtis 30-08-2015 12:45
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Food for though, given the current HQ direction we probably aren't that many years away from a time where a teams ability or willingness to pay to attend the Championships could become the limiting factor.

In 2018 will ~40% of FRC teams be able to attend a Championship? If that is followed up with 3Champs, will 50%+ of the teams in FRC be able to afford the trip?

If we sit on this issue for a few years, it just might solve itself.

Brian Maher 30-08-2015 13:42
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by dodar (Post 1494617)
If 2 teams are on equal standing, why should where they are from set them apart? If a team is from an area that has more "better" teams, why should that hurt their chances of making it?
Quote:
Originally Posted by Kevin Leonard (Post 1494619)
Any sort of equal representation system based on region will do this, and that's (I think) an unavoidable problem, albeit a lesser problem compared to how championship slots are currently allocated to regionals.
These are questions I've wondered about during my time in a district. Should districts be equally represented? Even if it means competitive teams from a more competitive district cannot attend?

I've thought a bit about this system, and have an idea for how to change it:
  • The total spots for all district teams is proportional to the total number of district teams
  • Half (or some other fraction) of these spots are distributed based on district team count, just like the current system.
  • For the previous year's Championship, district points (or an adapted version of district points) are calculated for all the district teams (who didn't qualify via waitlist).
  • These points are sums for each district, and sums are added to find a grand total.
  • The remaining slots are allocated based on each district's percentage of the total district points.

This system could potentially work well to adjust for variation of skill between districts. If a district's teams don't fare well at champs, indicating a weaker district, it will send fewer the next year. If a district has many successful teams at champs, it will send more (probably also competitive) teams to champs. The system should adjust itself.

However, it may also get out of balance, over-favoring some districts over time. This could probably be fixed by increasing the fraction of spots awarded based on population to more than one half.

I would run the numbers, but there are many waitlist teams and I don't want to have to figure out manually which teams attended from the waitlist and which ones didn't.

GKrotkov 30-08-2015 16:41
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by BMSOTM (Post 1494631)
However, it may also get out of balance, over-favoring some districts over time. This could probably be fixed by increasing the fraction of spots awarded based on population to more than one half.
That's an interesting idea. Not sure if I'm totally behind it, but it's a new way to look at the problem of district representation.

I don't think that that is guaranteed to work in the long term, because it doesn't address the underlying problem that you send fewer teams to Champs, you get fewer Champs points. Could we perhaps solve that by taking the median* of the calculated scores from each district and comparing them, instead of the sum?

* Or mean; I'm not sure if it'd be good to minimize the effect of outliers.

Doug Frisk 30-08-2015 17:05
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Gregor (Post 1494622)
Not all regional areas are underrepresented.
Big 60+ team regionals (like all five Minnesota regionals) end up underrepresented.

Thirty to forty team regionals tend to have a higher representation at champs. (The largest Canadian regional last year had 48 teams, the smallest couple only had 30. Not picking on Canada, but based on the maple leaf I'm guessing that was what you were referring to.)

Past that, week 1 regionals are over-represented and week 6 regionals are underrepresented.

That's simply because each regional is allocated 6 slots and a maximum of 3 wildcards regardless of size and once a team has won, winning again at another event can prevent another team from going to St. Louis.

A week 1 regional will always send 6 teams to worlds regardless of size.

Last year, North Star in Minneapolis (week 6) sent two new teams out of 60 teams attending to worlds.

So, what would I do?

Assuming a goal of 10% of regional participants moving to worlds.

Set a "wildcard threshold" of floor(regional participants /10)

So, if the regional has 30 participants the wildcard threshold is 3, if it's 67 it's 6.

Now, at a regional as many as six teams may go on to worlds. If the winning alliance, Engineering Inspiration, Chairman's, and Rookie All-star are all different teams who aren't already going to worlds they all go.

But if those winners or some of those winners are already going to worlds you'd look into creating wildcards.

Let's say that of those 6 winners there were 4 already going (only generating 2 new invitations) either because of duplicates at the event or previous wins.

So, if this is a 30 team regional it would generate 1 wildcard slot, if it were a 60 team regional it would generate 4 wildcards.

The first 3 wildcard slots are easy and already defined, that's the members of the finalist alliance.

Past those first 3, I'd suggest moving down the seedings established by the qualification matches so that the highest seeded teams who weren't already going get invites.

It would still be advantageous to go to a week 1, or 30 team regional, but not quite as much as it currently is.

jajabinx124 30-08-2015 17:27
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by DareDad (Post 1494641)

Past that, week 1 regionals are over-represented and week 6 regionals are underrepresented.

That's simply because each regional is allocated 6 slots and a maximum of 3 wildcards regardless of size and once a team has won, winning again at another event can prevent another team from going to St. Louis.

A week 1 regional will always send 6 teams to worlds regardless of size.

Last year, North Star in Minneapolis (week 6) sent two new teams out of 60 teams attending to worlds.
The 2015 North Star regional sent 967 and 2531 to champs because of the wildcard system, so North star sent 4 new teams to champs: 2491, 5576, 967, and 2531. 10k sent 6 new teams to champs: 2502, 3184, 5515, 2052, 4536, 4198.

MechEng83 30-08-2015 17:29
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by AdamHeard (Post 1494608)
Agreed. I wasn't criticizing the people, it's just that the process isn't currently setup to identify the best robots (as that's not the current goal of the process). Identifying the best robot would require a large shift, or addition of more judges, to allow substantial match view time by more than a single judge or two.
FYI, when there are enough judges, some are actually designated as "match watchers" to feed back some on-field performance observations to the other judges doing interviews in the pits. All judges are encouraged to watch matches as time allows. Your judge experience mileage may vary.

Doug Frisk 30-08-2015 18:42
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by jajabinx124 (Post 1494642)
The 2015 North Star regional sent 967 and 2531 to champs because of the wildcard system, so North star sent 4 new teams to champs: 2491, 5576, 967, and 2531. 10k sent 6 new teams to champs: 2502, 3184, 5515, 2052, 4536, 4198.
You are correct, of the regional winners and 3 major awards, only 2 of those 6 teams weren't already going. Which is what I was thinking about, mea culpa.

That generated a possibility of at most 3 wild cards even though only 2 of a potential 6 teams had received invites.

Of those three wild cards allocated to the finalist alliance, 5576 had already won rookie all star so it didn't use one. That essentially left championship spots on the table.

Under the system I proposed, 2 additional championship spots would have been offered to the highest seeded teams who weren't already advancing. That would have been the number 5 seed 3883 (Data Bits) and number 6 seed 2667 (Knights of the Valley).

Nemo 31-08-2015 15:27
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by DareDad (Post 1494653)
You are correct, of the regional winners and 3 major awards, only 2 of those 6 teams weren't already going. Which is what I was thinking about, mea culpa.

That generated a possibility of at most 3 wild cards even though only 2 of a potential 6 teams had received invites.

Of those three wild cards allocated to the finalist alliance, 5576 had already won rookie all star so it didn't use one. That essentially left championship spots on the table.

Under the system I proposed, 2 additional championship spots would have been offered to the highest seeded teams who weren't already advancing. That would have been the number 5 seed 3883 (Data Bits) and number 6 seed 2667 (Knights of the Valley).
Yeah, we should go deeper than finalist alliances to make sure all six of those champs slots get awarded to somebody at that regional. Personally, I'd be looking at semifinalist captains and first picks, with the alliance that got beat by the winners being higher on the pecking order. I like basing it off elimination rounds. But basing it off qualification seed would still be an improvement over losing those slots. It would also be possible to send another award winner, perhaps Engineering Excellence.

Caleb Sykes 31-08-2015 15:28
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
To be honest, I don't really think that the under-representation in MN is a problem that HQ should even try to solve. The solution, plain and simple, is that if we want a higher representation at champs, we just have to create more events, either within the regional system or in a new district system.

AdamHeard 31-08-2015 15:29
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by MechEng83 (Post 1494643)
FYI, when there are enough judges, some are actually designated as "match watchers" to feed back some on-field performance observations to the other judges doing interviews in the pits. All judges are encouraged to watch matches as time allows. Your judge experience mileage may vary.
Yes, but this is only a few judges. You'd want more discussion and input than that.

Andrew Schreiber 31-08-2015 15:36
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by MechEng83 (Post 1494643)
FYI, when there are enough judges, some are actually designated as "match watchers" to feed back some on-field performance observations to the other judges doing interviews in the pits. All judges are encouraged to watch matches as time allows. Your judge experience mileage may vary.
That's entirely up to the Judge Advisor/Judge Coordinator. Most think it's important but this is a relatively new change. And, we ran 2x Match Observers at NEU with only 13 judges, it just meant a little extra work for the other judges.

Edit: Also, the big issue with talking about how judging 'is' is that it varies from event to event because there's a fair bit of freedom in how exactly to get things done. In short, there's no One True Way.

Brian Maher 31-08-2015 16:33
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Nemo (Post 1494751)
Yeah, we should go deeper than finalist alliances to make sure all six of those champs slots get awarded to somebody at that regional. Personally, I'd be looking at semifinalist captains and first picks, with the alliance that got beat by the winners being higher on the pecking order. I like basing it off elimination rounds. But basing it off qualification seed would still be an improvement over losing those slots. It would also be possible to send another award winner, perhaps Engineering Excellence.
Are you familiar with the district point system? It's an excellent system for ranking based on the criteria you suggested:
  • 2 points per qual win, 1 per tie
  • 0-16 based on alliance selection
  • 10 per round of playoffs won (0 for quarterfinalist, 10 for semifinalist, 20 for finalist, 30 for winner)
  • 10 for Chairman's Award, 8 for Engineering Inspiration, 8 for Rookie All-Star, and 5 for all other judged awards
Earlier in the thread, I talked about how the distribution typically works out, after winner captain, winner first pick, finalist captain, finalist second pick:

Quote:
Originally Posted by BMSOTM (Post 1494611)
From my experience in a district, I've noticed that there tends to be one or two captain/first pick semifinalists that outrank the second pick winner, and a few more semifinalists that outrank the second pick finalist.

Mr V 31-08-2015 17:00
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Caleb Sykes (Post 1494752)
To be honest, I don't really think that the under-representation in MN is a problem that HQ should even try to solve. The solution, plain and simple, is that if we want a higher representation at champs, we just have to create more events, either within the regional system or in a new district system.
I agree fully, MN needs to join the district system soon, ok a year or two ago, and effort should not be waisted on trying to come up with someway to equalize things in what should be a very short term.

cadandcookies 31-08-2015 18:26
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Mr V (Post 1494761)
I agree fully, MN needs to join the district system soon, ok a year or two ago, and effort should not be waisted on trying to come up with someway to equalize things in what should be a very short term.
Unfortunately it still doesn't appear to be a majority of people in MN that want, let alone are willing to push districts. And it certainly doesn't appear to be being pushed at a decision making level. As has been pointed out in the "Current Districts Maps" thread, there are some major organizational hurdles MN needs to overcome first, and the first step to that is realizing (at all levels-- not just a team and not just at a decision making) that they need to be overcome for us as a state to move forward.

mwmac 31-08-2015 20:28
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Mr V (Post 1494761)
I agree fully, MN needs to join the district system soon, ok a year or two ago, and effort should not be waisted on trying to come up with someway to equalize things in what should be a very short term.
What about the teams from geographic areas that lack the population density to perhaps ever support a district system? I believe some thoughtful consideration of the issue would not be a waste.

Jacob Bendicksen 31-08-2015 21:17
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by mwmac (Post 1494786)
What about the teams from geographic areas that lack the population density to perhaps ever support a district system? I believe some thoughtful consideration of the issue would not be a waste.
It definitely wouldn't be a waste, and it'll become more of an issue as these low-density areas get 'boxed out' of the events they've been attending for years. I remember an idea coming up a while back about keeping a few regionals around for these teams (LA, Dallas, cities with big airports?), and I think this is where we'll probably end up. It's not an idea scenario for these teams in low-density areas, for sure.

Mr V 31-08-2015 21:28
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by mwmac (Post 1494786)
What about the teams from geographic areas that lack the population density to perhaps ever support a district system? I believe some thoughtful consideration of the issue would not be a waste.
I was referring specifically to MN with my comment. Overall I agree but at this point there are lots of districts ready to come on line soon and others that are well past a good time to start. So I believe that needs to be the top priority and once that shakes out then it is time to try and balance the few remaining Regionals.

To have true geographic representation you need to close borders otherwise what happens when someone from outside of that state border comes in and is in the position to get one of those for lack of a better term "bonus spots"? Do they get passed over because they are not from within that political boundary? Do they get the spot and risk the "but they stole our spot"?

mwmac 02-09-2015 17:01
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
A couple of comments:

1. One of the most striking issues I find is that no one has mentioned the problem of awarding Championship slots to districts based on their proportion of the FRC population in a post-Championsplit world. Using 2015 figures, and assuming them to remain constant for this discussion, Michigan (not picking on MI, just the easiest to make the point) teams comprised 11.86% of registered FRC teams which corresponds to roughly 71 slots for St. Louis. The same should hold true for 2016. However, beginning in 2017, Michigan's 345 teams will represent either 11.86% of the overall FRC teams or 23.73% of the corresponding Championsplit pool. Given that both Championsplit venues will host 400 FRC teams, will Michigan teams comprise 47 slots or 95? Clearly, something has to change with respect to the awarding of Championship slots on a proportional basis to district participants.

2. I have posted previously about the need for there to be a single set of consistent and fair qualification criteria for Championships. This post by BMSOTM looks like a potential solution, (assuming a return to W-L-T). The district point system has always served as a comprehensive way of separating wheat from chaff and advancing quality teams to Championships.

Quote:
Originally Posted by BMSOTM (Post 1494758)
Are you familiar with the district point system? It's an excellent system for ranking based on the criteria you suggested:
  • 2 points per qual win, 1 per tie
  • 0-16 based on alliance selection
  • 10 per round of playoffs won (0 for quarterfinalist, 10 for semifinalist, 20 for finalist, 30 for winner)
  • 10 for Chairman's Award, 8 for Engineering Inspiration, 8 for Rookie All-Star, and 5 for all other judged awards
In the closing paragraph of his blog post, Frank states that he is "not closing the door on this forever. I'm willing to take a look at any specific proposal anyone has on this, or any other concern."

Glad to see Frank is keeping an open mind on this issue as I believe the application of a district scoring model to the FRC population as a whole is a potential solution rather than a complicating factor. In 2015, we implemented average scores. Could apply the same methodology to account for teams attending 1 or more events with the top 400 teams advancing to their corresponding Championsplit venue. On the minus side, teams near the cut-off point face uncertainty in the closing weeks of the season but how does that differ from the current system used by districts?

Such a change would not necessarily address under-representation of geographic areas at Championships but would level the playing field to the greatest extent possible.

Christopher149 02-09-2015 21:34
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by mwmac (Post 1494997)
A couple of comments:

1. One of the most striking issues I find is that no one has mentioned the problem of awarding Championship slots to districts based on their proportion of the FRC population in a post-Championsplit world. Using 2015 figures, and assuming them to remain constant for this discussion, Michigan (not picking on MI, just the easiest to make the point) teams comprised 10% of the FRC teams and sent roughly 60 teams to St. Louis. The same should hold true for 2016. However, beginning in 2017, Michigan's 250 teams will represent either 10% of the overall FRC teams or 20% of the corresponding Championsplit pool. Given that both Championsplit venues will host 400 FRC teams, will Michigan teams comprise 40 slots or 80? Clearly, something has to change with respect to the awarding of Championship slots on a proportional basis to district participants.

2. I have posted previously about the need for there to be a single set of consistent and fair qualification criteria for Championships. This post by BMSOTM looks like a potential solution, (assuming a return to W-L-T). The district point system has always served as a comprehensive way of separating wheat from chaff and advancing quality teams to Championships.



In the closing paragraph of his blog post, Frank states that he is "not closing the door on this forever. I'm willing to take a look at any specific proposal anyone has on this, or any other concern."

Glad to see Frank is keeping an open mind on this issue as I believe the application of a district scoring model to the FRC population as a whole is a potential solution rather than a complicating factor. In 2015, we implemented average scores. Could apply the same methodology to account for teams attending 1 or more events with the top 400 teams advancing to their corresponding Championsplit venue. On the minus side, teams near the cut-off point face uncertainty in the closing weeks of the season but how does that differ from the current system used by districts?

Such a change would not necessarily address under-representation of geographic areas at Championships but would level the playing field to the greatest extent possible.
Michigan has 350, not 250.

mwmac 02-09-2015 22:02
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by Christopher149 (Post 1495017)
Michigan has 350, not 250.
Thank you for catching my error. I have corrected my figures in the post using 2908 as registered team head count (http://www.chiefdelphi.com/forums/sh...tration+201 5) and 345 as Michigan team count (https://my.usfirst.org/myarea/index....250#FRC_teams).

The issue is still significant for the allocation of post-Championsplit district slots for their corresponding venues. Just the spread is now 47 to 95 slots in the case of Michigan.

TJP123 02-09-2015 23:28
Re: [FRC Blog] Frank Answers Friday: Championship Slots
 
Quote:
Originally Posted by mwmac (Post 1494997)
1. One of the most striking issues I find is that no one has mentioned the problem of awarding Championship slots to districts based on their proportion of the FRC population in a post-Championsplit world. Using 2015 figures, and assuming them to remain constant for this discussion, Michigan (not picking on MI, just the easiest to make the point) teams comprised 11.86% of registered FRC teams which corresponds to roughly 71 slots for St. Louis. The same should hold true for 2016. However, beginning in 2017, Michigan's 345 teams will represent either 11.86% of the overall FRC teams or 23.73% of the corresponding Championsplit pool. Given that both Championsplit venues will host 400 FRC teams, will Michigan teams comprise 47 slots or 95? Clearly, something has to change with respect to the awarding of Championship slots on a proportional basis to district participants.
To stay proportional, it's either 11% of 800, or 22% of 400, because MI teams would theoretically make up 22% of the smaller population attending their Championship event. They equal the same number.


All times are GMT -5. The time now is 11:38.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi