[FRC Blog] Frank Answers Friday: Championship Slots

[Andrew Schreiber]

Originally Posted by AdamHeard

I've judged at an event, and I wouldn't trust Judges to pick the "best robot".

They aren't supposed to.

And most of the times judges don't get to watch many of the matches.

[AdamHeard]

Originally Posted by Andrew Schreiber

They aren't supposed to.

And most of the times judges don't get to watch many of the matches.

Agreed. I wasn't criticizing the people, it's just that the process isn't currently setup to identify the best robots (as that's not the current goal of the process). Identifying the best robot would require a large shift, or addition of more judges, to allow substantial match view time by more than a single judge or two.

[Kevin Leonard]

Originally Posted by Andrew Schreiber

They aren't supposed to.

And most of the times judges don't get to watch many of the matches.

I had a student complain to me once that since most judges weren't FRC people, they generally didn't know how to objectively judge for awards.

And I just said "That's the point." Having judges do what they do isn't about objectively determining who the "Innovation in Controls" award winner should be- it's about exposing industry leaders who are often judges to these amazing students and exposing these students to industry leaders.

Ex: I talk to Team A and Team B, and Team A's students tell me in detail about their sloppy control system for their mechanism, while Team B wins the regional with their tightly and highly controlled mechanism but can't discuss it with the judges. I, as an FRC person who understands what these students are doing, might still award the award to the winning team, whereas an outsider judge will award it to the team who can talk about what they built better. That is (I think) an intentional part of the system. I place more focus on results, whereas a non-FRC person will place more emphasis on the attempt and the innovation than the results, while also learning about what FIRST-er's do.

I think the main problem with the regional system is when finalists aren't invited to championships or when the second best robot at an event loses in the semifinals because they were on the wrong side of the bracket or something.

What if (bold idea), before week 7, FIRST polls a number of experts on who the best 20 or so teams to not make championships are (kind of like this, but two weeks earlier) and invites them to the championship event.

[jajabinx124]

Originally Posted by Ginger Power

Or give EI to the best engineered robot, which is likely one of the be one of the best on the field. There isn't a simple solution like there is with Districts.

If they were to give EI to the best engineered robot, they would have to change the definition of the award Engineering Inspiration. Right now it celebrates outstanding success in advancing respect and appreciation for engineering within a team’s school and community.

Maybe your saying make the Excellence In Engineering award qualify a robot to worlds too?

[Jon Stratis]

Originally Posted by BMSOTM

My proposed solution is three-pronged:

• Replace Finalist alliance position with district points for Wildcard distribution. This creates a system that can scale to include more than three teams, including those not on the Finalist alliance.
• If a team declines a CMP spot, pass a wildcard down the district points rankings until all spots are filled. The Stand Points Model was created to be an ideal assessment of team performance, so it seems perfect for this.
• Distribute extra Wildcard slots, which I call Bonus Slots, to large regionals. It seems silly to me that the 31-team Waterloo Regional and the 66-team Palmetto Regional qualify the same number of teams for CMP. This can be done by awarding the remaining "waitlist" slots to regionals based on the number of teams attending minus a certain value, which I used 40 for. While this doesn't create a strictly geographic distribution, it would be a much better approximation than the current system.

I put together a Google Spreadsheet demonstrating my proposal.

Personally, I really like this idea. You can use the district point system to rank teams at a regional event based on their performance (at that event only). Give the initial 6 slots to the winners, RA, CA, and EI, then any additional slots to the top X teams at the event, based on the event size. So a 30 team regional might just get those first 6 slots, while a 60 team regional would get 12. Obviously, the numbers would need to be tweaked accordingly each year, as they are now for districts, but it would completely solve the problem we currently have in MN - 4 60-team events at 12 teams per event would be 48 teams, and some of those would go to Wisconsin, Iowa, the Dakota's, or whoever else comes here to play.

Originally Posted by Ginger Power

Make it another judge's award. "Best on Field Robot Performance" or something along those lines. Or give EI to the best engineered robot, which is likely one of the be one of the best on the field. There isn't a simple solution like there is with Districts.

Why isn't there? The district point system can be adopted to a single event format easily, in order to give points to teams at that event based on robot and award performance.

[Knufire]

Originally Posted by Jon Stratis

Why isn't there? The district point system can be adopted to a single event format easily, in order to give points to teams at that event based on robot and award performance.

In a fairly large amount of cases, this would just qualify the first two robots on the winning and finalist alliances. Right now, a massive chunk of points come from getting picked high and going far. Culture changing awards points would need to be rebalanced to work in a single event format. Someone could run numbers and see how things would pan out.

[Nemo]

Originally Posted by Knufire

In a fairly large amount of cases, this would just qualify the first two robots on the winning and finalist alliances. Right now, a massive chunk of points come from getting picked high and going far. Culture changing awards points would need to be rebalanced to work in a single event format. Someone could run numbers and see how things would pan out.

Those are usually the four robots that I'm most interested in seeing compete at the Championship level. I'm a fan of any system that sends those four.

[Brian Maher]

Originally Posted by Knufire

In a fairly large amount of cases, this would just qualify the first two robots on the winning and finalist alliances. Right now, a massive chunk of points come from getting picked high and going far. Culture changing awards points would need to be rebalanced to work in a single event format. Someone could run numbers and see how things would pan out.

I took a look at District Rankings for the North Brunswick District Event:

Points 	Team	Selection	Finish	Awards
73	303	1 Capt		W	Winner, Inn. in Cont.
72	2590	1 First		W	Winner, Entr.
60	11	2 Capt		F	Finalist, Judges'
51	193	2 First		F	Finalist
48	4285	3 Capt		SF	Exc in Eng
46	1923	5 First		SF	Chairman's
46	25	3 First		SF	Creativity
46	3340	1 Second	W	Winner
44	869	5 Capt		SF	Quality
43	1257	2 Second	F	Finalist, Spirit, Safety
36	4954	4 Capt		QF	Ind Des
33	5666	5 Second	SF	RAS
33	3314	7 Capt		SF	EI
31	219	4 First		QF
...

If District Events were regionals, CMP qualification would look something like this:

Winners: 303 (prequalified, Mount Olive Chairman's), 2590 (Montreal Winner), 3340
CA: 1923 (prequalified, TVR EI)
EI: 3314
RAS: 5666

Generating 3 wild cards:
Current System: 11, 193, 1257
District points: 11, 193, 4285

Looking at the OPR for that event, 4285 averaged ~43 points per match, and 1257 ~21 points per match. Using district points for wildcards sends a slightly different but more competitive set of teams to CMP.

[mwmac]

A couple of comments:

1. One of the most striking issues I find is that no one has mentioned the problem of awarding Championship slots to districts based on their proportion of the FRC population in a post-Championsplit world. Using 2015 figures, and assuming them to remain constant for this discussion, Michigan (not picking on MI, just the easiest to make the point) teams comprised 11.86% of registered FRC teams which corresponds to roughly 71 slots for St. Louis. The same should hold true for 2016. However, beginning in 2017, Michigan's 345 teams will represent either 11.86% of the overall FRC teams or 23.73% of the corresponding Championsplit pool. Given that both Championsplit venues will host 400 FRC teams, will Michigan teams comprise 47 slots or 95? Clearly, something has to change with respect to the awarding of Championship slots on a proportional basis to district participants.

2. I have posted previously about the need for there to be a single set of consistent and fair qualification criteria for Championships. This post by BMSOTM looks like a potential solution, (assuming a return to W-L-T). The district point system has always served as a comprehensive way of separating wheat from chaff and advancing quality teams to Championships.

Originally Posted by BMSOTM

Are you familiar with the district point system? It's an excellent system for ranking based on the criteria you suggested:

• 2 points per qual win, 1 per tie
• 0-16 based on alliance selection
• 10 per round of playoffs won (0 for quarterfinalist, 10 for semifinalist, 20 for finalist, 30 for winner)
• 10 for Chairman's Award, 8 for Engineering Inspiration, 8 for Rookie All-Star, and 5 for all other judged awards

In the closing paragraph of his blog post, Frank states that he is "not closing the door on this forever. I'm willing to take a look at any specific proposal anyone has on this, or any other concern."

Glad to see Frank is keeping an open mind on this issue as I believe the application of a district scoring model to the FRC population as a whole is a potential solution rather than a complicating factor. In 2015, we implemented average scores. Could apply the same methodology to account for teams attending 1 or more events with the top 400 teams advancing to their corresponding Championsplit venue. On the minus side, teams near the cut-off point face uncertainty in the closing weeks of the season but how does that differ from the current system used by districts?

Such a change would not necessarily address under-representation of geographic areas at Championships but would level the playing field to the greatest extent possible.

[Christopher149]

Originally Posted by mwmac

A couple of comments:

1. One of the most striking issues I find is that no one has mentioned the problem of awarding Championship slots to districts based on their proportion of the FRC population in a post-Championsplit world. Using 2015 figures, and assuming them to remain constant for this discussion, Michigan (not picking on MI, just the easiest to make the point) teams comprised 10% of the FRC teams and sent roughly 60 teams to St. Louis. The same should hold true for 2016. However, beginning in 2017, Michigan's 250 teams will represent either 10% of the overall FRC teams or 20% of the corresponding Championsplit pool. Given that both Championsplit venues will host 400 FRC teams, will Michigan teams comprise 40 slots or 80? Clearly, something has to change with respect to the awarding of Championship slots on a proportional basis to district participants.

2. I have posted previously about the need for there to be a single set of consistent and fair qualification criteria for Championships. This post by BMSOTM looks like a potential solution, (assuming a return to W-L-T). The district point system has always served as a comprehensive way of separating wheat from chaff and advancing quality teams to Championships.



In the closing paragraph of his blog post, Frank states that he is "not closing the door on this forever. I'm willing to take a look at any specific proposal anyone has on this, or any other concern."

Glad to see Frank is keeping an open mind on this issue as I believe the application of a district scoring model to the FRC population as a whole is a potential solution rather than a complicating factor. In 2015, we implemented average scores. Could apply the same methodology to account for teams attending 1 or more events with the top 400 teams advancing to their corresponding Championsplit venue. On the minus side, teams near the cut-off point face uncertainty in the closing weeks of the season but how does that differ from the current system used by districts?

Such a change would not necessarily address under-representation of geographic areas at Championships but would level the playing field to the greatest extent possible.

Michigan has 350, not 250.

[mwmac]

Originally Posted by Christopher149

Michigan has 350, not 250.

Thank you for catching my error. I have corrected my figures in the post using 2908 as registered team head count (http://www.chiefdelphi.com/forums/sh...tration+201 5) and 345 as Michigan team count (https://my.usfirst.org/myarea/index....250#FRC_teams).

The issue is still significant for the allocation of post-Championsplit district slots for their corresponding venues. Just the spread is now 47 to 95 slots in the case of Michigan.

[TJP123]

Originally Posted by mwmac

1. One of the most striking issues I find is that no one has mentioned the problem of awarding Championship slots to districts based on their proportion of the FRC population in a post-Championsplit world. Using 2015 figures, and assuming them to remain constant for this discussion, Michigan (not picking on MI, just the easiest to make the point) teams comprised 11.86% of registered FRC teams which corresponds to roughly 71 slots for St. Louis. The same should hold true for 2016. However, beginning in 2017, Michigan's 345 teams will represent either 11.86% of the overall FRC teams or 23.73% of the corresponding Championsplit pool. Given that both Championsplit venues will host 400 FRC teams, will Michigan teams comprise 47 slots or 95? Clearly, something has to change with respect to the awarding of Championship slots on a proportional basis to district participants.

To stay proportional, it's either 11% of 800, or 22% of 400, because MI teams would theoretically make up 22% of the smaller population attending their Championship event. They equal the same number.