[FRC Blog] Two Championship Survey Results and Path Forward

[Anupam Goli]

Originally Posted by R2D2DOC

Transitions are always difficult. The USA never quite converted to the metric system. . . .

Ultimately, if we saturate the season with districts (even that is contentious), then district champs would be the primary players at world champs. The remaining participants would be up for discussion: HOF, Rookie, Chairmans, etc. . . .

Would we than go back to a smaller single championship event?

At this point, I don't know if we can tell. Part of FIRST HQ's understanding is the Championship "experience" is what they want as many students to be able to experience as possible. Converting that experience to the district championship level will hopefully be more feasible in the coming future. But what defines the championship "experience"? What does FIRST want to provide to as many students as possible, and how can we make it reach as many students as possible?

While I don't know what the answers to those questions are, the reasoning and logic behind the championsplit, to me, indicates that FIRST is trying to bring the experience by having more championship-scale events. It's entirely possible we go from here to 4 super regionals, and not have a culminating championship event. At least that's what I see is in the realm of possibilities, following the logic of this decision.

[scottandme]

Originally Posted by wilsonmw04

So what this data tells me is that a small number of teams really hate this idea and are very vocal about it. This seems to jive with what typically happens here on CD.

26% of respondents really, really, really hate the idea, and it's also the most popular answer. I wouldn't look at that as being a "small number" or insignificant.

Looking at the "important elements" - #1,#3, and #4 are all impossible or highly diluted by implementing 2 championships.

[wilsonmw04]

Originally Posted by scottandme

26% of respondents really, really, really hate the idea, and it's also the most popular answer.

That's the thing: 26% of RESPONDENTS don't like it. When they gave each team "1 vote," it was average out as favorable. Therefore, my statement stands. Certain groups don't like this new idea and were very vocal about it.

[Kevin Leonard]

Originally Posted by wilsonmw04

That's the thing: 26% of RESPONDENTS don't like it. When they gave each team "1 vote," it was average out as favorable. Therefore, my statement stands. Certain groups don't like this new idea and were very vocal about it.

Except technically, On a scale of 1-10, 5.1 is negative. For the scale to be accurate, it would have to have 0-10 or 5.5 as the average "I don't care" response.

I'd love to see the raw data and make my own biased set of statistics too.

[Alan Anderson]

Originally Posted by wilsonmw04

That's the thing: 26% of RESPONDENTS don't like it.

Count again. I get 55%.

When they gave each team "1 vote," it was average out as favorable.

If only two people vote, one for "Strongly Oppose" and one for "Strongly Favor", a naive average comes out at 5.5, which is a larger value than the 5.2 that you're calling favorable. The numbering of options, with neutral placed below the center point, breaks any simple attempt to do averaging.

The strongest takeaway I get from this chart is that over half the respondents (55%) are opposed, and less than a third (33%) are in favor.

[wilsonmw04]

Originally Posted by Alan Anderson

Count again. I get 55%.



If only two people vote, one for "Strongly Oppose" and one for "Strongly Favor", a naive average comes out at 5.5, which is a larger value than the 5.2 that you're calling favorable. The numbering of options, with neutral placed below the center point, breaks any simple attempt to do averaging.

The strongest takeaway I get from this chart is that over half the respondents (56%) are opposed, and less than a third (32%) are in favor.

Not arguing any of that. The fact that this is voluntary would tend to favor the extremes. Since this is a change, those who favor the change are probably less motivated to complete this survey. I wonder why they even bothered to send the survey out.

[drwisley]

This continues to get more ridiculous, I'm actually insulted by this blog.

[Hot_Copper_Frog]

Taken as a whole, I think there is nothing within these survey results that is surprising. They do reinforce the idea that some within our community are strongly opposed to the two Championships concept, and that we should be using the elements identified by teams as most important to them as a guide to refining the concept to help ensure the best experience for all teams as we work through this significant change.

When I think "some", I don't think 26% and I CERTAINLY don't think 55%. This opinion is very representative of the organization as a whole. I find it interesting that it's only stated that results indicate FIRST HQ should use these elements to guide refinement of the concept, not that FIRST HQ is taking them into consideration.

Our intent is to form committees, including representatives from the community outside FIRST HQ, to make recommendations to FIRST leadership in addressing the two key challenges listed below.

• Identifying what geographic regions will be assigned to which FIRST Championship as their ‘home’ Championship, including the way in which teams outside the United States would be handled
• Identifying a potential way in which teams may volunteer and be selected to attend their non-home Championship

You will hear more about these committees over the next few months. As we noted in the Championship informational session, the facts that there will be two Championships starting in 2017, and that all FIRST programs will be represented at each Championship, will not be changing, and so won’t be part of the discussions undertaken by these groups.

So...the committees are only dedicated to identifying possible solutions to issues related to geographical assignment. And these committees will make recommendations, which may or may not be implemented. Nothing else will be addressed, as nothing else is up for discussion.

Additionally, FIRST HQ will be exploring the possibility of some culminating event to take place after the two Championships, at which we would bring together the top teams from each Championship in some final competition of the season. This idea is still in the early exploratory phase, and we will share additional details, including potential areas for community input, as appropriate.

So, in order to appease a community that is upset that their thoughts were not taken into account for such a massive change, we are going to ask the two champion alliances to participate in an afterthought showdown event. Ask mentors who have already drained most of their vacation days to take MORE time off to travel, ask students who have already missed many days of school to miss some more, and ask teams who have already spent thousands of dollars on travel to spend a few more.

I want to be excited about this. I really do. I AM excited to bring championships home to Detroit. I'm just...uncomfortable with how all of this is being handled.

[Siri]

Originally Posted by wilsonmw04

So what this data tells me is that a small number of teams really hate this idea and are very vocal about it. This seems to jive with what typically happens here on CD.

More than a quarter of the respondents are as against this proposal as it is possible to register on this scale. That's closed to 2,000 people in itself (numerically 1912.3 people). 55% of respondents are against the proposal; that's more than 4045 people. CD is vocal, but even assuming the relationship between CD and the survey sample (which is a weird assumption when n=7355), opposition by definition is not the minority opinion. And despite the scale shift, the "strongly opposed" outnumbers all those who voted 10, 9, and 8 combined. More people voted for 1 or 2 than voted for anything above 5.



Does anyone know if there's a standard method of "centering" a scale like this? (The true center is at 5.5, the average of 1 and 10). I don't have a statistical method of turning 4 buckets into 5, but I think the worst-case scenario would be that everyone who voted 1 would've voted 0, and everyone in 2 took 1 (no one votes 4). This creates a new weighted average of 3.92, which represents the low end of possibility: thus the average is somewhere between 3.92 and 4.47 when centered about 5. Did I handle that correctly?

[DanielleSisk]

Originally Posted by Siri

More than a quarter of the respondents are as against this proposal as it is possible to register on this scale. That's closed to 2,000 people in itself (numerically 1912.3 people). 55% of respondents are against the proposal; that's more than 4045 people. CD is vocal, but even assuming the relationship between CD and the survey sample (which is a weird assumption when n=7355), opposition by definition is not the minority opinion. And despite the scale shift, the "strongly opposed" outnumbers all those who voted 10, 9, and 8 combined. More people voted for 1 or 2 than voted for anything above 5.



Does anyone know if there's a standard method of "centering" a scale like this? (The true center is at 5.5, the average of 1 and 10). I don't have a statistical method of turning 4 buckets into 5, but I think the worst-case scenario would be that everyone who voted 1 would've voted 0, and everyone in 2 took 1 (no one votes 4). This creates a new weighted average of 3.92, which represents the low end of possibility: thus the average is somewhere between 3.92 and 4.47 when centered about 5. Did I handle that correctly?

This is simple to do by reducing to number of bins instead of increasing them. Put all responses below the Neutral choice, the 55%, in one bin and call it say bin 4 (to keep our median at 5), then put the 12% which are Neutral into bin 5, and those above neutral, the 33%, into bin 6. Finding the average this way, (55*4+12*5+33*6)/100 = 4.78. Interesting... this shows that giving more options for a positive response gave their average, 4.45 (which should have been 4.47) a skew in the negative response direction. Now the same calculation cannot done for a '1 vote per team' because that data is unavailable.

[Siri]

Originally Posted by DanielleSisk

This is simple to do by reducing to number of bins instead of increasing them. Put all responses below the Neutral choice, the 55%, in one bin and call it say bin 4 (to keep our median at 5), then put the 12% which are Neutral into bin 5, and those above neutral, the 33%, into bin 6. Finding the average this way, (55*4+12*5+33*6)/100 = 4.78. Interesting... this shows that giving more options for a positive response gave their average, 4.45 (which should have been 4.47) a skew in the negative response direction. Now the same calculation cannot done for a '1 vote per team' because that data is unavailable.

I'm not sure I understand: your representative bin values are entirely arbitrary, as is the average they produce. I could repeat this same calculation calling "negative" 3 and "positive" 7: I get 4.56. 2 and 8 yields 4.34; 1 and 9 is 4.12; 0 and 10 is 3.9. The problem is that the logical value to assign to each bin is the average of the values in it--1 through 4 as 2.5 and 6 to 10 as 8--but these averages are not centered about the neutral. Shrinking the bin count does not remove this problem.

[fargus111111111]

Originally Posted by wilsonmw04

So what this data tells me is that a small number of teams really hate this idea and are very vocal about it. This seems to jive with what typically happens here on CD.

of those who gave a team number, 52% of teams were represented, that is over half of the teams in FRC.

[fargus111111111]

I am somewhat relieved that they are even considering an event after "Champs" to become the new Championships because, as my team was discussing the other day, that is what IRI would otherwise become. I am concerned about this format though due to travel costs. Our team certainly is not one of the richest, but we have decent funding and two regionals plus champs stretched our budget this year. I am more optimistic about the district model and advancing through a series of competitions that way, resulting in lower travel costs but potentially more competitions. Also if FIRST is so focused on getting a set percentage of FRC teams to champs why do they not seem to care about FTC or FLL. I find their interpretation of the data odd. If the average response is 4.45 then that suggests to me that while it is not a strong opposition there is an opposition. I am concerned that FIRST seems to be ignoring the community they are supposed to serve. If they truly wanted the community's opinion they would have done a study asking x number of people, students and mentors from each team to complete the survey instead of whoever in the community felt like it. I am concerned about the direction FRC is going. While the game this year was exiting and competitive it did not have the same viewing appeal that many other games have had. If FIRST wants to keep this competition interesting and keep encouraging new people to get involved they need the high level of competition ON the field with the same Olympic high stakes. Last year we went to an off season event and took a number of new members with us. When we returned we asked them what they thought about the competition and one replied, "I thought is was going to be just a bunch of nerds standing quietly around a field watching their robot, I could not have been more wrong." This is the impression that FIRST needs to make on people however I am concerned that if they continue straight down the path they are on the competition will slowly die off and it will become a bunch of nerds standing around a field quietly watching their robot. Please FIRST hear us out, we want competition, this is supposed to be like the olympics right, not Tee-Ball. (although that could be an interesting robot game)

[JB987]

Can we stop suggesting that IRI would become a substitute for a Championship? Many years there are Einstein teams that are unable to make it to Indiana (just look at this summer's team list to see current examples) and often the drive teams are not the same anyways...

[brrian27]

Something used in business is the Net Promoter Score (NPS) based on the simple question to customers of whether they would recommend a product to others on a scale of 1-10 (actually 0-10, but we'll set the bottom at 1 as FIRST did). For the NPS, responses of 1-6 are "detractors," 7-8 are "passives," and 9-10 are "promoters." The NPS is calculated by the percentage of promoters minus the percentage of detractors.

This model doesn't perfectly apply to this survey question, since it is not a purely recommendation question, but we can view it as basically asking if you would recommend the championship split to FIRST. Despite the imperfect application, this model does remind us that people who vote 6-8 aren't as satisfied as we think.

Anyway, for this question the championship split has an NPS of -55, which is not pretty. An average company gets an NPS between of between 5 and 10. Here is a benchmark for NPS.

Here's more info about NPS. it's not a perfect application, but it's an interesting perspective.