[FRC Blog] Two Championship Survey Results and Path Forward

[Nate Laverdure]

Taylor, here's how you upload an image.

[Alan Anderson]

Originally Posted by wilsonmw04

That's the thing: 26% of RESPONDENTS don't like it.

Count again. I get 55%.

When they gave each team "1 vote," it was average out as favorable.

If only two people vote, one for "Strongly Oppose" and one for "Strongly Favor", a naive average comes out at 5.5, which is a larger value than the 5.2 that you're calling favorable. The numbering of options, with neutral placed below the center point, breaks any simple attempt to do averaging.

The strongest takeaway I get from this chart is that over half the respondents (55%) are opposed, and less than a third (33%) are in favor.

[wilsonmw04]

Originally Posted by Alan Anderson

Count again. I get 55%.



If only two people vote, one for "Strongly Oppose" and one for "Strongly Favor", a naive average comes out at 5.5, which is a larger value than the 5.2 that you're calling favorable. The numbering of options, with neutral placed below the center point, breaks any simple attempt to do averaging.

The strongest takeaway I get from this chart is that over half the respondents (56%) are opposed, and less than a third (32%) are in favor.

Not arguing any of that. The fact that this is voluntary would tend to favor the extremes. Since this is a change, those who favor the change are probably less motivated to complete this survey. I wonder why they even bothered to send the survey out.

[Karthik]

FIRST's analysis of the survey numbers and this thread are great examples of the old theory of "people are just going to see what they want to see in the numbers."

When I looked at the numbers, I immediately grouped the data into buckets.

Strongly oppose (ratings 1-3): 48%
Neutral (ratings 4-7): 29%
Strongly favour (ratings 8-10): 23%

Of course, my parsing of the data is probably skewed by my own biases. Your mileage may vary.

[drwisley]

This continues to get more ridiculous, I'm actually insulted by this blog.

[DCA Fan]

Originally Posted by Andrew Schreiber

Now, I'd like to have A championship that celebrates all the FIRST programs, the values of FIRST, and celebrates STEM.

To be honest, in my opinion, if FLL were allowed to have its own championship - not just a world festival, but a true, world championship, it would be a true powerhouse STEM event. Right now, since FLL only gets a certain number of slots - they make due with this festival and then rely on event partners to have Open Championships (I believe there are 5 now around the world).

Sadly, the willpower is not there to make this happen.

[IronicDeadBird]

I sincerely hope that FIRST has a solid plan for those community input channels.

[jman4747]

I worry about "social desirability bias" within and between teams.

And I hinted at it before but as I think about it how is "Keeping attendance costs reasonable" so low? It bothers me. To me it says that a higher proportion of teams don't need to worry about paying the same or more money for registration and travel. That is not indicative of a lot of teams.

Going deeper:

We do know more people care about (or people care more about):

• Seeing and competing with the teams with the best robots in FRC
• The experience of attending a major, multi-day event with my team.
• Participating in a competition that identifies the best teams playing the game
• Seeing teams you have built relationships/partnerships with over the years

Who would care most about these things but also not need to worry about costs?

Lets look as attendance costs as registration fee first. Reasonable is most likely not less that $5,000 as clearly that's whats been needed. It also says "keep reasonable" implying the $5,000 we've had is reasonable. So if the majority of teams/people answering don't need registration to stay = or < $5,000...

Lets even assume that the vast majority of teams actually have a sponsor who will upon the team qualifying pay the fee in full; money they would not have had otherwise. In that case travel costs are the only major money concern. Those costs go over $3,000 for hotels alone very easily.

Who would care that much more about the above 4 than costs staying at least where they are?

[AdamHeard]

Originally Posted by jman4747

I worry about "social desirability bias" within and between teams.

And I hinted at it before but as I think about it how is "Keeping attendance costs reasonable" so low? It bothers me. To me it says that a higher proportion of teams don't need to worry about paying the same or more money for registration and travel. That is not indicative of a lot of teams.

Going deeper:

We do know more people care about (or people care more about):

• Seeing and competing with the teams with the best robots in FRC
• The experience of attending a major, multi-day event with my team.
• Participating in a competition that identifies the best teams playing the game
• Seeing teams you have built relationships/partnerships with over the years

Who would care most about these things but also not need to worry about costs?

Lets look as attendance costs as registration fee first. Reasonable is most likely not less that $5,000 as clearly that's whats been needed. It also says "keep reasonable" implying the $5,000 we've had is reasonable. So if the majority of teams/people answering don't need registration to stay = or < $5,000...

Lets even assume that the vast majority of teams actually have a sponsor who will upon the team qualifying pay the fee in full; money they would not have had otherwise. In that case travel costs are the only major money concern. Those costs go over $3,000 for hotels alone very easily.

Who would care that much more about the above 4 than costs staying at least where they are?

Maybe there is a flaw with ranking things as singularly more important than others in cascading order... World might be more complicated than that.

[evanperryg]

Originally Posted by BrendanB

Because it makes their case look better.

Originally Posted by PayneTrain

The survey providers' need for validation.

Originally Posted by wilsonmw04

So what this data tells me is that a small number of teams really hate this idea and are very vocal about it. This seems to jive with what typically happens here on CD.

Case in point. Angry people tend to be louder than happy people. However, Holtzman makes a very good point that we, and FIRST, should consider. The numbers show a larger negative trend than it might seem.
My problem isn't completely with the 2 championship idea. I think with the right refinements, it will work just fine. I have a problem with how FIRST is dividing the event, because, at least in my not-so-humble opinion, they are moving the wrong way. Here's a quote from Don Bossi, copied from the transcript of the 2 Champs informational session:
"At this event last year we announced that we found a way here within St. Louis
to increase that and try to get that back into the 20 percent [of all FRC teams qualifying for champs] range by going to 600 teams...
The story for FIRST® LEGO® League, FIRST® Tech Challenge, Junior FIRST® LEGO® League is
much worse. FIRST Tech Challenge has the capacity for about 3 percent of their teams at
Championship. FIRST LEGO League, it kills me when I talk to a FIRST LEGO League partner
for a country and I say, oh we can’t even send a team this year, we don’t have a slot this year."
He then discusses some thrown-out options for changing FRC. The fact that he doesn't go into detail about how changing the timing of FRC would improve qualification rates leads me to believe that the ideas he discusses wouldn't have fixed anything. As far as I'm concerned, his presentation of these ideas is little more than a way to strengthen his point through shocking the audience.
Rhetoric aside, the most viable solution to the problems experienced by FLL and FTC are most easily resolved by the one solution that was discussed for the least amount of time. Why not put FTC at one event, and FRC at another? I recognize that FIRST wants to keep their programs intertwined, going along with the "progression of programs," but by giving FTC their own world championship venue, the size and scale of the FRC venue, they will be able to boost qualification rates astronomically. An FTC/FLL championship would also permit FTC to gain its own public identity. When asked about student robotics competitions, I have never come across a non-FIRSTer who knew anything about FTC. Most of them will answer a question about student robotics with something related to "I saw one that plays basketball!" or "oh, the little lego robots, right?" Perhaps it's time FIRST allowed FTC to gain their own identity, and make their championship event into their championship event. Perhaps it would be more logical to expand FLL into a double championship format, as head-to-head competition is a small, even nonexistent part of their program. By splitting FLL, you avoid the problems with not deciding a single winner of a highly competitive program, and you offer more space for more FLL teams to qualify. By putting FRC at one event and FTC at another, you keep the 2 most competitive events together, while significantly increasing the qualification rates for FTC.

[GreyingJay]

Originally Posted by jman4747

We do know more people care about (or people care more about):

• Seeing and competing with the teams with the best robots in FRC
• The experience of attending a major, multi-day event with my team.
• Participating in a competition that identifies the best teams playing the game
• Seeing teams you have built relationships/partnerships with over the years

Who would care most about these things but also not need to worry about costs?

We worry about cost. Of course we do. But cost variability is not huge. I drive or fly to St. Louis or to Houston, or Detroit, or, heck, pick any other major city in the USA. The cost will vary plus or minus a little bit, but not that much in the long run. Not enough to rank cost above those first four items.

Now, if FIRST had announced that the championships would now be held in Paris, I might complain a little louder and say, Hey, as much as I want to see my friends, and all the best teams, I simply can't afford that. Now I have to make harder decisions about whether I want to see all the best teams or whether I settle for something smaller and closer and cheaper.

[jman4747]

Originally Posted by AdamHeard

Maybe there is a flaw with ranking things as singularly more important than others in cascading order... World might be more complicated than that.

Well it was and people answered.

Ranking things as singularly more important than others in cascading order is a very necessary tactic of survival in the very real very harsh world. People do it a lot when resources are tight. People around the world are always having to chose between things that should never be opposing each other.

This survey asked what was more important and Ranking things as singularly more important than others is a good way to draw out true motivations and see what actually can be sacrificed for what if you absolutely must chose one. Sometimes you'll find that one thing is actually not as important than another after you are forced to think about it.

A related example was when I got my second desktop I obviously thought I would need to transfer files from the old one. And there were a lot. After using the new computer for longer and longer I never did. It turned out none of that stuff was so irreplaceable or necessary. I never would have thought that while I had it and yet within a year I dissembled the hard drive to see how the insides worked. To think that everything on the hard drive was less important than my basic understanding of the engineering behind it.

That is the sort of thing "ranking things as singularly more important than others in cascading order" can sometimes have and for good benefit. Don't discount the method.

[Hot_Copper_Frog]

Taken as a whole, I think there is nothing within these survey results that is surprising. They do reinforce the idea that some within our community are strongly opposed to the two Championships concept, and that we should be using the elements identified by teams as most important to them as a guide to refining the concept to help ensure the best experience for all teams as we work through this significant change.

When I think "some", I don't think 26% and I CERTAINLY don't think 55%. This opinion is very representative of the organization as a whole. I find it interesting that it's only stated that results indicate FIRST HQ should use these elements to guide refinement of the concept, not that FIRST HQ is taking them into consideration.

Our intent is to form committees, including representatives from the community outside FIRST HQ, to make recommendations to FIRST leadership in addressing the two key challenges listed below.

• Identifying what geographic regions will be assigned to which FIRST Championship as their ‘home’ Championship, including the way in which teams outside the United States would be handled
• Identifying a potential way in which teams may volunteer and be selected to attend their non-home Championship

You will hear more about these committees over the next few months. As we noted in the Championship informational session, the facts that there will be two Championships starting in 2017, and that all FIRST programs will be represented at each Championship, will not be changing, and so won’t be part of the discussions undertaken by these groups.

So...the committees are only dedicated to identifying possible solutions to issues related to geographical assignment. And these committees will make recommendations, which may or may not be implemented. Nothing else will be addressed, as nothing else is up for discussion.

Additionally, FIRST HQ will be exploring the possibility of some culminating event to take place after the two Championships, at which we would bring together the top teams from each Championship in some final competition of the season. This idea is still in the early exploratory phase, and we will share additional details, including potential areas for community input, as appropriate.

So, in order to appease a community that is upset that their thoughts were not taken into account for such a massive change, we are going to ask the two champion alliances to participate in an afterthought showdown event. Ask mentors who have already drained most of their vacation days to take MORE time off to travel, ask students who have already missed many days of school to miss some more, and ask teams who have already spent thousands of dollars on travel to spend a few more.

I want to be excited about this. I really do. I AM excited to bring championships home to Detroit. I'm just...uncomfortable with how all of this is being handled.

[jman4747]

Originally Posted by GreyingJay

We worry about cost. Of course we do. But cost variability is not huge. I drive or fly to St. Louis or to Houston, or Detroit, or, heck, pick any other major city in the USA. The cost will vary plus or minus a little bit, but not that much in the long run. Not enough to rank cost above those first four items.

Now, if FIRST had announced that the championships would now be held in Paris, I might complain a little louder and say, Hey, as much as I want to see my friends, and all the best teams, I simply can't afford that. Now I have to make harder decisions about whether I want to see all the best teams or whether I settle for something smaller and closer and cheaper.

What that data tells me is many think current costs could change (rise) to make the top 4 options happen. This means that those individuals are very confidant that they can pay even more to get what they really value which are the top 4 options. 5k plus travel (3-4K) is a lot to some of us still...

[Drakxii]

Taken as a whole, I think there is nothing within these survey results that is surprising. They do reinforce the idea that some within our community are strongly opposed to the two Championships concept, and that we should be using the elements identified by teams as most important to them as a guide to refining the concept to help ensure the best experience for all teams as we work through this significant change.

Our intent is to form committees, including representatives from the community outside FIRST HQ, to make recommendations to FIRST leadership in addressing the two key challenges listed below.

This section continues to make me believe that frank and FRC staffers had no input in to the championsplit. Which make me sad, since this decision was based on FRC...

Also can we stop assuming what teams that didn't vote want? The sample size of this survey was 7355 people and 52% of teams. That is huge, there is no reason to assume that this not an accurate view of teams/people as whole.

Finally only 4.3 avg from students? That is disheartening, FIRST needs to remember this about the students and encouraging them, not about making everyone believe in the HQ's plan.