[FRC Blog] Two Championship Survey Results and Path Forward

[Siri]

http://www.usfirst.org/roboticsprogr...d-path-forward

As many of you know, after we announced that FIRST would be moving to having two Championships starting in 2017, we released a survey to FIRST Robotics Competition teams on this change.

7,355 individuals responded. This represents about 10% or so of the total number of mentors and students we have in FRC. About 75% of the 7,355 provided a team number, and those respondents were from 1,501 teams, about 52% of the total number of FRC teams we had in 2015.

One question we asked was just a simple ‘How do you feel about having two FIRST Championships starting in 2017?’ with an answer of 1 representing ‘Strongly Oppose’, an answer of 5 identified as “Neither Oppose nor Favor” and an answer of 10 identified as “Strongly Favor”. The average answer to this questions among all respondents was 4.45, somewhat below the 5 "Neither Oppose nor Favor" rating. Here is a graph of the full results:

We conducted additional analysis to better understand the rating score. FRC Teams that had never attended Championship had an average rating on this question of 5.85, while teams that had attended Championship had an average rating of 4.05. Mentors and Students had ratings relatively close to each other, at 4.54 and 4.30, respectively. When we determined average ratings from individual teams, then averaged those ratings (essentially giving every team one ‘vote’, and assuming the average rating of all team respondents was the one ‘vote’ for the team), the average rating was 5.2.*

We also asked teams what elements of the ‘Championship Experience’ were most important to them. There were 22 elements to choose from. Teams identified these as their top 10, in order from most to least important:

• Seeing and competing with the teams with the best robots in FRC
• The experience of attending a major, multi-day event with my team.
• Participating in a competition that identifies the best teams playing the game
• Seeing teams you have built relationships/partnerships with over the years
• Keeping attendance costs reasonable
• Participating in a very large scale event with tens of thousands of others
• Seeing and meeting international teams
• Participating in a competition that identifies the teams most deserving of awards, such as the Chairman's award
• Seeing and meeting top teams, like prior Chairman's Award winners (Hall of Fame teams)
• Having your matches in an impressive, large scale space

Taken as a whole, I think there is nothing within these survey results that is surprising. They do reinforce the idea that some within our community are strongly opposed to the two Championships concept, and that we should be using the elements identified by teams as most important to them as a guide to refining the concept to help ensure the best experience for all teams as we work through this significant change.

Our intent is to form committees, including representatives from the community outside FIRST HQ, to make recommendations to FIRST leadership in addressing the two key challenges listed below.

• Identifying what geographic regions will be assigned to which FIRST Championship as their ‘home’ Championship, including the way in which teams outside the United States would be handled
• Identifying a potential way in which teams may volunteer and be selected to attend their non-home Championship

You will hear more about these committees over the next few months. As we noted in the Championship informational session, the facts that there will be two Championships starting in 2017, and that all FIRST programs will be represented at each Championship, will not be changing, and so won’t be part of the discussions undertaken by these groups.

Additionally, FIRST HQ will be exploring the possibility of some culminating event to take place after the two Championships, at which we would bring together the top teams from each Championship in some final competition of the season. This idea is still in the early exploratory phase, and we will share additional details, including potential areas for community input, as appropriate.

Frank

*The analysis of average ratings from individual teams was completed by our Director of Research and Evaluation. She was the only one to see individual average team ratings, and after completing the analysis, deleted the team numbers from the data set.

[Steven Donow]

That's actually a surprisingly large amount of people filling out the survey (I've always wondered how many people fill out the surveys; I generally fill out most of the offseason ones and occasionally two or three of the weekly ones).

But srsly FIRST has made it clear since the town hall meeting that they have absolutely zero interest in listening to the community and doing anything other than two geographically split championship events, as opposed to two championship events divided in some other way*


*as much as I love these proposals, I just have so much hesitance over them because of teams possibly having to 'shuffle' between championships (ie. finalist at a week 1 event, then winning a week 6)

[Madison]

What is the perceived value in the idea of one team, one vote?

[BrendanB]

Originally Posted by Madison

What is the perceived value in the idea of one team, one vote?

Because it makes their case look better.

[PayneTrain]

Originally Posted by Madison

What is the perceived value in the idea of one team, one vote?

The survey providers' need for validation.

[Taylor]

Engineers love data.

Originally Posted by Madison

What is the perceived value in the idea of one team, one vote?

Originally Posted by BrendanB

Because it makes their case look better.

Originally Posted by PayneTrain

The survey providers' need for validation.

Raar. Too much data. By being exploratory, they're manipulating it!

[/s]

[PayneTrain]

Originally Posted by Taylor

Engineers love data.





Raar. Too much data. By being exploratory, they're manipulating it!

[/s]

The survey itself was pretty manipulative. The options for the response would bias toward a positive trend, and options on why championships are important to teams were presented in a way that would present responses that would invalidate proposals like the festival/champs split.

But since you earned it:

[NotInControl]

FIRST this is a total failure in my opinion.

I stated this to some of the FIRST HQ representatives while at champs this year who were telling me to go fill out the survey.

What is the point of even having a survey, if contracts were signed for the venues before the survey or even this idea was announced?

The survey would just capture how the community felt about the decision, not influence the decision at all.

Furthermore, when going to two championships, I've heard speculation of possibly holding an official event where both championships alliances compete to have 1 world championship, to possibly rectify a majority of concerns.

In my opinion, this doesn't rectify anything, and in fact unless FRC picks up the travel bill for the teams, it punishes the winning alliances because their season is now extended, and they will most likely need to re-travel to wherever this new event takes place.

2016, were going to be doing everything possible to earn our way to championships, its going to be the end of an era.

P.S. I did fill out the survey even though I knew it had no real effect on the decision. And this data looks manipulated to the point of uselessness.

[Gregor]

The survey results were very surprisingly close to neutral, with the average response just under 5.

I was expecting a stronger bias.

[Holtzman]

Originally Posted by Gregor

The survey results were very surprisingly close to neutral, with the average response just under 5.

I was expecting a stronger bias.

What is a little misleading about the results is the survey scale. On a scale from 1-10 with 5 being neutral, we have 4 options that are negative, one that's neutral, and 5 options that are positive. This skews the data on the positive side. The average answer of 4.45 is misleading since the left and right sides of the data set have different weights.

Another way of looking at these results is that 55% oppose two championships, 12% are neutral, and only 33% favor two championships. To me, that’s is a much more powerful statement about how the community really feels.

[marshall]

Originally Posted by Holtzman

What is a little misleading about the results is the survey scale. On a scale from 1-10 with 5 being neutral, we have 4 options that are negative, one that's neutral, and 5 options that are positive. This skews the data on the positive side. The average answer of 4.45 is misleading since the left and right sides of the data set have different weights.

Another way of looking at these results is that 55% oppose two championships, 12% are neutral, and only 33% favor two championships. To me, that’s is a much more powerful statement about how the community really feels.

I was literally just saying this. The scale is really not right.

[Michael Blake]

Originally Posted by Holtzman

Another way of looking at these results is that 55% oppose two championships, 12% are neutral, and only 33% favor two championships. To me, that’s is a much more powerful statement about how the community really feels.

BAZINGA.

And... it would be interesting to know of the 33% who favor two championships what percentage only competes at ONE competition?

[northstardon]

My beef with the "Favor/Oppose 2 Championships" question is that it was asked without context (or, more charitably, that it was asked assuming that the respondents all knew that context). It's like asking people if they favor/oppose eating their vegetables.

Better questions might have been:

"Do you favor/oppose two championships, if FIRST brought the two winning alliances together to crown one true champion?"

or,

"Do you favor/oppose two championships that are tiered, with all of the highest-ranked, most competitive robots attending one of the two events to determine the one true champion?"

or,

"The highest number of teams that could be accommodated by a single championship event is 650. The total number of FRC teams is increasing each year. Do you favor/oppose two championships, knowing that the number of teams attending a single championship each year will (on a percentage basis) continue to decrease?"

[Monochron]

Originally Posted by Holtzman

What is a little misleading about the results is the survey scale. On a scale from 1-10 with 5 being neutral, we have 4 options that are negative, one that's neutral, and 5 options that are positive. This skews the data on the positive side. The average answer of 4.45 is misleading since the left and right sides of the data set have different weights.

Another way of looking at these results is that 55% oppose two championships, 12% are neutral, and only 33% favor two championships. To me, that’s is a much more powerful statement about how the community really feels.

I have read all the posts on this thread, but I'm hoping that this point got a lot more attention than it did on the first page. The way FIRST collected data and they way they have presented it in this blog post is legitimately very misleading. The number of negative options versus the number of positive options really rubs me the wrong way.

[Steven Donow]

I wish they gave actual analysis based off team number. Not even to show that,'this elite team felt this way!' but just to see how votes were split across team age, events attended, teams that have been to champs vs teams that haven't, etc... Then we'd be able to fully understand what the demographic reach that responded to the survey was.