[FRC Blog] Two Championship Survey Results and Path Forward

[Holtzman]

Originally Posted by Gregor

The survey results were very surprisingly close to neutral, with the average response just under 5.

I was expecting a stronger bias.

What is a little misleading about the results is the survey scale. On a scale from 1-10 with 5 being neutral, we have 4 options that are negative, one that's neutral, and 5 options that are positive. This skews the data on the positive side. The average answer of 4.45 is misleading since the left and right sides of the data set have different weights.

Another way of looking at these results is that 55% oppose two championships, 12% are neutral, and only 33% favor two championships. To me, that’s is a much more powerful statement about how the community really feels.

[marshall]

Originally Posted by Holtzman

What is a little misleading about the results is the survey scale. On a scale from 1-10 with 5 being neutral, we have 4 options that are negative, one that's neutral, and 5 options that are positive. This skews the data on the positive side. The average answer of 4.45 is misleading since the left and right sides of the data set have different weights.

Another way of looking at these results is that 55% oppose two championships, 12% are neutral, and only 33% favor two championships. To me, that’s is a much more powerful statement about how the community really feels.

I was literally just saying this. The scale is really not right.

[Michael Blake]

Originally Posted by Holtzman

Another way of looking at these results is that 55% oppose two championships, 12% are neutral, and only 33% favor two championships. To me, that’s is a much more powerful statement about how the community really feels.

BAZINGA.

And... it would be interesting to know of the 33% who favor two championships what percentage only competes at ONE competition?

[northstardon]

My beef with the "Favor/Oppose 2 Championships" question is that it was asked without context (or, more charitably, that it was asked assuming that the respondents all knew that context). It's like asking people if they favor/oppose eating their vegetables.

Better questions might have been:

"Do you favor/oppose two championships, if FIRST brought the two winning alliances together to crown one true champion?"

or,

"Do you favor/oppose two championships that are tiered, with all of the highest-ranked, most competitive robots attending one of the two events to determine the one true champion?"

or,

"The highest number of teams that could be accommodated by a single championship event is 650. The total number of FRC teams is increasing each year. Do you favor/oppose two championships, knowing that the number of teams attending a single championship each year will (on a percentage basis) continue to decrease?"

[Monochron]

Originally Posted by Holtzman

What is a little misleading about the results is the survey scale. On a scale from 1-10 with 5 being neutral, we have 4 options that are negative, one that's neutral, and 5 options that are positive. This skews the data on the positive side. The average answer of 4.45 is misleading since the left and right sides of the data set have different weights.

Another way of looking at these results is that 55% oppose two championships, 12% are neutral, and only 33% favor two championships. To me, that’s is a much more powerful statement about how the community really feels.

I have read all the posts on this thread, but I'm hoping that this point got a lot more attention than it did on the first page. The way FIRST collected data and they way they have presented it in this blog post is legitimately very misleading. The number of negative options versus the number of positive options really rubs me the wrong way.

[Steven Donow]

I wish they gave actual analysis based off team number. Not even to show that,'this elite team felt this way!' but just to see how votes were split across team age, events attended, teams that have been to champs vs teams that haven't, etc... Then we'd be able to fully understand what the demographic reach that responded to the survey was.

[efoote868]

Originally Posted by Steven Donow

I wish they gave actual analysis based off team number. Not even to show that,'this elite team felt this way!' but just to see how votes were split across team age, events attended, teams that have been to champs vs teams that haven't, etc... Then we'd be able to fully understand what the demographic reach that responded to the survey was.

What safeguards were in place to stop individuals from falsely identifying the wrong team? What about stop them from filling out multiple surveys? Prevent FIRST unaffiliated or unaffected from responding?

Why should we trust these results as anything more than a voluntary online survey?

[Drakxii]

Originally Posted by efoote868

What safeguards were in place to stop individuals from falsely identifying the wrong team? What about stop them from filling out multiple surveys? Prevent FIRST unaffiliated or unaffected from responding?

Why should we trust these results as anything more than a voluntary online survey?

Why do you suspect that these results do not match the views of FRC teams?

[efoote868]

Originally Posted by Drakxii

Why do you suspect that these results do not match the views of FRC teams?

Because individuals without a strong opinion do not invest their time in voluntary surveys.

[scottandme]

Originally Posted by efoote868

Because individuals without a strong opinion do not invest their time in voluntary surveys.

I fill out every FRC (and MAR) survey that I get. Lots of those are pretty mundane, but I do them anyway since they take all of 5 minutes, and (presumably) give useful feedback to the respective groups looking at the data.

If that was actually the case - there should be a lot more 1's and 10's in the results.

[efoote868]

Originally Posted by scottandme

I fill out every FRC (and MAR) survey that I get. Lots of those are pretty mundane, but I do them anyway since they take all of 5 minutes, and (presumably) give useful feedback to the respective groups looking at the data.

If that was actually the case - there should be a lot more 1's and 10's in the results.

38% of the response was 1's and 10's.... that's very bi-polar.

[dodar]

Originally Posted by efoote868

38% of the response was 1's and 10's.... that's very bi-polar.

And that seems odd to you? Were you not here for the original thread and saw how bi-polar the CD community was?

[efoote868]

Originally Posted by dodar

And that seems odd to you? Were you not here for the original thread and saw how bi-polar the CD community was?

A public forum suffers from the same response bias that an online voluntary survey suffers.

[Nate Laverdure]

Originally Posted by efoote868

A public forum suffers from the same response bias that an online voluntary survey suffers.

Why cater to an opinion that doesn't care to reveal itself in a measurable way?

[efoote868]

Originally Posted by Nate Laverdure

Why cater to an opinion that doesn't care to reveal itself in a measurable way?

Whose opinion matters more, the 10% that took the survey or the 90% that didn't take the survey?

If the survey accurately reflected the opinion of everyone in FIRST, one might expect that 26% of the FIRST population would quit in the near term. And one might be shocked to find that only 4% quit instead, and that the 90% of the community that didn't respond to the survey fell between "mildly dislike to mildly like, mostly don't care."