[FRC Blog] Two Championship Survey Results and Path Forward

[Nate Laverdure]

Originally Posted by efoote868

There are means to get a statistically accurate picture of the opinion of a community, but voluntary online survey is not one of them.

We agree. To close the loop on your earlier comments, then:

Originally Posted by efoote868

Whose opinion matters more, the 10% that took the survey or the 90% that didn't take the survey?

Of course the respondents' opinions matter more, at least in the context of providing FIRST some insight into the community's collective view of the topic at hand. The non-respondents had the opportunity to matter just as much, but chose not to use that opportunity. It's great fun to imbue those non-respondents with opinions and motivations for not sharing those opinions, but it's all fiction until measured.

Originally Posted by efoote868

Because individuals without a strong opinion do not invest their time in voluntary surveys.

Because there's no measurement to refute this, I am free to claim that the survey is biased in the opposite direction: perhaps some of those 90% were just so upset at the decision that they refused to fill out the survey. Certainly some number of those people exist-- who's to say how many?

[Drakxii]

Originally Posted by efoote868

That's a false equivalency. Responding to the survey had no guarantee of future action.

What I'm arguing is that the results of the survey mean much less than what some in this discussion are giving weight to it, and that putting the numbers in a positive or negative light doesn't matter when the numbers don't mean much.

Numbers never matter till you give them weight.

[efoote868]

Originally Posted by Nate Laverdure

Because there's no measurement to refute this, I am free to claim that the survey is biased in the opposite direction: perhaps some of those 90% were just so upset at the decision that they refused to fill out the survey. Certainly some number of those people exist-- who's to say how many?

I don't know, but voluntary online survey wouldn't be a way to find out

I'm trying to caution everyone about the limitations of the data presented here, as well as show why it doesn't matter if the response was more negative than it was portrayed by Frank. I think our collective efforts would be better to find a solution to the areas the 2 championship format is lacking.

[evanperryg]

Originally Posted by PAR_WIG1350

I have never found a study that confirms this, but I have heard it suggested that the widespread practice of using 75 as the 'center' of a 100 point grading scale in US schools has predisposed the people who attended those schools to center their rating on 75%, rather than on 50%. I feel that the most significant thing FIRST did correctly for this survey question was specifying a center, which I imagine would at least slightly help to fix that bias.

Interesting idea, and definitely something that is very easily observed in day-to-day life. However, I think we are reading into these numbers a little too far. Any kind of mapping or analysis we make is based on a limited amount of information, just the numbers in the chart and the numbers pointed out in the blog post. Our own interpretations of the data will have our own biases, and each will have an inherent flaw of some kind. I wouldn't read into those numbers a whole lot; although they are obviously skewed, it proves one important point without any special interpretation: The number of people who strongly oppose the switch account for the number of "neutral" and "strongly favor" voters combined. That says something, regardless of how this poll may or may not have been intentionally weighted in favor of the poller's preference.

Originally Posted by gblake

I suspect that the companies donating noticeable $ to FIRST will continue to let FIRST inspire students, without being the least bit interested in micromanaging how FIRST does it.

Among the members of the FIRST board of directors and executive advisory board are executives from Boeing, JCPenny, Rockwell Collins, Qualcomm, BAE Systems, Rockwell Automation, and Lego, all major contributors to FIRST and FIRST teams. Even if the biggest contributors aren't "micromanaging," they definitely have a hand in the workings of FIRST.

Regardless of whether or not the poll was weighted, I believe FIRST will take into account at least some of the complaints we have made. These sort of heated protests happen every year, with every game release. Admittedly, the restructuring of champs has a much longer-term impact on the culture of FIRST, but change had to come at some point- it was inevitable. Sure, it would have been nice to know there were talks about major changes to the championship structure coming soon, but it's not like they didn't tell us something was going to change back in 2012. Again, it would have been nice to get some more specific info before the announcement, but it's not like they never said anything. At the end of they day, no matter how much we analyze, map, or dissect this poll, FIRST is going to change, and it has to change in order to become a universally-recognized program.

[Citrus Dad]

Originally Posted by waialua359

The point that FIRST is growing and yet wants to give the same % of students the Championship experience, will not find a happy medium to address having one set of Champions and everyone playing under the same roof. Too many pros AND cons. Is it really too late or impossible to find a venue 2020 and beyond that can hold 800 teams?

Solutions to having a single champion while still having 2 events have been posted here. It's just that FIRST HQ isn't interested in entertaining any real feasible alternatives. I'll speculate about the reason for that in a separate post.

[Citrus Dad]

Originally Posted by grstex

But you can't say "62.5% of respondents oppose the split." That's just not true. the "mandate" is that 55% oppose the split. you CAN'T just discard 12% of the responses. That's more misleading than average from the blog.

I approached this issue as a presidential election. You can't vote for "neither of the above" or "both of the above." You have to choose. The "5s" refused to choose. In an election, those folks don't vote--it's a very common assumption by pollsters making projections for election results.

Similarly, we don't apply an intensity of like or dislike to presidential candidates. It's either "A" or "B". There's some indication in 2012 that Romney supporters were more intense in their positions, but there were fewer of them. Ultimately, I believe we should really care about which side people fall on.

One other polling note: while this is a voluntary poll so it could be biased, pollsters find that usually the opinions of respondents generally reflect the views of non respondents.

I used a set of common polling assumptions to provide a clearer view of how community preferences fall out. I see others have provided other metrics that arrive at the same conclusion--that opposition is running 2 to 1 against.

[Citrus Dad]

Originally Posted by Basel A

I don't think you can look at this data and reasonably say "most of FRC is opposed to two Championships," especially when the nonvoters likely don't care/are neutral.

Given the low turnout for elections in the US, that argument would lead to us to the conclusion that we haven't actually elected anyone. The fact is that elections and decisions are determined by those who care enough to respond. If we want a democratic process, allowing indifference to have weight becomes an overwhelming defeating burden.

[grstex]

Originally Posted by Citrus Dad

I approached this issue as a presidential election. You can't vote for "neither of the above" or "both of the above." You have to choose. The "5s" refused to choose. In an election, those folks don't vote--it's a very common assumption by pollsters making projections for election results.

Similarly, we don't apply an intensity of like or dislike to presidential candidates. It's either "A" or "B". There's some indication in 2012 that Romney supporters were more intense in their positions, but there were fewer of them. Ultimately, I believe we should really care about which side people fall on.

One other polling note: while this is a voluntary poll so it could be biased, pollsters find that usually the opinions of respondents generally reflect the views of non respondents.

I used a set of common polling assumptions to provide a clearer view of how community preferences fall out. I see others have provided other metrics that arrive at the same conclusion--that opposition is running 2 to 1 against.

There is such a thing as a 3rd party candidate. Ross Perot captured almost 19% of the popular vote in 1992, and over 8% in 1996. in 1928 Robert La Follette even won a state. (of course, you could hold the Kang and Kodos perspective of politics)

But more importantly, in this survey you DID NOT HAVE TO CHOOSE. You WERE GIVEN A NEUTRAL OPTION. 12% chose that option. FIRST could have structured the survey as a simple for or against, but they didn't. As others in this thread have already stated, the people who chose neutral did so for a reason. Their response counts too.

[gblake]

Originally Posted by evanperryg

...
Among the members of the FIRST board of directors and executive advisory board are executives from Boeing, JCPenny, Rockwell Collins, Qualcomm, BAE Systems, Rockwell Automation, and Lego, all major contributors to FIRST and FIRST teams. Even if the biggest contributors aren't "micromanaging," they definitely have a hand in the workings of FIRST.
...

I realize that you didn't disagree with me, so please notice in return that I'm not disagreeing with you.

Members of the FIRST Board of Directors, when they are carrying out their duties as Board Members, are not supposed to let their duties as members of any other organization bias them (their life experiences should give them wisdom that helps "inform" their decisions; but when they are carrying out board business, they are carrying out FIRST business, not the business of any other entity).

And, when they take their FIRST BoD hats off, those folks have bigger fish to fry, in their primary jobs.

Advising, setting goals, and contributing to high-level policy/strategy is what a good Board does, micromanaging is what a good board doesn't do.

Before we go off on a tangent - I'll claim that debating in CD whether the Championsplit is high-level policy, or a lower-level detail, won't be useful. If there is any confusion about that among the BoD members, or among the people who report to the BoD, they will straighten it out, on their own.

Blake

[gblake]

Originally Posted by Citrus Dad

...
The "5s" refused to choose.
...

No, they told you and anyone else who looked at their votes, that they were neutral.

Neutral could and does mean many things.

It is simply incorrect to distill it down to a "refused to choose" sound bite.

Blake

[Basel A]

Originally Posted by Citrus Dad

I approached this issue as a presidential election. You can't vote for "neither of the above" or "both of the above." You have to choose. The "5s" refused to choose. In an election, those folks don't vote--it's a very common assumption by pollsters making projections for election results.

This is absolutely nothing like a presidential election. That is a choice between two people, whereas this is a statement of approval/disapproval of a policy. A good example of what that should look like is here. Note how the approval and disapproval numbers don't add up to 100%. That's because some people are neutral. Here's another potentially enlightening link: http://en.wikipedia.org/wiki/Self-selection_bias

[PAR_WIG1350]

Originally Posted by gblake

No, they told you and anyone else who looked at their votes, that they were neutral.

Neutral could and does mean many things.

It is simply incorrect to distill it down to a "refused to choose" sound bite.

Blake

Regardless of how neutral responses should be counted, the reality is they are probably going to be counted as being in favor of FIRST's proposal.

The reason for this is simple: anybody who is neutral will be just as supportive whether or not FIRST reverses its decision, and FIRST is quite sure that it wants what it said it wants.

Thus, from FIRST's perspective, the results of the poll are 55-45 against the proposal. Given the small sample size, this is probably close enough to 50-50 for an entity with even a slight confirmation bias to say that the community is largely undecided.

So, in reality, the survey gave you 4 options to say that you were against the proposal, and 6 options to say you were for it.

[northstardon]

I voted 5.

It was not because I didn't care one way or the other. It was because, "I haven't decided yet" wasn't an option.

The survey went out a week before Champs, and a day or two after it was announced that there was going to be a Town Hall meeting in St. Louis. I wanted to wait until after the Town Hall and after more data/information was made available before voicing an opinion. And that's exactly what I said in the comment box beneath my response.

[EricH]

Originally Posted by northstardon

I voted 5.

It was not because I didn't care one way or the other. It was because, "I haven't decided yet" wasn't an option.

The survey went out a week before Champs, and a day or two after it was announced that there was going to be a Town Hall meeting in St. Louis. I wanted to wait until after the Town Hall and after more data/information was made available before voicing an opinion. And that's exactly what I said in the comment box beneath my response.

Not specifically aimed at you, but you happened to be handy.

For all those who voted 5, would you care to share "undecided (at this time)", "withholding judgement", or "don't care" status at the time of the survey, and have you changed your response at this point?


Full disclosure: I didn't fill out the survey. If I had to respond, I would be leaning in about the 4-5 range: I don't like it, but I think there's enough room to improve (in a variety of ways) that I could be persuaded to go the other way. I could also end up working my way down towards the 1s and 2s, if that improvement doesn't go the way of improvement.



One thing that's finally gotten through to me: Any way you slice the data, if the average is less than 5.5 on this scale, you ain't winnin' no election. Just the way the scale works. The fact that they're trying to SPIN it... Sorry, Frank, but sometimes you gotta bite the bullet! This isn't a popular decision, and not even by going by team number with the data are you going to be able to change that! "There are lies, d****d lies, and statistics."--attributed to Mark Twain.

[Taylor]

Originally Posted by EricH

For all those who voted 5, would you care to share "undecided (at this time)", "withholding judgement", or "don't care" status at the time of the survey, and have you changed your response at this point?

I voted 5. As of right now, I have a very strong opinion on the subject, but I am not sure what it is.
I can tell you it's certainly not "don't care" - probably more along the lines of "withholding judgement".
I could be swayed either way; however, I do have a voice and I'd like it to count.
To me, a 5 is telling HQ "If you play this right, I could jump on board. If not, well, add me to the disgruntled California teams."

If you lump 4s, 5s, and 6s together, that's a quarter of respondents who may think likewise.
I'm comfortable speaking on behalf of them in saying we didn't 'throw away our votes'