[FRC Blog] Stop Build Day Survey Results

[Knufire]

Posted on the FRC Blog, 10/21/16: http://www.firstinspires.org/robotic...survey-results

Stop Build Day Survey Results
Written by Frank Merrick.

In September, as we announced here, we surveyed our community on their thoughts about Stop Build Day. The intent was to gather data in support of our further discussions on this topic. And as we had clarified before, other than switching to a worldwide, simultaneous stop build time, there are no Stop Build Day changes for the 2017 season. Any significant changes we would make would be for 2018 or later, as we want to give this topic very careful consideration, and announce changes (if there will be any) long before the season starts so teams can make plans.

We are presenting these results without editorial comment. As we are in the middle of season prep right now and have a very busy season right on the horizon, we are not likely to be able to turn our attention back to this matter in the short term.

You can see the very long and detailed report here. You will see that our Research and Evaluation team sliced and diced the responses in many different ways. If the thought of reading through 16 pages of information with hundreds of numbers makes your head spin, here is a brief summary:

• We had 9,286 responses total, though of course not all respondents made it to the end of the survey or answered every single question.
  
  
• Those responses were from 2,196 unique teams.
  
  
• Graph of full results to the question about what would be best for the FIRST Robotics Competition overall (7,507 responses):
  
  
  
  
• Graph of full results to the question about what would be best for ensuring equity and inclusion of students and schools in the FIRST Robotics Competition program (7,470 responses):
  
  
  
  
• The two largest groups of respondents were Mentors, at 50.8% of respondents, and Students, at 36.7% of respondents.
  
  
• Graph of results of 'best for program overall' question, Mentors only (4,130 responses):
  
  
  
  
• Graph of results of 'best for program overall' question, Students only (2,554 responses):
  
  

Finally, the 'Averaging Responses by Team' data in the detailed report probably needs a bit of explanation. We wanted to see what the results would be like if every team that responded essentially got one 'vote', and that vote was the average of all the individual responses from that one team. You can compare this to the Senate in the United States, in which every state gets two votes, regardless of state population. As this process naturally resulted in many fractional votes, we wanted to break the results up more finely, as rounding the results to the nearest whole number could have given an inaccurate impression of the underlying information.

I hope you find this data interesting. As I say above, we won't be spending much time on this topic in the short term, as we have an awesome FIRST STEAMWORKS season to get ready for!

Frank

[mlantry]

Is Hallry alive?

[Jon K.]

I would be interested in seeing the results of the survey of time in the program vs the keep vs stop data similar to how they did team age vs the data. I wonder if it is still as evenly distributed.

[tickspe15]

This survey was clearly rigged.

It'd be interesting to see how metrics such as OPR or BBQ influenced survey responses.

[SenorZ]

Keeping or eliminating "stop" build day was a bit simplistic.

There should have been multiple choices, ranked by preference:

• Keep things as they are
• Continue Kickoff in January but eliminate SBD
• Move Kickoff to an earlier date and keep SBD in February (longer season with a hard stop)
• Pull a VEX, and have months to work, including all the way to the event.

...etc.

But just dealing with this one questions is sure to cause them huge headaches.

[marshall]

This is at the top of the report:

61.5% of all respondents said their team builds an additional robot or robots, in addition to the robot that gets bagged. 50.2% of teams said their team builds an additional robot or robots, in addition to the robot that gets bagged.

So a majority of responding teams (2,196) build more than one robot...

[Mark McLeod]

I would call it half, not a majority.
4 teams out of 2196 isn't statistically significant.

[marshall]

Originally Posted by Mark McLeod

I would call it half, not a majority.
4 teams out of 2196 isn't statistically significant.

Fair point. It's still a surprising number to me. I'm glad they asked the question and we know the results.

[headlight]

If I'm reading this correctly, there is a somewhat significant portion of respondents in most categories who think that stopping bag day would be "best for ensuring equity and inclusion of students and schools in the FIRST
Robotics Competition program", but for some reason don't think that it would be "Best for the FIRST Robotics Competition program".

Is the equality and inclusion of students and schools undervalued or is there some other priority?

[AdamHeard]

Even if the survey had said everyone was in favor of eliminating SBD, I wouldn't trust the results.

The survey phrasing was pretty darn awful in terms of phrasing.

It also would have been helpful to have a panel of consulting mentors or something present 3-5 options to FIRST. FIRST would critique these, then present them to the masses with some education of their details.

I bet everyone had different interpretations of what their answers mean.

Weekly unbag windows seems like a pretty darn good compromise.

[Jared Russell]

Originally Posted by AdamHeard

Weekly unbag windows seems like a pretty darn good compromise.

Agreed. This would be a relatively non-controversial change; one that simply gives the advantage that district teams already enjoy (especially those who attend extra events) to everybody.

Even if you have gripes with the survey methodology and put large error bars on each bar in the graph, I'd wager that ~90% of those in the "1: Eliminate Stop Build Day" camp would think this is an improvement on the status quo, and probably 1/3 to 1/2 of those in the "5: Keep Stop Build Day" category would agree. Even if the survey is biased by a factor of 2x in either direction, it adds up to consensus.

[Nate Laverdure]

Originally Posted by AdamHeard

The phrasing was pretty darn awful in terms of phrasing.

Alanis Morissette-style irony?

[Caleb Sykes]

Originally Posted by AdamHeard

Even if the survey had said everyone was in favor of eliminating SBD, I wouldn't trust the results.

The survey phrasing was pretty darn awful in terms of phrasing.

It also would have been helpful to have a panel of consulting mentors or something present 3-5 options to FIRST. FIRST would critique these, then present them to the masses with some education of their details.

I bet everyone had different interpretations of what their answers mean.

Weekly unbag windows seems like a pretty darn good compromise.

This post's accuracy is pretty darn good in terms of accuracy.

[PayneTrain]

Originally Posted by AdamHeard

It also would have been helpful to have a panel of consulting mentors or something present 3-5 options to FIRST. FIRST would critique these, then present them to the masses with some education of their details.

I bet everyone had different interpretations of what their answers mean.

Weekly unbag windows seems like a pretty darn good compromise.

Yeah Adam but just like they didn't have a baked-in focus group of teams for split postseason events they don't have a baked-in in focus group of mentors for this

[PayneTrain]

You can say that the plurality-winning opinion is keeping sbd, but you will also have to point out that a supermajority of respondents think there should be some kind of change to the existing rules.