The 6 Week Build Season and 'Mentor Burnout'

[Jim Zondag]

In 2013, despite all the successes of all the teams represented on these forums, the Median OPR of the league after 1 event played was 10.3.
Over 1200 teams had a net contribution of 10 points or less per match. This level of accomplishment could be achieved in Ultimate Ascent by simply building a kit chassis with two stationary hooks on top. Since HALF of the league cannot achieve this basic level of play in their first outing, it is very hard for me to understand statements that say that we are giving teams enough time to be successful. We are not. 13% of the league has a sub-zero OPR at their first event. This means that our league is producing over 300 teams per year who NEVER EVEN SCORE AT ALL in their first event. This is a design failure of our system. (and I doubt than any of those teams are represented here).

Sure, many of these teams would still not be successful if we gave them more time, but many of them would get much better. Look at the data. After two outings the median MORE THAN DOUBLES to 24 points per team per match. This is huge. The average of the top teams does not change very much, but number of negative almost completely disappears.

What this is telling us is that with some time to work on their machines, get on a playing field, get help from other teams, and benchmark other's solutions,
HUNDREDS of teams move from a position of NOT BEING ABLE TO PLAY, to a position of BEING ABLE TO POSITIVELY CONTRIBUTE to their alliance's success. However, since 1400 teams only played once this year; many, many teams never get this opportunty to improve. And because of this, many of them fail and do not return to the FRC. And this is the real problem. The most difficult place to start a FIRST team is at a school where they previously decide to quit FRC in the past. Our current system is somewhat designed to kill teams early in their life cycle.



These trends are essentially the same, year after year after year, regardless of game design. When registration closes next fall, take a look at where all of the teams who drop out of FRC fall on these charts. The key to improving the sustainablity of FRC is to get teams to a reasonable success plateau early in their team history, ideally in their first season. Then they are likely to stay. Right now the FRC system works against this goal in many ways.

Some of the people on this post say they may quit if the rules were changed, however we already have a system which actively kills teams by the hundreds each year. Which is worse?

[s_forbes]

Originally Posted by Jim Zondag

... HUNDREDS of teams move from a position of NOT BEING ABLE TO PLAY, to a position of BEING ABLE TO POSITIVELY CONTRIBUTE to their alliance's success. However, since 1400 teams only played once this year; many, many teams never get this opportunty to improve.

I think this may be a misleading graphical representation of the time input vs performance for the reason you stated here. I'd suspect many teams that put in the effort to raise the funds to attend two regionals also put in more effort to build a robot that performs well. Their performance isn't based on playing time at regional events, there's just a correlation between the effort spent building a high performance robot and the effort spent raising funds for multiple regionals. I do agree that robot performance increases if a team has the opportunity to attend multiple regionals, however, and also agree with the point you're making.

With the way our team operates during the build season, I would prefer to have no bag day. We build a practice robot to get more tweaking time in, and in doing so we waste a lot of man-hours and resources just to build the second robot. With no bag day, and no need for a practice robot, we would be expending fewer resources to get the same performance out of our competition robot (which would allow us to take more days off during build season).

With the amount we spent doing fast R&D and building a practice robot to work around the bag deadline, we could have potentially gone to another regional instead (if it was priced similar to district events).

[Ian Curtis]

Originally Posted by s_forbes

I think this may be a misleading graphical representation of the time input vs performance for the reason you stated here. I'd suspect many teams that put in the effort to raise the funds to attend two regionals also put in more effort to build a robot that performs well. Their performance isn't based on playing time at regional events, there's just a correlation between the effort spent building a high performance robot and the effort spent raising funds for multiple regionals. I do agree that robot performance increases if a team has the opportunity to attend multiple regionals, however, and also agree with the point you're making.

Jim, can you do the same thing comparing 1st and 2nd event just Michigan and MAR teams? I have to imagine it shows the same trend.

Originally Posted by Jim Zondag

So ask yourself, are we really doing something smart with the machine access rules, or is all of this just a collection of old rules, imposed for obsolete reasons which have been forgotten?

Originally Posted by Jim Zondag

Some of the people on this post say they may quit if the rules were changed, however we already have a system which actively kills teams by the hundreds each year. Which is worse?

And if we do it for a season and it does really end the world as we know it... who says we can't go back?

Originally Posted by Tetraman

Open build would give too much advantage to teams with the capability to travel long distances to their events or have events nearby for attending both a week 1-3 and week 4-6.

I absolutely disagree. Closed build season gives the 2 event team at least 8 matches on the real field, plus a few practice matches, PLUS 3 days to work on their robot for their event registration. Meanwhile a team that is not competing does not get to practice, AND gets no time to work on their robot. Multi-event teams HAVE a huge advantage as is. Instead of leaving 1 event teams to be sitting ducks, moving to an open build season at least lets them learn, make changes, and most importantly practice like multi-event teams.

[Tristan Lall]

A couple of methodological points, just to be rigorous about this:

Originally Posted by Jim Zondag

Over 1200 teams had a net contribution of 10 points or less per match.
[...]
13% of the league has a sub-zero OPR at their first event. This means that our league is producing over 300 teams per year who NEVER EVEN SCORE AT ALL in their first event.

How are fouls being accounted for, if at all? As negative points? Could a portion of those 300 therefore be scoring fewer points than they receive in fouls? (This is of course still awful, but it doesn't mean they didn't score.)

Originally Posted by Jim Zondag

Sure, many of these teams would still not be successful if we gave them more time, but many of them would get much better. Look at the data. After two outings the median MORE THAN DOUBLES to 24 points per team per match. This is huge. The average of the top teams does not change very much, but number of negative almost completely disappears.

Isn't that confounded by the likelihood that many of the worst teams only participate in one event (and thus their badness isn't reflected in the event 2 statistics)?

To measure the actual improvement, we'd want to separate out the event 1 performance of teams that participate in 2 or more events, and compare that to those teams' 2nd events. (And even after doing that, you wouldn't expect this 2-or-more-week group to be representative of the 1-week teams, because attendance at a second event—especially outside of district play—is probably strongly correlated with greater resources and organization.)

Originally Posted by Jim Zondag

Our current system is somewhat designed to kill teams early in their life cycle.

I think "designed" is probably too strong a characterization. It's nevertheless completely fair to say that the system is pretty good at it.

[Nemo]

Originally Posted by Tristan Lall

Isn't that confounded by the likelihood that many of the worst teams only participate in one event (and thus their badness isn't reflected in the event 2 statistics)?

To measure the actual improvement, we'd want to separate out the event 1 performance of teams that participate in 2 or more events, and compare that to those teams' 2nd events. (And even after doing that, you wouldn't expect this 2-or-more-week group to be representative of the 1-week teams, because attendance at a second event—especially outside of district play—is probably strongly correlated with greater resources and organization.)

Here's just the teams that attended 2 or more events.

[Mr. Van]

Jim brings up some valid points. Obviously teams that are not successful are not inspired to continue, but I don't believe that extending the build season will help these teams to the extent hoped. The reasons a team is not able to build a functioning robot (a drivable base with some articulated hooks on it, for example) are many and varied, but I would argue that a build season that is too short is not one of them.

One of the major reasons is that teams perpetually "bite off more than they can chew". Another is that they refuse to use items like the kit-bot chassis, or they may simply not have the technical skills necessary to build a working robot at all. These problems will happen if the season is opened-ended or if there is a time limit, unless the teams have better mentorship.

As Jim notes, real improvements occur when teams get on an official game field, play against other teams and gain the experience of driving a robot in a competition. It is a real shame of our system is that half of the teams are "one-timers". At least we agree on that! It seems that the district system will help in that basic registration includes two events, but overall, this is a major problem.

To me, however, the solution to improving robot (and therefore team) performance is to improve mentorship and provide more opportunity to actually play the game. This does not require a longer build season.

I doubt that the CD community will be able to come to any sort of consensus on this issue, but I'm glad that we've brought this question up for debate.

- Mr. Van
Coach, Robodox

[Siri]

First, Jim, thank you very much. This offers very, very valuable perspective on the matter that we've only really alluded to thus far. I'd look forward to any MAR/FiM exclusive data you might have. It might not be as dramatic--among other causes, they're quite competitive regions, both due to and resulting in the district model--but I'd bet it's something.

Originally Posted by Tetraman

Open build would give too much advantage to teams with the capability to travel long distances to their events or have events nearby for attending both a week 1-3 and week 4-6.

I would hope that in many if not most places, an open season would make FRC practice more like FLL., FTC and VEX: scrimmages aren't for "off-season". If some off-seasons moved to off-event weeks (in their area) during competition season, teams wouldn't have to travel so far or pay so much just to play on an at least semi-real field with other teams. (Semi-real given the logistics of getting a real field in competition season.) It might make easier replicablity/shippablity a constraint on the FRC GDC the way it is on the others--but heck, shouldn't it be anyway? It's certainly well past worth it.

Originally Posted by Mr. Van

One of the major reasons is that teams perpetually "bite off more than they can chew". Another is that they refuse to use items like the kit-bot chassis, or they may simply not have the technical skills necessary to build a working robot at all. These problems will happen if the season is opened-ended or if there is a time limit, unless the teams have better mentorship. [emphasis mine]

Another culture change I find likely--perhaps naively--if the season got more open is further inter-team mentorship. If the strong teams weren't trying to cram so much into so little time, there'd be more of a chance to help others. This would be particularly potent if the powerhouses we're all trying to keep up with decided it was a good use of their time--which I suspect many would.

We helped and worked with a few teams a bit even with build, but if I look at what FLL does and the scrimmage we host, I know it could be a lot more. We couldn't add more more collaboration meetings or get more people on our pyramid in 6 weeks--but we did have one team that actually used their unbag time with us. A open season makes scheduling stuff like this much more doable. There's only so much you can do to help before the game comes out--so many of the struggles (though not in the teams we collaborated with directly) I see come from misinterpreting the game or mis-prioritizing what one's team is capable of managing.


In short, it's not just the time itself; it's the culture change it could catalyze.

[dtengineering]

I was originally planning to post my expression of support for the six week build period... but Jim's well-researched posts, in particular, have caused me to re-think this. Having an extended build season with some form of milepost along the way might be a worthwhile experiment.

One thing that bothers me during tech inspection is that there are teams that clearly have not read the rule book. It would be great to announce the game, publish the rule book, and then require teams to pass on online rules test before receiving their KoP. (I wish I could remember who originally posted this suggestion... I think it is brilliant.)

I've also thought that it would be good to have a pre-build, design-only period, where teams are allowed to design, model, sketch and plan, but not actually build anything. Unfortunately, this would be difficult to define and impossible to enforce... however with a longer build period I could have told my team, "no one builds anything for the first two weeks". As it turns out, the more experienced we got at the game, the more time we spent on CAD and the less time we spent re-building... more time for design would have been nice.

It would have also been nice to have more time to spend refining the control system. We tried building practice bots, but usually by the end of build we were so burned out (and often getting things ready for our first event) that we never really used them to full potential. Perhaps we could have teams document their robot photographically, uploading the photos on a certain date, and allow no physical changes to the machine other than wiring and sensors between that date and the team's first event. (At which point they could bring in 30 pounds of withholding allowance and make physical changes at the event.) Again, it would be awkward to define, and difficult to enforce, but our programmers deserved more time with the machine. Heck, even a few more days of driver practice would have helped sometimes.

It was also difficult to find working professionals who could make the comittment to attend build sessions three or four nights each week. Without that kind of comittment, it is hard to have a real impact on robot design, especially in the first couple weeks when prototypes and models are created, evaluated, modified and replaced in very short order.

Perhaps most disappointingly, however, we were so busy building our robot that I was only able to get out to assist other local teams a few times during build season.

There are good things about the six-week window, and once we learned to build the best robot that our team could build, rather than trying (unsucessfully) to build the best robot that 1114 could build, it actually wasn't as crazy as it was our first few years. Like I say, I was originally planning to post my support for continuing the six-week limitation.

But I think the arguments put forth in this thread have convinced me that it might be worth trying something new.

Jason

[Zuelu562]

I think we're reaching a crossroads with this issue. In the world where Regionals exist and teams exist that only go to one, I think the 6 week build is necessary. With the proliferation of the district system (FiM, MAR, and this year, NE), I think that an open build is not only plausible, but probably the correct solution. I don't think we need to go to that now, nor need to segment the teams that participate in the district system from the rest of the FRC world via rules changes.

Granted, if we allow open build in the district system, going to a week 1 and week 6 district to allow for the "most" time on field and for fixes. Just as the initial 6 week rule was added for a logistical issue, so too will another system accounting for districts. Do I like the 6 week system, even if it is a marketing slogan at this point? Yes. Do we need it forever? No. Do we need to change now? Not necessarily.

[IanW]

Originally Posted by dtengineering

... Having an extended build season with some form of milepost along the way might be a worthwhile experiment.

One thing that bothers me during tech inspection is that there are teams that clearly have not read the rule book. It would be great to announce the game, publish the rule book, and then require teams to pass on online rules test before receiving their KoP. (I wish I could remember who originally posted this suggestion... I think it is brilliant.)
...
Jason

I'm surprised fewer people haven't touched on these ideas.

First, having a milestone during an extended build season would not only give teams more time to identify and address issues, but also prepare students for the real world where, as I understand it, milestones in design projects are the norm. I recognize that most logical milestones, such as a preliminary inspection, would be hard to implement. However, if one was developed, I feel that it could be very beneficial.

Second, using rules tests as a barrier to entry isn't anything new in student design competitions. I joined a Formula SAE team this year (Go GFR, btw), where rules tests are part of the process of applying to competitions. I feel that this would be easier to implement, though there would need to be penalties in place for late completion/failing to complete the test. However, the penalties also couldn't prevent teams from competing. A rules test could even serve as the milestone described above.

Just some thoughts I had.

[Mark Sheridan]

Originally Posted by IanW

I feel that this would be easier to implement, though there would need to be penalties in place for late completion/failing to complete the test. However, the penalties also couldn't prevent teams from competing. A rules test could even serve as the milestone described above.
.

How about penalizing teams who have not passed or taken the rules test by forcing them to put their robot into a bag after 6 week while everyone else can keep working on their robot?


This tread is a little too serious. Well a more piratical solution we used this year was: we talked to an inspector who mentors a neighboring team. he pointed out our shooter needed a shield. Saved us some time on thursday.

[Siri]

Originally Posted by IanW

Second, using rules tests as a barrier to entry isn't anything new in student design competitions. I joined a Formula SAE team this year (Go GFR, btw), where rules tests are part of the process of applying to competitions. I feel that this would be easier to implement, though there would need to be penalties in place for late completion/failing to complete the test. However, the penalties also couldn't prevent teams from competing. A rules test could even serve as the milestone described above.

Agreed. I think they really could pull off a rules test without much trouble, actually. Just throw the ref & inspector tests (or lite versions) onto STIMS and TIMS, and require say 70% of those registered to pass before the team has access to their PDV voucher codes. All electronic, totally enforceable. Even if teams go open book or take group tests, they'll still absorbing the information better than some do now.


Err...I think I've forgotten how this sub-topic connects to the thread title. Put me down for what Jim and Chris said.


I also agree with Taylor's paths of inspiration. We've never been 0-N exactly, but we went 2-9 at Pittsburgh 2007, with similar results elsewhere 07-08. We moved-ish mostly, in a direction or two. The most inspiring years of our team's history. I can trace it directly to our swerve drive, and in fact to every banner we've ever achieved. I just wish we'd had the opportunity to iterate more back then and receive more team mentorship to get better faster. The team is 9 years old and our first winning season was 2011. (thanks MOE!) We're now happy to be a mid-level team eagerly--but manageably--chasing the elites for 4 solid months of the year.

[waialua359]

Perhaps I may be a little biased being where we're from.
Since participating in FIRST in the 2000 season, and from personal observations, I think the #1 reason new/fairly new teams quit is because of money.
Many grants and/or regionals have startup funds to get teams going. After that, the real struggle begins on addressing sustainability, as opposed to mentor retention and being "not-inspired" as the primary reasons.

I personally like the 6 week build season. It teaches a lot of life lessons and skills for everyone on the team.
However, I am not opposed to eliminating Bag/Tag and allowing for continuous improvement throughout the season. Less time and less resources building a practice bot.

[Tetraman]

I see a lot of posters tossing around the word 'inspiration' as if it blusters an argument. The assumption of "something that could add inspiration being inherently good for FIRST" is not a healthy one. Remember what inspires students more than any other part of the FIRST program: Mentors/teachers/coaches/parents/people. I really don't care if additional time to the build season inspires these kids - I want to know if this additional time actually solves the problem of mentor burnout so our mentors can continue to be the main source of leadership, innovation and inspiration for students.

And I think the answer is pretty basic: We don't know. As some of the mentors have already posted, if the build season does become extended they will feel hurt even more to keep up with the added time demands. On the other hand, other mentors say the additional time will increase their flexibility. So ultimately we won't know the true effect on ourselves and our teams if the build period is extended.

I think the worst part about this issue is that changing it will anger a lot of people but not changing it keeps a lot of people angry. The true answer is for FIRST to choose in what direction the program wants to take. If FIRST doesn't want to alter the system then that's fine with me. If FIRST does change I may not like it but I'll still do everything I can for my team and the problem because I believe in the program.

Otherwise I really have no answers to solve the problem of mentor burnout - except one: Is FIRST really the program for you?

[Ed Law]

Originally Posted by Tetraman

I see a lot of posters tossing around the word 'inspiration' as if it blusters an argument. The assumption of "something that could add inspiration being inherently good for FIRST" is not a healthy one. Remember what inspires students more than any other part of the FIRST program: Mentors/teachers/coaches/parents/people. I really don't care if additional time to the build season inspires these kids - I want to know if this additional time actually solves the problem of mentor burnout so our mentors can continue to be the main source of leadership, innovation and inspiration for students.

And I think the answer is pretty basic: We don't know. As some of the mentors have already posted, if the build season does become extended they will feel hurt even more to keep up with the added time demands. On the other hand, other mentors say the additional time will increase their flexibility. So ultimately we won't know the true effect on ourselves and our teams if the build period is extended.

I think the worst part about this issue is that changing it will anger a lot of people but not changing it keeps a lot of people angry. The true answer is for FIRST to choose in what direction the program wants to take. If FIRST doesn't want to alter the system then that's fine with me. If FIRST does change I may not like it but I'll still do everything I can for my team and the problem because I believe in the program.

Otherwise I really have no answers to solve the problem of mentor burnout - except one: Is FIRST really the program for you?

I like how you summarized the sentiment on this subject. I can't help but draw a parallel to when Michigan proposed the district model and piloted it. There were a lot of oppositions, both inside and outside of Michigan. I don't need to repeat what people had said. I am sure there are still people who think the district model is a bad idea and they oppose it, but a majority of people finally saw the benefits of it and especially after Jim Zondag explained the thinking behind every change.

Change is always hard for some people when they don't know if it will benefit them. By allowing teams the option to access their robot any time if they want to, I don't see how it can hurt any team that does not build a practice robot. The only thing I can think of is for teams that do not build a practice robot thinks it will benefit teams that build a practice robot more than it benefits them. Because it will give these teams more time to concentrate on improving their robot rather than building and wiring another robot and raising more money to pay for it.

For my team, we plan everything to death. Right now in our build schedule, I see days to assemble the practice robot and to wire it. Based on our resources for each year, we decide what kind of robot to design and build so we can finish on time. Then we schedule enough work sessions to finish what we planned to do. We do not meet 7 days a week. In our first two years, we only met 4 days/17 hours a week. In the last few years, we increased it to 5 days/20 hours a week. My goal is to create a sustainable program, hence mentor/student burnout is part of my consideration when I decide on the meeting schedule.

How will it affect us if we have unlimited access to the robot? The first thing I will do is to reduce the number of meetings back down to 4 days 17 hours a week because we can accomplish the same tasks in 6 and a 1/2 weeks. It takes us about 20 hours to build a second robot. That will allow me to take out 3 hours of meeting time each week. On top of that, I will be a lot less stressed out when the two robots behave differently due to quality of build by students and some mentors.

I can handle 4 days a week. 5 days is a stretch for me. Not building a second robot is not an option for me being in Michigan and trying to field a somewhat competitive robot and have a sustainable program. My goal is not to have the most competitive robot and stress everybody out by doing so. I know it is not all about the robot but I also know students joined the robotics team to compete in robotics.

If teams do not want to build a second robot with the current rules, that is their choice. How many hours they meet is their choice. By removing the artificial stop build date, it does not change what teams have to do. It is still their choice. Removing the barrier to access the robot will make my life easier and a better experience for my students.

If you are wondering why it will be a better experience for my students, I will elaborate. Very few of them comes 5 days a week during build season. When students cannot attend all work sessions, they get lost. Things move so fast during build season that it is hard to keep up with the design and decisions if you miss too many meetings. Some students would loose interest and feel disconnected. If I can reduce the number of meetings to 4, a bigger percentage of students will be able to attend most meetings. Our team will be more cohesive. Everybody will be on the same page. We do have weekly team meetings where each subgroup will report on their work but it is not the same.

So for those of you who oppose this possible change, were you opposed to the district model before also? Would you be willing to give it a chance?