The 6 Week Build Season and 'Mentor Burnout'

[Jim Zondag]

So, after all the back and forth in this thread, and all of the anecdotal and revisionist commentary on the history of what we do and why, I decided to do a little archeology. I went back and reviewed the actual game manuals from the early years (92, 93, 95, 96, 97, 98). There is no available 94 Game Manual I could find in any of the archives.

In 92 and 93, there were no machine access restrictions at all. There was a kickoff, and there was a tournament. That was it....make a robot and show up to play.

In 95 we see the first indication of limits, as FIRST expanded to have more than one event. The rules of engagement were the same in 95 and 96. The excerpt from the 1996 manual is below:
-------------------------------------
Shipping Deadlines
To provide every team, regardless of events in which they participate, approximately the same number of design and build days, the following shipping regulations and dates apply:
New England Tournament (Manchester, NH) Competitors
1. Teams may either ship of bring their machine with them to the tournament.
2. After the tournament, all teams competing in the National Championship will have five days to make repairs and/or changes to their machines, within all rules outlined in this document
3. By end of business on Friday, April 5, machines must be picked up by a shipper for transport. This will give all New England teams five additional days to work on their machines.
National Championship participants only
1. Teams must ship their machines by end-of-business on Tuesday, April 2, 1996.
2. This will give all teams competing in only the National Championship an equal number of days to work on their machines as team competing in both events.------------------------------------

So, if you notice, this was not done to limit involvement by participants. It was done to try to equalize the number of workdays depending on if teams went to one or two events. You were allowed to work on your robot all the way upto and through the regional if you chose. Since the CMP required shipping robots to Disney, equalizing dates were imposed. Teams had 5 days to work on their robots after the first event before being required to give it up.

Actual "shipday" rules were not imposed until 1997. From the 97 manual:
----------------------------------------------------------
1. Machines MUST BE OUT OF TEAM HANDS by 5:00 p.m. on Tuesday, Feb 25, 1997. This means you many ship the robot or drive the robot to the drayage/storage facility of your first event by 5:00 p.m. on February 25.

Regional Competitors
1. After competing in a Regional, any teams competing in another event will have two days to make repairs and/or changes to their machines, within all rules outlined in this document.
2. Machines MUST ARRIVE as the next site by the next Tuesday at 5:00 p.m.
-----------------------------------------------
After talking with some oldtimers from this era, they believe that the main reason for these changes in 1997 were due to LOGISTICS concerns. When events started to be scheduled on back to back weekends in 97, FIRST had to reduce the amount of time teams had to do repairs in order to make sure the crates could get to their destination in time.

Eventually (2002) the weekend hold back period was eliminated completely, again this was mainly to avoid the logistical complexity of hundreds of teams trying to all ship from various locations and instead allowed FIRST to control all of the logistics of all of the machines from the beginning to the end of the season by shipping direct from event to event. Again, the removal of these hands on repair and improve windows had nothing to do with limiting access by team, it was for LOGISTICS.

Phase forward to today. ALL of the Logistics for ALL of the teams are now entirely up to the teams and not any centralized control. So why does FIRST still maintain these rules? Seriously, they are nothing more than an artifact of an obsoleted system. Everyone who put the original rule in place at FIRST HQ is gone, yet these rules remain. Teams now attend anywhere from 1 to 6 events per season, so equalization of access time is functionally impossible, yet these rules remain.
So ask yourself, are we really doing something smart with the machine access rules, or is all of this just a collection of old rules, imposed for obsolete reasons which have been forgotten?

[Mark Sheridan]

Originally Posted by Jim Zondag

So ask yourself, are we really doing something smart with the machine access rules, or is all of this just a collection of old rules, imposed for obsolete reasons which have been forgotten?

+1

[Tetraman]

Originally Posted by Jim Zondag

So ask yourself, are we really doing something smart with the machine access rules, or is all of this just a collection of old rules, imposed for obsolete reasons which have been forgotten?

I'd say half and half. It's not smart at all to just keep doing things because thats how they have been done, but I am still in the camp that the rules as they exist (with some tweaks) are the best way to go.

To argue for why an "open access" build is in my opinion wrong for FIRST, is because of the competition schedule. Consider that there was open access to the robot at all times. Some teams don't have their first Regional event until week 4 or 5. A good majority of teams in those week 4 or 5 regional have at least been to one other event. I would argue that the teams who have been to a previous event have a massive advantage over those who hadn't.

Teams that know how their robot plays in a game of first and can correct it over the course of 2 to 3 weeks have a gigantic advantage over those who have so many more extra weeks of build and gameplay footage but no knowledge on how their robot actually does with 5 others on the real field. I'd argue further to say that in this scenario the best teams will HAVE to make week 1 event if they want to have any sort of competitiveness for the future events.

Open build would give too much advantage to teams with the capability to travel long distances to their events or have events nearby for attending both a week 1-3 and week 4-6.

Keeping the robot out of the hands of teams during the competition season maintains that robots and teams can be as equal as they were at the end of the competition season since the beginning of it - but doesn't stop teams from fixing their problems and getting better over time. This year so many teams got better as the events went on, but it wasn't overwhelming to push out teams who hadn't got a taste of action on the field.

I think the 30lb allowance is fine. It allows for robots to evolve and can keep teams busy, if they want to be busy, during the competition events but isn't that much unfair as being able to redo half of a robot in 2-3 weeks.

[Jim Zondag]

In 2013, despite all the successes of all the teams represented on these forums, the Median OPR of the league after 1 event played was 10.3.
Over 1200 teams had a net contribution of 10 points or less per match. This level of accomplishment could be achieved in Ultimate Ascent by simply building a kit chassis with two stationary hooks on top. Since HALF of the league cannot achieve this basic level of play in their first outing, it is very hard for me to understand statements that say that we are giving teams enough time to be successful. We are not. 13% of the league has a sub-zero OPR at their first event. This means that our league is producing over 300 teams per year who NEVER EVEN SCORE AT ALL in their first event. This is a design failure of our system. (and I doubt than any of those teams are represented here).

Sure, many of these teams would still not be successful if we gave them more time, but many of them would get much better. Look at the data. After two outings the median MORE THAN DOUBLES to 24 points per team per match. This is huge. The average of the top teams does not change very much, but number of negative almost completely disappears.

What this is telling us is that with some time to work on their machines, get on a playing field, get help from other teams, and benchmark other's solutions,
HUNDREDS of teams move from a position of NOT BEING ABLE TO PLAY, to a position of BEING ABLE TO POSITIVELY CONTRIBUTE to their alliance's success. However, since 1400 teams only played once this year; many, many teams never get this opportunty to improve. And because of this, many of them fail and do not return to the FRC. And this is the real problem. The most difficult place to start a FIRST team is at a school where they previously decide to quit FRC in the past. Our current system is somewhat designed to kill teams early in their life cycle.



These trends are essentially the same, year after year after year, regardless of game design. When registration closes next fall, take a look at where all of the teams who drop out of FRC fall on these charts. The key to improving the sustainablity of FRC is to get teams to a reasonable success plateau early in their team history, ideally in their first season. Then they are likely to stay. Right now the FRC system works against this goal in many ways.

Some of the people on this post say they may quit if the rules were changed, however we already have a system which actively kills teams by the hundreds each year. Which is worse?

[s_forbes]

Originally Posted by Jim Zondag

... HUNDREDS of teams move from a position of NOT BEING ABLE TO PLAY, to a position of BEING ABLE TO POSITIVELY CONTRIBUTE to their alliance's success. However, since 1400 teams only played once this year; many, many teams never get this opportunty to improve.

I think this may be a misleading graphical representation of the time input vs performance for the reason you stated here. I'd suspect many teams that put in the effort to raise the funds to attend two regionals also put in more effort to build a robot that performs well. Their performance isn't based on playing time at regional events, there's just a correlation between the effort spent building a high performance robot and the effort spent raising funds for multiple regionals. I do agree that robot performance increases if a team has the opportunity to attend multiple regionals, however, and also agree with the point you're making.

With the way our team operates during the build season, I would prefer to have no bag day. We build a practice robot to get more tweaking time in, and in doing so we waste a lot of man-hours and resources just to build the second robot. With no bag day, and no need for a practice robot, we would be expending fewer resources to get the same performance out of our competition robot (which would allow us to take more days off during build season).

With the amount we spent doing fast R&D and building a practice robot to work around the bag deadline, we could have potentially gone to another regional instead (if it was priced similar to district events).

[Ian Curtis]

Originally Posted by s_forbes

I think this may be a misleading graphical representation of the time input vs performance for the reason you stated here. I'd suspect many teams that put in the effort to raise the funds to attend two regionals also put in more effort to build a robot that performs well. Their performance isn't based on playing time at regional events, there's just a correlation between the effort spent building a high performance robot and the effort spent raising funds for multiple regionals. I do agree that robot performance increases if a team has the opportunity to attend multiple regionals, however, and also agree with the point you're making.

Jim, can you do the same thing comparing 1st and 2nd event just Michigan and MAR teams? I have to imagine it shows the same trend.

Originally Posted by Jim Zondag

So ask yourself, are we really doing something smart with the machine access rules, or is all of this just a collection of old rules, imposed for obsolete reasons which have been forgotten?

Originally Posted by Jim Zondag

Some of the people on this post say they may quit if the rules were changed, however we already have a system which actively kills teams by the hundreds each year. Which is worse?

And if we do it for a season and it does really end the world as we know it... who says we can't go back?

Originally Posted by Tetraman

Open build would give too much advantage to teams with the capability to travel long distances to their events or have events nearby for attending both a week 1-3 and week 4-6.

I absolutely disagree. Closed build season gives the 2 event team at least 8 matches on the real field, plus a few practice matches, PLUS 3 days to work on their robot for their event registration. Meanwhile a team that is not competing does not get to practice, AND gets no time to work on their robot. Multi-event teams HAVE a huge advantage as is. Instead of leaving 1 event teams to be sitting ducks, moving to an open build season at least lets them learn, make changes, and most importantly practice like multi-event teams.

[Tristan Lall]

A couple of methodological points, just to be rigorous about this:

Originally Posted by Jim Zondag

Over 1200 teams had a net contribution of 10 points or less per match.
[...]
13% of the league has a sub-zero OPR at their first event. This means that our league is producing over 300 teams per year who NEVER EVEN SCORE AT ALL in their first event.

How are fouls being accounted for, if at all? As negative points? Could a portion of those 300 therefore be scoring fewer points than they receive in fouls? (This is of course still awful, but it doesn't mean they didn't score.)

Originally Posted by Jim Zondag

Sure, many of these teams would still not be successful if we gave them more time, but many of them would get much better. Look at the data. After two outings the median MORE THAN DOUBLES to 24 points per team per match. This is huge. The average of the top teams does not change very much, but number of negative almost completely disappears.

Isn't that confounded by the likelihood that many of the worst teams only participate in one event (and thus their badness isn't reflected in the event 2 statistics)?

To measure the actual improvement, we'd want to separate out the event 1 performance of teams that participate in 2 or more events, and compare that to those teams' 2nd events. (And even after doing that, you wouldn't expect this 2-or-more-week group to be representative of the 1-week teams, because attendance at a second event—especially outside of district play—is probably strongly correlated with greater resources and organization.)

Originally Posted by Jim Zondag

Our current system is somewhat designed to kill teams early in their life cycle.

I think "designed" is probably too strong a characterization. It's nevertheless completely fair to say that the system is pretty good at it.

[Nemo]

Originally Posted by Tristan Lall

Isn't that confounded by the likelihood that many of the worst teams only participate in one event (and thus their badness isn't reflected in the event 2 statistics)?

To measure the actual improvement, we'd want to separate out the event 1 performance of teams that participate in 2 or more events, and compare that to those teams' 2nd events. (And even after doing that, you wouldn't expect this 2-or-more-week group to be representative of the 1-week teams, because attendance at a second event—especially outside of district play—is probably strongly correlated with greater resources and organization.)

Here's just the teams that attended 2 or more events.

[Mr. Van]

Jim brings up some valid points. Obviously teams that are not successful are not inspired to continue, but I don't believe that extending the build season will help these teams to the extent hoped. The reasons a team is not able to build a functioning robot (a drivable base with some articulated hooks on it, for example) are many and varied, but I would argue that a build season that is too short is not one of them.

One of the major reasons is that teams perpetually "bite off more than they can chew". Another is that they refuse to use items like the kit-bot chassis, or they may simply not have the technical skills necessary to build a working robot at all. These problems will happen if the season is opened-ended or if there is a time limit, unless the teams have better mentorship.

As Jim notes, real improvements occur when teams get on an official game field, play against other teams and gain the experience of driving a robot in a competition. It is a real shame of our system is that half of the teams are "one-timers". At least we agree on that! It seems that the district system will help in that basic registration includes two events, but overall, this is a major problem.

To me, however, the solution to improving robot (and therefore team) performance is to improve mentorship and provide more opportunity to actually play the game. This does not require a longer build season.

I doubt that the CD community will be able to come to any sort of consensus on this issue, but I'm glad that we've brought this question up for debate.

- Mr. Van
Coach, Robodox

[Siri]

First, Jim, thank you very much. This offers very, very valuable perspective on the matter that we've only really alluded to thus far. I'd look forward to any MAR/FiM exclusive data you might have. It might not be as dramatic--among other causes, they're quite competitive regions, both due to and resulting in the district model--but I'd bet it's something.

Originally Posted by Tetraman

Open build would give too much advantage to teams with the capability to travel long distances to their events or have events nearby for attending both a week 1-3 and week 4-6.

I would hope that in many if not most places, an open season would make FRC practice more like FLL., FTC and VEX: scrimmages aren't for "off-season". If some off-seasons moved to off-event weeks (in their area) during competition season, teams wouldn't have to travel so far or pay so much just to play on an at least semi-real field with other teams. (Semi-real given the logistics of getting a real field in competition season.) It might make easier replicablity/shippablity a constraint on the FRC GDC the way it is on the others--but heck, shouldn't it be anyway? It's certainly well past worth it.

Originally Posted by Mr. Van

One of the major reasons is that teams perpetually "bite off more than they can chew". Another is that they refuse to use items like the kit-bot chassis, or they may simply not have the technical skills necessary to build a working robot at all. These problems will happen if the season is opened-ended or if there is a time limit, unless the teams have better mentorship. [emphasis mine]

Another culture change I find likely--perhaps naively--if the season got more open is further inter-team mentorship. If the strong teams weren't trying to cram so much into so little time, there'd be more of a chance to help others. This would be particularly potent if the powerhouses we're all trying to keep up with decided it was a good use of their time--which I suspect many would.

We helped and worked with a few teams a bit even with build, but if I look at what FLL does and the scrimmage we host, I know it could be a lot more. We couldn't add more more collaboration meetings or get more people on our pyramid in 6 weeks--but we did have one team that actually used their unbag time with us. A open season makes scheduling stuff like this much more doable. There's only so much you can do to help before the game comes out--so many of the struggles (though not in the teams we collaborated with directly) I see come from misinterpreting the game or mis-prioritizing what one's team is capable of managing.


In short, it's not just the time itself; it's the culture change it could catalyze.

[dtengineering]

I was originally planning to post my expression of support for the six week build period... but Jim's well-researched posts, in particular, have caused me to re-think this. Having an extended build season with some form of milepost along the way might be a worthwhile experiment.

One thing that bothers me during tech inspection is that there are teams that clearly have not read the rule book. It would be great to announce the game, publish the rule book, and then require teams to pass on online rules test before receiving their KoP. (I wish I could remember who originally posted this suggestion... I think it is brilliant.)

I've also thought that it would be good to have a pre-build, design-only period, where teams are allowed to design, model, sketch and plan, but not actually build anything. Unfortunately, this would be difficult to define and impossible to enforce... however with a longer build period I could have told my team, "no one builds anything for the first two weeks". As it turns out, the more experienced we got at the game, the more time we spent on CAD and the less time we spent re-building... more time for design would have been nice.

It would have also been nice to have more time to spend refining the control system. We tried building practice bots, but usually by the end of build we were so burned out (and often getting things ready for our first event) that we never really used them to full potential. Perhaps we could have teams document their robot photographically, uploading the photos on a certain date, and allow no physical changes to the machine other than wiring and sensors between that date and the team's first event. (At which point they could bring in 30 pounds of withholding allowance and make physical changes at the event.) Again, it would be awkward to define, and difficult to enforce, but our programmers deserved more time with the machine. Heck, even a few more days of driver practice would have helped sometimes.

It was also difficult to find working professionals who could make the comittment to attend build sessions three or four nights each week. Without that kind of comittment, it is hard to have a real impact on robot design, especially in the first couple weeks when prototypes and models are created, evaluated, modified and replaced in very short order.

Perhaps most disappointingly, however, we were so busy building our robot that I was only able to get out to assist other local teams a few times during build season.

There are good things about the six-week window, and once we learned to build the best robot that our team could build, rather than trying (unsucessfully) to build the best robot that 1114 could build, it actually wasn't as crazy as it was our first few years. Like I say, I was originally planning to post my support for continuing the six-week limitation.

But I think the arguments put forth in this thread have convinced me that it might be worth trying something new.

Jason

[Zuelu562]

I think we're reaching a crossroads with this issue. In the world where Regionals exist and teams exist that only go to one, I think the 6 week build is necessary. With the proliferation of the district system (FiM, MAR, and this year, NE), I think that an open build is not only plausible, but probably the correct solution. I don't think we need to go to that now, nor need to segment the teams that participate in the district system from the rest of the FRC world via rules changes.

Granted, if we allow open build in the district system, going to a week 1 and week 6 district to allow for the "most" time on field and for fixes. Just as the initial 6 week rule was added for a logistical issue, so too will another system accounting for districts. Do I like the 6 week system, even if it is a marketing slogan at this point? Yes. Do we need it forever? No. Do we need to change now? Not necessarily.

[IanW]

Originally Posted by dtengineering

... Having an extended build season with some form of milepost along the way might be a worthwhile experiment.

One thing that bothers me during tech inspection is that there are teams that clearly have not read the rule book. It would be great to announce the game, publish the rule book, and then require teams to pass on online rules test before receiving their KoP. (I wish I could remember who originally posted this suggestion... I think it is brilliant.)
...
Jason

I'm surprised fewer people haven't touched on these ideas.

First, having a milestone during an extended build season would not only give teams more time to identify and address issues, but also prepare students for the real world where, as I understand it, milestones in design projects are the norm. I recognize that most logical milestones, such as a preliminary inspection, would be hard to implement. However, if one was developed, I feel that it could be very beneficial.

Second, using rules tests as a barrier to entry isn't anything new in student design competitions. I joined a Formula SAE team this year (Go GFR, btw), where rules tests are part of the process of applying to competitions. I feel that this would be easier to implement, though there would need to be penalties in place for late completion/failing to complete the test. However, the penalties also couldn't prevent teams from competing. A rules test could even serve as the milestone described above.

Just some thoughts I had.

[Mark Sheridan]

Originally Posted by IanW

I feel that this would be easier to implement, though there would need to be penalties in place for late completion/failing to complete the test. However, the penalties also couldn't prevent teams from competing. A rules test could even serve as the milestone described above.
.

How about penalizing teams who have not passed or taken the rules test by forcing them to put their robot into a bag after 6 week while everyone else can keep working on their robot?


This tread is a little too serious. Well a more piratical solution we used this year was: we talked to an inspector who mentors a neighboring team. he pointed out our shooter needed a shield. Saved us some time on thursday.

[Siri]

Originally Posted by IanW

Second, using rules tests as a barrier to entry isn't anything new in student design competitions. I joined a Formula SAE team this year (Go GFR, btw), where rules tests are part of the process of applying to competitions. I feel that this would be easier to implement, though there would need to be penalties in place for late completion/failing to complete the test. However, the penalties also couldn't prevent teams from competing. A rules test could even serve as the milestone described above.

Agreed. I think they really could pull off a rules test without much trouble, actually. Just throw the ref & inspector tests (or lite versions) onto STIMS and TIMS, and require say 70% of those registered to pass before the team has access to their PDV voucher codes. All electronic, totally enforceable. Even if teams go open book or take group tests, they'll still absorbing the information better than some do now.


Err...I think I've forgotten how this sub-topic connects to the thread title. Put me down for what Jim and Chris said.


I also agree with Taylor's paths of inspiration. We've never been 0-N exactly, but we went 2-9 at Pittsburgh 2007, with similar results elsewhere 07-08. We moved-ish mostly, in a direction or two. The most inspiring years of our team's history. I can trace it directly to our swerve drive, and in fact to every banner we've ever achieved. I just wish we'd had the opportunity to iterate more back then and receive more team mentorship to get better faster. The team is 9 years old and our first winning season was 2011. (thanks MOE!) We're now happy to be a mid-level team eagerly--but manageably--chasing the elites for 4 solid months of the year.