Correlation, Causation, and the Isolation of variables in FiM District Structure

FIRST General Forum
1

I expect a lot of discussion to occur in this thread and others about expanding the FiM District structure to other regions after the success of Michigan teams at the Championship event.

The very first thing I’m going to say is that I feel this is a little of a gut reaction by some people. Of the four Michigan teams on Einstein this year, two were there last year (67 and 217).
Michigan teams in final four/Einstein by year:
2001: 4 (of 20 teams, including 217)
2002: 2
2003: 3
2004: 3 (including 67)
2005: 5 (including 67 and 217)
2006: 3 (including 217)
2007: 1
2008: 2 (67 and 217)
2009: 4 (including 67 and 217)

As you can see, four appearances is not entirely outside the realm of previous values (we had 5 in 2005). Additionally, since 2004 two teams (67 and 217) have represented the state of Michigan on Einstein four times each. In other words, 217 and 67 have earned 11% of the total Einstein appearances and 44% of the Michigan appearances on Einstein since 2004.
From 2003-2008, Michigan earned 22% of the Einstein appearances using a normal regional system. In 2009 they earned 33% using the district system. Both the mean and median values of Michigan teams on Einstein is 3, this year is only one team higher. While a jump, it’s not so incredible that I’m willing to say that it’s entirely because of the district system (especially given that 50% of those teams were in the finals last year too). The sample size is simply too small to claim that.
It’s no different than all three of the 2006 winners being from the GTR “Super-regional” or the entire Buckeye winning alliance reaching Einstein in 2004. Should FIRST have spread the super regional model or moved all regionals to Ohio after that? No, of course not.
If anything, teams need to adopt whatever 67, 217, and 177 are doing right (each have 4 Einstein appearances in the past 6 years).

Now, that being said, it’s virtually impossible to argue that the Michigan district structure didn’t have some very very positive aspects. But the district system isn’t one complete “package” that needs to be adopted all or nothing. It is a set of many changes to the “standard” FRC regional system.

Example one, the increased number of matches:
Each FiM district ran 12 qualification matches per team. With ~40 teams at the event, that roughly translates to 80 total matches. While many “large” regionals run roughly the same amount of matches, similar sized events typically do not.
Sacramento (44 teams) ran 66 total (9/team).
St. Louis (37 teams) ran 66 total (11/team).
Palmetto (44 teams) ran 66 total (9/team).
Bayou (31 teams) ran 57 total (11/team).
Waterloo (26 teams) ran 44 total (10/team).

Clearly, it isn’t purely the district system that is causing the increase in matches. The additional events (multiple districts and State Championship) are definite factors, but if each event ran 76-80 matches (like FiM districts) we’d see a definite increase in matches/team.

Example two, “bagging” robots
One of the ways that FiM and teams cut cost was the “bagging” of robots instead of shipping them to events. While not always an ideal solution (obviously for teams that need to fly to events), it could potentially be applied to teams that compete at local regionals.

This thread is not to discuss whether or not the FiM structure is scalable, where it could be applied, or should it be applied elsewhere. There are plenty of other threads for that. But rather, concentrate on isolating individual factors within the district structure that worked, and those that did not work. I’d very much appreciate any feedback from both Michigan teams, Mi veterans, Mi rookies (particularly those who also went to Championship or ideally a regular regional as well), and members of the FiM planning committees.

Regardless of whether or not the whole structure is adopted elsewhere, aspects of it could be made to work at standard regional events.

2

As Michigan veteran and mentor I believe that the system gave the teams a chance to raise the level of competition. You noticed by the end of the third week everyone was already experienced and smart enough to control there robot to at least get in the way of other robots, and keep there trailers empty if they chose to avoid scoring threats. It turned more into the scorers scoring on the dregs or the other scorers when they would stop to score.

I would fully expect to see the Einstein level increase because of the new state championship. There will be more quality robots going because they are allowing high ranking teams, which have had success this year, to go to Atlanta. By nature of the system they are picking up more teams and better teams to send to the championship and I fully expect to see a rise in the number of Michiganders on Einstein. You are getting a lot less defensive or less capable bots going to the championship by assisting the top scorers, and instead you are taking the top scorers.

In the next 3 or 4 years you WILL see a higher percentage of Michigan robots on Einstein.

3

You know, I had typed a response to the folks claiming that the districts were the reason so many Michigan teams were in the finals, but then decided against it because my tone was less than… polite.

Playing matches helps. But those 85 matches the Chickens played amount to a total of 3 hours of driving time folks, broken up into 2 minute chunks. Having a practice robot FAR outweighs the perhaps 1.5 extra hours of match time any one of these teams got.

By far, I’d say big benefits to the Michigan teams were:

  1. The bagging system. Having your robot*** in your facility ***to work on was HUGE.
  2. A greatly reduced amount of luck involved in which teams went from Michigan. Teams had to basically do well in TWO regionals + State champs to go. A team that got lucky with seeding in one regional had very little chance of qualifying compared to years past.
  3. Volunteering. It’s amazing what you learn about what the judges are looking for when you are able to have volunteers that participate in the process.

Teams in Michigan got the opportunity to get far more involved in the system of FIRST this year. I think it was a great success.

The District events weren’t cheaper just because of FIM. They were cheaper because they based themselves on the off-season models of volunteering and teams helping. We helped set up the field at Kettering, just because we were there and had the time.

I think the lower-cost events can be applied across the country without utilizing a FIM style district system, and be successful.

4

Thank you Sean, I would also like to see Midwest teams NOT from Michigan. We don’t operate in a vacuum up here. Perhaps Canadian teams as well. We are focusing on how it affected Michigan teams but we really need to look at how it affected the region.

At first I thought that the teams from Michigan were on Einstein because of the district model but over the last week I have thought about it and have concluded otherwise. 67 and 217 seem to have some sort of rivalry that can ONLY be fought out on Einstein, with the level of play from these teams I am ok with this, I like seeing the matches they are explosive. 68, do I need to defend why T3 was there? I remember in their 1st match they came straight across the field and pinned us in place, they then proceeded to fill our trailer. Nobody Pushes Truck showed us that even this year a good drive train could be a great asset. 247 has always been one of those teams you do not want to have set their sights on you. Defense has always seemed to be their strength and they are GOOD at it. This year they demonstrated that they had some power in their first regional way back in week 1.

5

I was actually tracking down the stats on Michigan teams on Einstein because I was pretty sure that this year wasn’t out of the normal range, statistically speaking. I’m glad someone else did it so I didn’t have to. Thanks.

6

Sean,

Thanks for pulling these statistics together. After reading many of the posts regarding the “success” of FiM and the district model, it seemed to me that the prevailing sentiment was that is was successful, based on the success of the MI teams at the Championships.

While I agree that the added matches and events allowed our teams additional driver experience and comfort under the pressure of elimination matches, I don’t think it is un-reasonalbe to think that 67 or 217 would have had as much success in Atlanta if they only competed in the “normal” two events this season. (No disrespect meant to 68 or 247…just addressing the stats above)

My subjective thoughts on the district model is that everything worked out great. The largest difference that I noticed between the districts and the regional style events we attended, was the intensity of the match schedule a the districts. At both the MSC and FIRST Championships, Thursdays seemed to drag on a bit too long. Also, the hour+ between matches seemed a bit excessive, after the short breaks between matches during district play.

Granted, we had a very trouble free year in terms of robot maintence at competitions. If the robot was not as robust or was less reliable would my opinion be different? I can’t really say. When I asked our pit crew, they said it wouldn’t matter.

So from the statement above, the bagging w/8 hour window, the increased # of matches, and the small awards banquet were all positives in my opinion.

As for the lower cost events with less A/V presence. I can honestly say, I did not notice any less of an overall experience at a district than I did at the MSC.

As for negatives, I really am not sure. I tend to focus only on the smaller picture when I am at an event…the robot, the next match, strategy, etc… I don’t notice any of the larger issues going on around the competition. I am sure that is a flaw that I should or should not address, depending on how focused my team needs me to be.

As was stated prior to the season, there needed to be some objective measure of the district model. I hope someone has statistics from this season on match turn around, volunteer output, etc…other than the $/match stats that could/was stated before the season ever started.

7

Sean, thank you for looking into this. Great Thread

I am going to look at this slightly different.

If you look at the top of the curve, you won’t see a ton of difference. Hot is still Hot and The Chickens are still the Chickens. The are both awesome and will likely continue to be that way as long as they keep putting in the great efforts that they do (keep it up guys).

One of the most important parts of the curve though is the bottom end. By this I mean the teams that only do 1 event and then dissappear or continually only do 1 event and never seem to improve. You won’t hear a lot fo saber rattling from these guys as typically they are too busy trying to get a bot to drive straight. It has been brought up many times that FRC has a horrible attrition rate. This I have been told is one of the main goals of FiM. Teams that have trouble at the first event have 12 matches to get their stuff sorted out, and then most importantly another event to go to where they can try again.

One solid metric you will find is that those that play more do better. What is meant by that is there are few that continuously play at the Championship level (elims at Championship) that only go to 1 regional. Most of the teams competing in the elims go to 2 events and some even do 3 before Nationals. Line up the usual suspects and check. 71, 1114, 2056, 217, 33, 67, 254, 233, … You will see that one common thread is multiple events (there are other threads). This is more than just the extra stick time (that does help), it is the opportunity to try different things, learn, and grow. I just got off the phone with a Rookie team that most years would have done 1 event. They didn’t get their auto mode going until the 5th match at their first event (death this year), but once they got going, they were good enough to get picked and played in the elims. They also made it to the State Championship. Because of their success, they are going to do off season events! This team most years would have had very little success and then folded into another high number dissappearing statistic. Instead, I have high hopes for their future.

To those focusing on the top of the curve, listen to Lil’ Lavery, there is nothing new there.
Spotting Champions is easy. They have big shiny metals around their necks and are grinning from ear to ear while standing on podiums. Focus in on the other parts of the curve. I saw a lot more smiles and success from Rookies and young teams that hadn’t quite figured out how to turn an OK season into a great one. I saw teams that have been mid pack really blosom when given the same chance at doing multiple events that bigger budget teams have.

I am not sure how to measure this other than number of matches and looking at possibly OPR shifts between events, and awards records. The other “evidence” I have is purely anecdotal which does have value, but often is skewed. In a couple of years, retention data should have statistical significance (though the economy would likely skew some of that too), but I can’t see the rest of FRC being happy with a FiM Only Pilot that goes on for several years…

=======================================
I did have an interesting idea of measuring some of this. I am just not good enough at programming. If someone could go on TBA or FIRST records and show the amount of matches and Elim matches for teams over the last 3 years in MI, I think this might show some good data. It should show a big birth of new teams getting to play in Elims and getting more matches. Personally I feel that is a good thing towards sustainable growth.

8

At the risk of sounding like a broken record…

If it’s not about the robots™, how can examining the success of teams on the field – at any end of the curve – be used as an accurate, meaningful metric of the program’s success?

9

Could not agree with you more, but I took this thread to be about outside metrics, measurable, and correlations. (As a side note, I think it is safe to say that Robot performance is only part of “Team” performance this year with some highly effective Human players).

Thinking about it though, there are some things that could be measured.

Some great, “not entirely about the robot” metrics.

7 teams were recognized as District Chairman’s award recipients instead of 3 from the regionals before. 3 went on from the State Championship to “The Championship”

A ton of Woody Flower Nominees were recognized at every event (regionals could do this too). I don’t know the official count, but I believe it to be on the order of 30 or more mentors that got recognized (it could have been a lot more).
7 - Imagery, Spirit, Rookie Allstars…

These were pretty good “not about the robot” boosts for FiM. While not everything can be measured with Lucite or Medals, many things can be. For those that can’t, please talk with those teams. Most that I have talked to really like the additional playing time, and the closer events. The second event was a huge thing to a lot of teams I talked to.

10

As for the recognizing the Woodie Flowers nominees, at the Wisconsin regional they put up a list on the screen of all the nominees before they announced the winner. I think that doing it this way is much better than the way they used to do it. You never knew who was nominated you just knew who won. Its nice to show the mentors what they really mean to you even if they don’t get a trophy to show for it.

11

At the FiM District events all of the WFA nominees were called down to the playing field for recognition (and a round of applause)for their contributions to their teams and FIRST.

12

Yes, I attended Traverse City and the Troy districts, i was saying this was the 1st time that in my 8 years that i have seen a regional show everyone that was nominated. If they had called everyone down it would have taken forever, seeing as it was at a sports arena and not a high school gymnasium, so they didn’t get the “honor” of standing in front of the crowd but it was still cool to see them all recognized.

13

Aspects of the FiM model that were successful and could be implemented to FIRST in general include:

  1. Two day events – lots of positive feedback from mentors taking less time off work.
  2. Bagging of robots – no shipping.
  3. Volunteer run events – huge cost savings.
  4. Events at High Schools – more exposure to teachers and non FIRST students, lower cost for venue.
  5. Events that run later on Friday so teams don’t have to try to figure out what to do Friday night, they are doing what they came to do.
  6. More matches per event.
  7. A point system to rank teams could be implemented eliminating the issue of crossing state lines to compete.
14

FIRST, the organization, the program, the message…is not about the robots…

The FIRST Robotics Competition…is all about the robots… The inspiration is the result…

And, I’m willing to say that a successful robot, more rounds, more time with the robot, all of the things mentioned above (that all have to do with the robot) makes the inspiration more successful…

And that makes the FIRST organization or program a success…