Chairman's Award Iteration

There has been a lot of talk about teams entering at multiple events this year but now that the regional season is over we need to look back at the process and see how well it worked.

Both of our feedback forms are posted here. It’s interesting to see that at Dallas where we didn’t win the judges seemed to give us overall better scores. We drastically overhauled our presentation for Lone Star and improved that score but regressed in others but we were selected as the Chairman’s Award winners.

How did other teams iterate on their awards process? How was the judging different from event to event for your team?

Anyone else have their feedback forms posted for comparison?

Great idea for a thread, Allen!

How did other teams iterate on their awards process?

FIRST Team 2468 presented twice, at Alamo (which we lost) and Hub City (which we won).

After not winning Chairman’s at Alamo our team was ready to make major changes to our presentation. However, after speaking to several key people we discovered that we very narrowly lost, the decision being very close between us and another team. So after speaking to, well you Allen :slight_smile: , we decided to only add a poster to the presentation, and keep everything else the same. The strategy paid off after we won Chairman’s at Hub City.

How was the judging different from event to event for your team?

The task for Judges at Alamo was very difficult, as it was a huge regional with a record number of teams competing (something like 24 teams) for Chairman’s. The presentations were scheduled by the event staff. The judges in the presentation seemed like traditional judges, warm but stern/professional. At Hub City we had another unique judging experience, as there were a very small number of judges for a more normal sized event. However, the quality of judges was incredible, the warmest and friendliest I have ever met, both in the pits and in the presentation room. While this made everyone more comfortable, I doubt it had any comparable effect on the outcome compared to Alamo.

Anyone else have their feedback forms posted for comparison?

Neither of the feedback forms were very revealing. They both marked excellent in all categories, and neither offered much insight into why we won or lost. The Hub City form, however, did offer some advice for our Championship presentation.

Ultimately the new process implemented this year did not affect our team. If we had to choose a regional at the start of the year to submit at, we likely would have presented at Hub City, which we won anyway. Though for many teams the iteration process helped improve their presentation, since we decided not to alter our presentation, for us this process didn’t change much.

Still, I think it was definitely a change for the better and I am glad FIRST looks to improve itself in ways like this.

-Mikey

We submitted at Orlando and Bayou. Between the two events, we modified our presentation script to further emphasize what we felt were our strong points.
Right after Orlando, we realized how much our RCA package is really “Bayou oriented”. For example, FTC has a very small presence in Louisiana (though not for long), in contrast to Florida. Therefore our lack of FTC mentoring isn’t seen as a “weakness” at Bayou, but it is at Orlando. While the presentation changes were definitely beneficial, I also think the different set of judges was a major factor in our feedback disparity. The correct focus needed to win RCA varies for each event.
I don’t have the feedback forms with me now, but we got neutral-mildly positive scores from Orlando and extremely positive scores from Bayou.

Going off of what the wonderful Team Combustion said above…

We submitted at the Lake Superior Regional and the 10,000 Lakes Regional. The different sets of judges and Chairman’s competition were a huge factor.

Our presentation was “Rock Band” themed. By that I mean, our Chairman’s team literally carried rock band instruments into the judge’s room and said a ton of music related terms. When you see something like that, you have to have an open mind and enjoy the weirdness.

According to our Chairman’s team, the judges at Lake Superior were a little surprised and kind of sort of “didn’t get it” (that is not a knock on the judges! we knew what we were getting into when we decided on Rock Band). If you ask me, the other teams at that competition had more professional types of presentations and that’s what the judges were expecting. Plus, the competition there was much tougher (many teams who have been doing Chairman’s for 5+ years). Compare that to 10,000 Lakes were 14 teams presented and I know of at least 4 of them that had never done it before. The judges were laughing as soon as the team walked in and were very receptive to the creativity.

We changed very little between the competitions. The only change we made was in our “Sponsor partnerships” section. We added more statistics and measurable outcomes to that section. Other than that, it was basically the same.

It was really nice for our Chairman’s team to have some practice at Lake Superior. We appreciate the chance to present two different times.

This was the first year we have taken a very serious approach and put in quite a lot of effort for Chairman’s. We received very high marks at Madera which left our students disappointed to not win but enthusiastic to present again at Long Beach. At Long Beach our marks varied greatly and were rather different from the marks at Madera, which left us with a feeling that we’ll never win it unless we drastically change our operation, which we are not going to do just for an award. We finished the season with the impression that FIRST has a very specific ideas of what they want in a chairman’s award team and we aren’t it. We’ll see what 2015 brings.

I have heard this from a few teams and I have to admit I don’t really get. I’ve watched your Chairman’s Video it seems right on track with what FIRST would want to see in a Chairman’s Award team.

The responses so far seem to suggest that the judging at different events is varying a lot.

I think one of the best ways we can find out if this is actually a problem or at least learn about the judging at different events is for teams to post their feedback forms. I know they are sometimes critical of the team but I do think a lot of teams would benefit from being able to see what the judges are saying.

Personally I am happy to see this thread started. I was skeptical about the new system (after seven years it feels weird changing the rules and I don’t know how personally I would have dealt with a few more weeks of edits and stress!) but am glad to hear it worked for some teams!

Unfortunately, AllenGregory, you are right. Judging varies at events a lot due to difference of opinions and experience in FIRST of judges. However, I feel like that is the case for all awards, not just Chairmans.

Also, while a video may be good, personally I feel as if the videos play a small role on the regional level. I feel like the communication of the team through the essay and presentation/interviews matters a lot more. I would always review the forms that we received (I still actually have photos of the feedback forms years later) and it did help to modify the next year. But there are just so many factors (who the judges are, who is your competition, the area you are entering in, etc) that it is really hard to predict why the judges decide one team over the other in the end.

I’m definitely a fan of the multiple submission rule this year. I’ll share 1675’s experience with it.

We tend to make “fun” presentations, skits almost. This year was our 10th year so we came up with the theme of the UPS 10th anniversary party which would be crashed by our villain from last year’s presentation, Dr. Ignorance (the team is superhero themed). Choice quote from the intro: “Fools! I was but a junior in high school last year. You haven’t had a taste of my true power!” Dr ignorance topples our cake prop off the table and while presenting the students build the cake up, each piece represents a part of the team (Pictures and words are on the inside of each cake piece). The foundation is sponsors and mentors, the middle layer is outreach, helping others, and GP, and the goals we reach at the top of the cake are impact on our team members and the community. I will post a pic of the prop to CD Media shortly.

Our first competition was the Wisconsin regional (WIMI). A number of years ago we decided to stop submitting Chairman’s at WIMI for a number of reasons, general team disorganization at our home regional, the Chairman’s “metagame”, and it’s usual position in our season are only a few of them.

This year FOURTY (40) teams submitted Chairman’s at WI, and by my estimation of previous history of teams I knew anywhere between 12-18 of them could have won it, and there were a lot of teams I didn’t know.

Here’s our feedback from WI (1 = strongly disagree, 7 = strongly agree)

The kids noted that the judges were very receptive to the presentation, and that the interview seemed to go very well.

We didn’t win, but were ready to improve for Midwest Regional (ILCH) in week 6. Both of our RCA wins have come from Chicago (2011 and 2013). We didn’t change much, just tweaked a few lines to highlight some points and events that we forgot about in the original. Here’s our feedback from ILCH:

We were initially a little worried about Chicago since we won RCA there in 2013 and our presentation was a tad bit similar (involved a giant comic book instead of a cake, but did have the 2 heroes vs Dr. Ignorance theme). The kids noted that the judges were more stern than the WIMI judges, but that might also be because they were tired (we got the final timeslot). In the end they said the judges still liked the presentation. They noted that they felt the Q&A went well but not as well as WIMI. The scores fluctuated a little bit, 2 or 3 sections down a point and 1 section up a point. We also did not win in ILCH.

Overall we had a great experience with the multiple submissions (though it sure was great only stressing out about RCA at 1 competition a year :wink: ).

I do agree that in different regions it appears to be judged a lot differently. Over the years our feedback forms have looked very similar - mostly top, -1, or -2 marks, with sparse or vague written feedback “good work, continue involvement, you’re going all the right things, keep it up” (and to be fair to the WIMI judges they had to watch presentations from 8:15-5:30 Friday and 8:15-lunch Saturday). These were all almost the same no matter if it was a year we won or if it was a year we didn’t win since about 2010 or 2011 (when I started really getting into helping the Chairman’s group). I wish we got as much as what was on Spectrum’s Dallas form!

(No comments in this post were meant to take any jabs. Just sharing my view of our experiences as a team with the Chairman’s Award)

This is one of the things I was wondering about when I made this post. Now that we are able to iterate during the season. FIRST really needs to emphasis to the judges how important their feedback is to teams. What ever process they used at Dallas to have typed feed back forms should be implemented across events. They were able to give much more useful feedback.

Thank you UPS for sharing your feedback.

I don’t have the forms on me but:
We received a perfect score at UNH but still lost to 3467 who also got a perfect score. At NU we got a perfect score in every catagory except for raising new FRC teams, which we got a 5 on I believe, and we won the Chairman’s award at that event. The major change between these two competitions was that we created a visual to give the judges something to remember us by, in the form of a pirate map where along the route was everything we currently do and the “x marks the spot” was on where we wanted our team to go in the future. So none of the information we presented changed, just how we presented it.

We presented at both the Alamo and Virginia Regionals. We were not expecting to win in Alamo, but we were surprised and encouraged with our response sheets. We had all perfect fields except for 2 sections where we were one off.

We were nervous about presenting in Alamo, because we just didn’t know what to expect. I was not one of our presenters, but when I talked to them they described the judges as not asking any questions after the 5 minute presentation and even being a little distracted. For example, when we mentioned working to change SOLs (our standards of learning in Virginia) one judge talked for about 2 minutes about the Texas Standards of learning instead. The judges also didn’t like how we had bullet listed our summaries to fit in more information. I think it also threw them off that our essay and presentation were organized differently. That’s something we are looking to change in the future. I think that this really comes down to the differences in Regionals and understanding how to control the entire meeting. In the future we will be preparing our presenters for other unexpected judging styles.

However, the judges were very enthusiastic about our presentation anyway and called it “The Best Presentation Ever!” on our sheet. Our presentation is fairly old-school. We have a large picture book style presentation that describes the facets of the team as the parts of The Mech Tech Dragon. It’s very heart felt and gets the point across succinctly and well.

We didn’t change our presentation very much except to practice, emphasize the strengths in the areas that weren’t given the high score in Alamo, and make the metaphor stronger.

At Virginia, our experience was standard and what we were used to. The judges asked questions and enjoyed our presentation openly. Our response sheet was perfect across the board and there weren’t any critiques (although we do have room to improve). We were very worried because we didn’t have one Engineering Inspiration judge come by our pit, and we were fairly certain after winning the Entrepreneurship Award that we wouldn’t be winning another non-technical award. However, the real reason we didn’t get to talk to the EI judges was that they already knew we had been selected for Chairman’s!

I don’t have the data on me right now, but I thought it would be good to point out that every RCA Winner this past week won at their second event. I’ll post the data from all the weeks in a bit.

Being the kind of team that wins RCA at a Week 6 event is very strongly correlated with being the kind of team that goes to more than one event.

Red2486,

I’d be very interested to see this data. In general I’m curious to see how many RCA winners this year won at their first, second, or third event.

In addition, I’d be curious to see how many teams submitted at each event. I know this is a harder data point to capture, but perhaps if someone were to crowd source the data collection via a Google Doc, we could get a decent snapshot of submission rates.

1629 GaCo won at our second regional(Chesapeake), I think among 22 teams that submitted. We also submitted at our first regional(Virginia), but I am not sure how many teams submitted there.

125 won at 4

Iterated content and presentation based on feedback at all events.

604 also submitted at two regionals, San Diego and Silicon Valley.

At SD, our feedback form was medium to high in the various categories. One thing that we got marked on low was our written submission, which we were concerned about since there was no way to improve upon that in between regionals. For SVR, we made modifications to our presentation based on some of the feedback we had gotten.

Our SVR feedback form was drastically different however. Everything was in the full category, including the written submission which had an exclamation mark after it. We’re still confused about that since that was the only thing we didn’t change in between regionals.

I made a Google doc for this data and started a new thread here as to not hijack this conversation about iteration between submissions.

Anyone willing to share data, please head over there with data and discussion on this point.

The Devastators 2950 completed our very first Chairman award ever and we submitted at 2 different regionals. (Dallas and OKC) We lost both time but we learned a lot in between!

In dallas we did absolutely horrible, we showed our chairmans video and kinda rambled on for a few minutes… Our feedback sheet mainly was telling us that we shouldn’t show the chairmans video and instead try a different approach. We did however get credit on our social media saying we are the most active team on twitter and facebook at the competition.

Then in OKC we learned from our mistakes and created a http://prezi.com/x_nk4qky5rj5/?utm_campaign=share&utm_medium=copy&rc=ex0share (online presentation). We brought to laptops in and did a presented by giving one laptop to the judges that we controlled using the second laptop. It gave us a more organized approach. The judges told us we need to find more problems… I am going to take that as a win. Besides that we scored fairly well.

I attached our chairmans and linked towards our presentation. http://imgur.com/a/89idc

We had a pretty interesting experience with iterations. After our presentation at Oregon City (first district), we got some minor feedback from the judges that wasn’t too hard to fix for Wilsonville (the second district). However, at Wilsonville we got some…let’s just say “constructive” feedback, and we realized that a total overhaul would be the only way to be competitive at OSU (our third and final district). We spent all of spring break rewriting the presentation, and the judges at OSU absolutely loved it - they said that they were surprised that we hadn’t won earlier.

Going into district champs this weekend, I was cautiously optimistic about our chances - I knew we had a powerful presentation, but we had only done it once for judges. However, we talked with them about our process (one of them was a judge at Oregon City), and after we won this afternoon (still freaking out internally) one of them told me that our iterations were part of the reason that we won.

Probably the biggest lesson that we learned from all of this is that you have 10,000 characters and five minutes - you don’t have to say the same things.

Also, while losing isn’t fun (especially twice in a row), it makes you better. I believe that if we had won at Oregon City or even Wilsonville, we would not have won this afternoon. I know it’s a bit too late for this season, but next year all of you should keep in mind: don’t give up after the first event. It’ll make you stronger.