Chairman's Award Iteration

[AllenGregoryIV]

There has been a lot of talk about teams entering at multiple events this year but now that the regional season is over we need to look back at the process and see how well it worked.

Both of our feedback forms are posted here. It's interesting to see that at Dallas where we didn't win the judges seemed to give us overall better scores. We drastically overhauled our presentation for Lone Star and improved that score but regressed in others but we were selected as the Chairman's Award winners.

How did other teams iterate on their awards process? How was the judging different from event to event for your team?

Anyone else have their feedback forms posted for comparison?

[H.A.L.]

Great idea for a thread, Allen!

How did other teams iterate on their awards process?

FIRST Team 2468 presented twice, at Alamo (which we lost) and Hub City (which we won).

After not winning Chairman's at Alamo our team was ready to make major changes to our presentation. However, after speaking to several key people we discovered that we very narrowly lost, the decision being very close between us and another team. So after speaking to, well you Allen , we decided to only add a poster to the presentation, and keep everything else the same. The strategy paid off after we won Chairman's at Hub City.

How was the judging different from event to event for your team?

The task for Judges at Alamo was very difficult, as it was a huge regional with a record number of teams competing (something like 24 teams) for Chairman's. The presentations were scheduled by the event staff. The judges in the presentation seemed like traditional judges, warm but stern/professional. At Hub City we had another unique judging experience, as there were a very small number of judges for a more normal sized event. However, the quality of judges was incredible, the warmest and friendliest I have ever met, both in the pits and in the presentation room. While this made everyone more comfortable, I doubt it had any comparable effect on the outcome compared to Alamo.

Anyone else have their feedback forms posted for comparison?

Neither of the feedback forms were very revealing. They both marked excellent in all categories, and neither offered much insight into why we won or lost. The Hub City form, however, did offer some advice for our Championship presentation.

Ultimately the new process implemented this year did not affect our team. If we had to choose a regional at the start of the year to submit at, we likely would have presented at Hub City, which we won anyway. Though for many teams the iteration process helped improve their presentation, since we decided not to alter our presentation, for us this process didn't change much.

Still, I think it was definitely a change for the better and I am glad FIRST looks to improve itself in ways like this.

-Mikey

[alex.lew]

We submitted at Orlando and Bayou. Between the two events, we modified our presentation script to further emphasize what we felt were our strong points.
Right after Orlando, we realized how much our RCA package is really "Bayou oriented". For example, FTC has a very small presence in Louisiana (though not for long), in contrast to Florida. Therefore our lack of FTC mentoring isn't seen as a "weakness" at Bayou, but it is at Orlando. While the presentation changes were definitely beneficial, I also think the different set of judges was a major factor in our feedback disparity. The correct focus needed to win RCA varies for each event.
I don't have the feedback forms with me now, but we got neutral-mildly positive scores from Orlando and extremely positive scores from Bayou.

[MrBydlon]

Going off of what the wonderful Team Combustion said above...

We submitted at the Lake Superior Regional and the 10,000 Lakes Regional. The different sets of judges and Chairman's competition were a huge factor.

Our presentation was "Rock Band" themed. By that I mean, our Chairman's team literally carried rock band instruments into the judge's room and said a ton of music related terms. When you see something like that, you have to have an open mind and enjoy the weirdness.

According to our Chairman's team, the judges at Lake Superior were a little surprised and kind of sort of "didn't get it" (that is not a knock on the judges! we knew what we were getting into when we decided on Rock Band). If you ask me, the other teams at that competition had more professional types of presentations and that's what the judges were expecting. Plus, the competition there was much tougher (many teams who have been doing Chairman's for 5+ years). Compare that to 10,000 Lakes were 14 teams presented and I know of at least 4 of them that had never done it before. The judges were laughing as soon as the team walked in and were very receptive to the creativity.

We changed very little between the competitions. The only change we made was in our "Sponsor partnerships" section. We added more statistics and measurable outcomes to that section. Other than that, it was basically the same.

It was really nice for our Chairman's team to have some practice at Lake Superior. We appreciate the chance to present two different times.

[sanddrag]

This was the first year we have taken a very serious approach and put in quite a lot of effort for Chairman's. We received very high marks at Madera which left our students disappointed to not win but enthusiastic to present again at Long Beach. At Long Beach our marks varied greatly and were rather different from the marks at Madera, which left us with a feeling that we'll never win it unless we drastically change our operation, which we are not going to do just for an award. We finished the season with the impression that FIRST has a very specific ideas of what they want in a chairman's award team and we aren't it. We'll see what 2015 brings.

[AllenGregoryIV]

Originally Posted by sanddrag

We finished the season with the impression that FIRST has a very specific ideas of what they want in a chairman's award team and we aren't it. We'll see what 2015 brings.

I have heard this from a few teams and I have to admit I don't really get. I've watched your Chairman's Video it seems right on track with what FIRST would want to see in a Chairman's Award team.

The responses so far seem to suggest that the judging at different events is varying a lot.

I think one of the best ways we can find out if this is actually a problem or at least learn about the judging at different events is for teams to post their feedback forms. I know they are sometimes critical of the team but I do think a lot of teams would benefit from being able to see what the judges are saying.

[JYang]

Originally Posted by AllenGregoryIV

II've watched your Chairman's Video it seems right on track with what FIRST would want to see in a Chairman's Award team.

The responses so far seem to suggest that the judging at different events is varying a lot.

Personally I am happy to see this thread started. I was skeptical about the new system (after seven years it feels weird changing the rules and I don't know how personally I would have dealt with a few more weeks of edits and stress!) but am glad to hear it worked for some teams!

Unfortunately, AllenGregory, you are right. Judging varies at events a lot due to difference of opinions and experience in FIRST of judges. However, I feel like that is the case for all awards, not just Chairmans.

Also, while a video may be good, personally I feel as if the videos play a small role on the regional level. I feel like the communication of the team through the essay and presentation/interviews matters a lot more. I would always review the forms that we received (I still actually have photos of the feedback forms years later) and it did help to modify the next year. But there are just so many factors (who the judges are, who is your competition, the area you are entering in, etc) that it is really hard to predict why the judges decide one team over the other in the end.

[BigJ]

I'm definitely a fan of the multiple submission rule this year. I'll share 1675's experience with it.

We tend to make "fun" presentations, skits almost. This year was our 10th year so we came up with the theme of the UPS 10th anniversary party which would be crashed by our villain from last year's presentation, Dr. Ignorance (the team is superhero themed). Choice quote from the intro: "Fools! I was but a junior in high school last year. You haven't had a taste of my true power!" Dr ignorance topples our cake prop off the table and while presenting the students build the cake up, each piece represents a part of the team (Pictures and words are on the inside of each cake piece). The foundation is sponsors and mentors, the middle layer is outreach, helping others, and GP, and the goals we reach at the top of the cake are impact on our team members and the community. I will post a pic of the prop to CD Media shortly.

Our first competition was the Wisconsin regional (WIMI). A number of years ago we decided to stop submitting Chairman's at WIMI for a number of reasons, general team disorganization at our home regional, the Chairman's "metagame", and it's usual position in our season are only a few of them.

This year FOURTY (40) teams submitted Chairman's at WI, and by my estimation of previous history of teams I knew anywhere between 12-18 of them could have won it, and there were a lot of teams I didn't know.

Here's our feedback from WI (1 = strongly disagree, 7 = strongly agree)

Originally Posted by WIMI Feedback

WIMI

1. Team demonstrates significant impact of FIRST on the team members 7
2. Team demonstrates significant impact of FIRST on the team community 7
3. Team uses innovative methods to spread the FIRST message 7
4. Team displays role model characteristics fpr pther teams to emulate 7
5. Team describes strong initiatives to help start or form other FRC teams 5
6. Team describes strong initiatives to help start or form other FIRST teams 6
7. Team assists other teams through the progression of FIRST programs 5
8. Team demonstrates mentorship for less experienced FIRST teams 7
9. Team has strong sponsors and lasting business relationships 7
10. Team can effectively convey the FIRST mission to others 7
11. Team has strong itnerest in STEM education and careers 7
12. Team shows continual improvement each year 6
13. Team exhibits numerous examples of gracious professionalism 7
14. The written submission effectively communicates the team's message 7
15. The Chairman's Award Interview was effective and informative 7

What do the Judges see as the team's main strengths?

We are so proud of your efforts -> great presentation
Continue all the great efforts

What areas could the team focus on for improvement?

You're doing all the right things!
Focus efforts on sustainability
Continue FLL involvement

Judges' final thoughts and comments?

Presentation was FUN - WONDERFUL & really got message across

The kids noted that the judges were very receptive to the presentation, and that the interview seemed to go very well.

We didn't win, but were ready to improve for Midwest Regional (ILCH) in week 6. Both of our RCA wins have come from Chicago (2011 and 2013). We didn't change much, just tweaked a few lines to highlight some points and events that we forgot about in the original. Here's our feedback from ILCH:

Originally Posted by ILCH Feedback

ILCH

1. Team demonstrates significant impact of FIRST on the team members 7
2. Team demonstrates significant impact of FIRST on the team community 6
3. Team uses innovative methods to spread the FIRST message 7
4. Team displays role model characteristics fpr pther teams to emulate 7
5. Team describes strong initiatives to help start or form other FRC teams 5
6. Team describes strong initiatives to help start or form other FIRST teams 5
7. Team assists other teams through the progression of FIRST programs 6
8. Team demonstrates mentorship for less experienced FIRST teams 7
9. Team has strong sponsors and lasting business relationships 7
10. Team can effectively convey the FIRST mission to others 7
11. Team has strong itnerest in STEM education and careers 7
12. Team shows continual improvement each year 7
13. Team exhibits numerous examples of gracious professionalism 7
14. The written submission effectively communicates the team's message 7
15. The Chairman's Award Interview was effective and informative 7

What do the Judges see as the team's main strengths?

Great presentation skills
Great outreach
TEDx

What areas could the team focus on for improvement?

More promotion of FIRST teams at all levels

Judges' final thoughts and comments?

Impressive team. Keep up the good work

We were initially a little worried about Chicago since we won RCA there in 2013 and our presentation was a tad bit similar (involved a giant comic book instead of a cake, but did have the 2 heroes vs Dr. Ignorance theme). The kids noted that the judges were more stern than the WIMI judges, but that might also be because they were tired (we got the final timeslot). In the end they said the judges still liked the presentation. They noted that they felt the Q&A went well but not as well as WIMI. The scores fluctuated a little bit, 2 or 3 sections down a point and 1 section up a point. We also did not win in ILCH.

Overall we had a great experience with the multiple submissions (though it sure was great only stressing out about RCA at 1 competition a year ).

I do agree that in different regions it appears to be judged a lot differently. Over the years our feedback forms have looked very similar - mostly top, -1, or -2 marks, with sparse or vague written feedback "good work, continue involvement, you're going all the right things, keep it up" (and to be fair to the WIMI judges they had to watch presentations from 8:15-5:30 Friday and 8:15-lunch Saturday). These were all almost the same no matter if it was a year we won or if it was a year we didn't win since about 2010 or 2011 (when I started really getting into helping the Chairman's group). I wish we got as much as what was on Spectrum's Dallas form!

(No comments in this post were meant to take any jabs. Just sharing my view of our experiences as a team with the Chairman's Award)

[AllenGregoryIV]

Originally Posted by BigJ

I wish we got as much as what was on Spectrum's Dallas form!

This is one of the things I was wondering about when I made this post. Now that we are able to iterate during the season. FIRST really needs to emphasis to the judges how important their feedback is to teams. What ever process they used at Dallas to have typed feed back forms should be implemented across events. They were able to give much more useful feedback.

Thank you UPS for sharing your feedback.

[Jay O'Donnell]

I don't have the forms on me but:
We received a perfect score at UNH but still lost to 3467 who also got a perfect score. At NU we got a perfect score in every catagory except for raising new FRC teams, which we got a 5 on I believe, and we won the Chairman's award at that event. The major change between these two competitions was that we created a visual to give the judges something to remember us by, in the form of a pirate map where along the route was everything we currently do and the "x marks the spot" was on where we wanted our team to go in the future. So none of the information we presented changed, just how we presented it.

[liviamarrs]

We presented at both the Alamo and Virginia Regionals. We were not expecting to win in Alamo, but we were surprised and encouraged with our response sheets. We had all perfect fields except for 2 sections where we were one off.

We were nervous about presenting in Alamo, because we just didn't know what to expect. I was not one of our presenters, but when I talked to them they described the judges as not asking any questions after the 5 minute presentation and even being a little distracted. For example, when we mentioned working to change SOLs (our standards of learning in Virginia) one judge talked for about 2 minutes about the Texas Standards of learning instead. The judges also didn't like how we had bullet listed our summaries to fit in more information. I think it also threw them off that our essay and presentation were organized differently. That's something we are looking to change in the future. I think that this really comes down to the differences in Regionals and understanding how to control the entire meeting. In the future we will be preparing our presenters for other unexpected judging styles.

However, the judges were very enthusiastic about our presentation anyway and called it "The Best Presentation Ever!" on our sheet. Our presentation is fairly old-school. We have a large picture book style presentation that describes the facets of the team as the parts of The Mech Tech Dragon. It's very heart felt and gets the point across succinctly and well.

We didn't change our presentation very much except to practice, emphasize the strengths in the areas that weren't given the high score in Alamo, and make the metaphor stronger.

At Virginia, our experience was standard and what we were used to. The judges asked questions and enjoyed our presentation openly. Our response sheet was perfect across the board and there weren't any critiques (although we do have room to improve). We were very worried because we didn't have one Engineering Inspiration judge come by our pit, and we were fairly certain after winning the Entrepreneurship Award that we wouldn't be winning another non-technical award. However, the real reason we didn't get to talk to the EI judges was that they already knew we had been selected for Chairman's!

[Red2486]

I don't have the data on me right now, but I thought it would be good to point out that every RCA Winner this past week won at their second event. I'll post the data from all the weeks in a bit.

[bduddy]

Originally Posted by Red2486

I don't have the data on me right now, but I thought it would be good to point out that every RCA Winner this past week won at their second event. I'll post the data from all the weeks in a bit.

Being the kind of team that wins RCA at a Week 6 event is very strongly correlated with being the kind of team that goes to more than one event.

[Karthik]

Originally Posted by Red2486

I don't have the data on me right now, but I thought it would be good to point out that every RCA Winner this past week won at their second event. I'll post the data from all the weeks in a bit.

Red2486,

I'd be very interested to see this data. In general I'm curious to see how many RCA winners this year won at their first, second, or third event.

In addition, I'd be curious to see how many teams submitted at each event. I know this is a harder data point to capture, but perhaps if someone were to crowd source the data collection via a Google Doc, we could get a decent snapshot of submission rates.

[1629GaCo]

1629 GaCo won at our second regional(Chesapeake), I think among 22 teams that submitted. We also submitted at our first regional(Virginia), but I am not sure how many teams submitted there.