Issues with the Impact Award (alt: A rant by accidental back to back champions?)

FIRST General Forum
1

Preface

As build season drawers ever closer, my window for posting this is quickly closing. In the spirit of summer CD, I have a rant to share. I wrote this post several months ago, and then sat on it.

I’m just a guy on a team. We won HOF in 2023, but that does not make me an authority on this - but may provide some perspective.

The actual post

Over the summer, I had the opportunity to attend the California FIRST Mentor Conference. It was a (long) weekend full of great conversations, presentations, and questions. One topic kept coming up from mentors of teams with various levels of experience, size, and Impact success - the numerous issues with the Impact award as it exists right now.

For reference, FIRST’s definition of the Impact award is as follows:

“The FIRST Impact Award is the most prestigious award at FIRST, it honors the team that best represents a model for other teams to emulate and best embodies the mission of FIRST. It was created to keep the central focus of FIRST Robotics Competition on the ultimate goal of transforming the culture in ways that will inspire greater levels of respect and honor for science and technology, as well as encouraging more of today’s youth to become science and technology leaders.”

I’m on the record as having some hot takes about the Impact award. One of the key lines in the talks that we give is that “the C in FRC stands for Competition, and the Impact award is the highest award available; as such, you should be competing for it like any other.”

When your robot hits the field, every person on your team, and every other team at the event, can see exactly what it is that you’re doing. When you lose, you can understand that you misplayed, another team completed an additional cycle, or that an alliance had a more robust strategy. Teams with consistent on-field excellence can be awarded with one of several awards that acknowledge a specific area of achievement. Even if you aren’t the best team at the event, there are several awards that your team can be awarded to be recognized for your efforts and accomplishments. There is a clear feedback loop that encourages progression in several ways.

With Impact, judges frequently say “there is no second place.” In FTC, runner-up teams are announced for most awards. FRC notably does not do this. Teams often feel that Engineering Inspiration is a second place Impact award (I think that it is really the first place, since winners have historically had their WCMP registration fees paid by NASA). This winner-takes-all system means that the team that performs the best wins everything - and no one else wins anything - an issue often acknowledged in reference to advancement via robot performance in the Regional model. EDIT: I wrote this post before the regional advancement changes… but the point stands.

The Issues with Impact

For the sake of this conversation, I’ve identified a handful of key issues. These could be further condensed or expanded upon, obviously.
  1. Impact is billed as the “most prestigious award,” but it is talked about least of any award in the program at the competition.

We won Impact and Champs back to back (2023 and 2024, respectively). Impact was a huge goal for our team for a long time. We stumbled into the championship win this year. Despite Impact being purportedly a bigger deal than the robot game, we received way more praise and recognition from the community and HQ for the robot win. In 2023, we didn’t appear on most (or any?) of FIRST’s socials on Saturday. Impact is (currently) announced between Finals 1 and Finals 2. We were quickly pushed off the field so that the Einstein run of show could continue. In 2024, our driver was interviewed and shown on jumbotrons across all fields. We had confetti and fog cannons, staff cheering us on, and cameras everywhere. There was a stark contrast, to say the least.

  1. The Impact award judging process is opaque - many teams don’t even realize that pit judging and Impact judging are separate components.

Teams have very little exposure to what a winning Impact presentation looks like. Winning written submissions are published on FIRST’s website, but these represent the arguably less important portion of the judging process. FIRST recently added a bonus question in the Executive Summaries that offers teams an opportunity to ask a question of their own to the judges. Judges hate this question, since they’re usually asked something nearly impossible to answer. Teams hate this question because no one has ever received actionable feedback that they could use to improve upon their outreach or their submission for the award.

  1. What defines Impact is unclear, and often contested. The award description does not mention starting teams, but it is often seen as the most valuable action.

We’re guilty of perpetuating the idea that winning Impact is only about starting FIRST teams. Our Impact work consisted almost entirely of training coaches, starting, funding, and mentoring FIRST teams, all entirely in Philadelphia. This was the need we identified in our community, but it isn’t the only viable activity. It is the only specific action called out in the expanded award description, specifically inside of the Executive Summaries section.

  1. Judges are wildly inconsistent; what training is provided does not provide suitable guidance or accountability. Inconsistent judging practices further degrade the community’s interest in the award.

I don’t fault volunteers for this. Individuals who sacrifice their time before an event to read through essays and executive summaries, and then again give up several days to serve as judges during an event, should be commended, not seen as the problem with the process. FIRST would be acting in the best interest of both volunteers and teams by providing more clear guidance on what constitutes a winning submission - I never want to hear a judge ask “but why didn’t you start more teams instead of X” ever again. In 2022, we didn’t win Impact at either of our district events. In 2023, for doing effectively the same work, we entered the Hall of Fame. Neither our work nor the narrative we used to talk about it changed significantly; it seems like judging is the third variable in the equation.

Distilling these further, the key issues I see are: community appreciation of the award, an unclear judging process (and no feedback loop to improve on), unclear definition of what constitutes impact, and inconsistent judging practices.

So what?

I won’t claim to have a cure-all solution to this issue, but I feel that it merits further discussion. For us, we aren’t allowed to present for a few more years, so we technically have no skin in the game, but I think that’s a bad attitude to take. I feel… disappointed? when coaches and students from other teams tell me that they feel like the award is a “scam”, “rigged”, or “not why they got in to robotics”.

HQ could...

HQ has been making *so many* amazing changes to FRC over the last several years - it seems plausible that this could change as well. I would love it if all presentations were recorded in the room, then played back on the field between playoff matches (similar to how the pre-recorded video submissions are played at WCMP). This one change would *massively* change how teams compete for this award.

The community could...

In the meantime, our team is going to make an effort to do what we can. We want to host Impact exchanges at every event we attend, encouraging transparency and sharing of ideas. We’re going to continue to encourage other teams to publish their executive summaries, essays, and presentations publicly for other teams to view, like we have here. Like with robots, transparency and collaboration between teams has massively raised the floor and ceiling for competitive engineering. The same could be true for Impact.

I’ll post more about this in early February, but I would love to see Impact reveals be a thing. It shouldn’t be a ton of additional work for teams, but would be super valuable.

101 Likes
2

I will also add judge’s award often had this sentiment as well, at least prior to EI.

Being on the bubble back “in the day” was very frustrating. We managed a host of Judges, an EI, then finally regional Chairmans in 2016. After that and the reframing of the award (focus on growth and rebranding) the focus became far more team sustainably oriented.

Your experience with “stumbling” into back to back wins is extremely valuable in how things are prioritized. Both at the event (show for everyone in the stands) and after the fact (that few get to see). I really appreciate the perspective.

23 Likes
3

I strongly agree. Going back to the creation of the award in 1992, we can see Dean talk about the meaning of the award: to make sure that teams were keeping with FIRST’s mission of inspiring kids. I feel that with just how much stuff you have to do to win impact (even more when you consider all of the factors listed above), most teams don’t even consider attempting (or thinking about) impact, which completely defeats the purpose of the award.

22 Likes
4

I really do agree with you on the inconsistent judging thing. Last year, some of the feedback we got from one of our events was that we didn’t change much from the year before (even though I personally believe we had). It is my understanding of the rules of the impact award that the judges cannot take into account any prior year submission for any reason.

25 Likes
5

You’re correct - they’re supposed to ignore everything from prior years. They’re actually supposed to ignore everything outside of the submitted materials and judging room. There’s also… nothing wrong with not changing significantly year to year - that could just mean you’re running a great, sustainable program.

34 Likes
6

“… promoting science and technology through FIRST programs.” would seem to put starting new teams as a, or perhaps as the highest goal. There are of course many other ways to promote STEM and the ideals of FIRST, but this seems to be a straightforward - dare I say simplistic - metric.

7

I’ve always thought about this in districts where the “most prestigious award”, even though it automatically qualifies you for DCMP, It only gets you 10 district points, on the other hand winning a district event gets a team 30 district points. Even the third place alliance gets 13 points. I honestly think that Impact should be worth more points in districts.

3 Likes
8

It only gets you 10 points, but also auto qualifies you for DCMP. Those 10 points come in to play if you don’t win Impact at the DCMP, but are trying to qualify for WCMP via the points route.

8 Likes
9

My understanding is that this is supposed to be interpreted as “we’re a FIRST team, and we’ve done XYZ” - that doesn’t need to be starting more FIRST teams. At the Impact chat, HOF teams and high level judges agree that you don’t need to start teams. I don’t know the exact stats, but most of the work that 2022 HOF winner, GaCo, did, was promoting interest in STEM, but not starting new teams. In their executive summaries, they cite mentoring a combined total of 34 FLL and FTC teams, but this doesn’t seem to play a major role in their narrative. Much more of their work revolved around drop-in programming in elementary classes, interactive activities, and promoting general STEM awareness.

4 Likes
10

CD was being cranky and cut off part of my post. I did mean to say that every Impact Winner I know of was genuinely deserving. The emphasis on starting/mentoring other FIRST teams is a long standing perception. Might be less true going forward. It is worth noting that there are some teams, we happen to be one, where building an FLL/FTC infrastructure faces significant obstacles. If you are a relatively small team or in a rural setting, or short on funding (all three sometimes!) you can’t practically mentor 34 other teams! Don’t consider this a gripe. I would like to see FRC take a look at more innovative ways teams can fulfill the goals of Impact. But, its been a while since Manchester called me up to chat…

4 Likes
11

CD has been nuking my posts as well. I didn’t even mean to post this when it went up - I was still formatting.

I think that part of the point of the award, at least for me, is that the struggle to start teams is part of the process. We’re in an incredibly underfunded school district, but started well over 100 official teams, advocated for funding, mentored many of those teams, etc. I don’t think the cool part of what we accomplished is the number of teams, but rather all of the logistics, planning, and problem solving that went in to making that possible. The judges seemed to agree, as the “award speech” focused on the fact that we organized transportation, hosted targeted workshops, and more.

I think my third point aligns with what you’re saying. If it is true that things other than starting teams matters, FIRST should help us understand that by providing more clarity in the award criteria and feedback in the judging process.

9 Likes
12

It is critically important to the future of FIRST that teams continue to get created. So, to me, it is no surprise that some may see Impact as the “top salesperson award”.

7 Likes
13

I’ve been disappointed with impact for a long, long time. When a robot wins on the field, everyone knows why. But the most important award in FIRST happens almost entirely behind closed doors. I routinely hear comments at competitions like “how did THEY win?” and “they must have lied”.

The impact award is beyond opaque. It’s a black box. And it shouldn’t be.

30 Likes
14

We’ve always looked at Impact as something that we would do if we were a team that was triple our size and had a more dedicated business team, which I think is a real shame. We’ve had ideas for things we’ve done that we could present for impact, but we figure why try when the bar is so high, when we could use our limited resource (students) to work towards robot performance.

23 Likes
15

Since a lot of people are bringing up you don’t know why someone won, I believe every impact winning essay since 2017 is located here . While this doesn’t help at the event a few days later you can read the winning essay.

Now do I always agree with the winner based on the essay? No, but at least I can now see said essay and not just go by my personal knowledge.

9 Likes
16

Top Salesperson award…harsh, but fair. Much as we all want FIRST as an organization to succeed the sort of communities and teams where a big investment in “infrastructure” is possible are so often the very places where resources already are abundant. Kudos to the teams that have started programs in inner cities and such. I tend to have an eccentric view of the world, including the FIRST world*, and would love to see a team win by for instance, starting a single FRC/FTC team…at the local Alternate School. Or in a community where the educational system is struggling to cope with large numbers of migrants from other cultures. Do one new thing magnificently…I consider it is more worthy than doing dozens of things by template. *Note, I’m rarely asked to serve on committees for some reason…

7 Likes
17

This is a great starting point, but I believe that the presentation and q&a is far more significant.

There’s also a secondary issue created by only publishing the winning essay - you have no idea what the actual field of competition looks like.

15 Likes
18

Couple more points worth making. FIRST does recognize the exceptional with regularity. I recall the team of girls from Afghanistan a few years back. Hope they are safe somewhere. And, even if a team is not in a position where a “classic” Impact campaign is feasible, it is good to enter anyway. You learn more about what you could/should be doing, and the process is a valuable one for your Impact prepare/present group.

1 Like
19

To be fair to the judges, it is very easy to see the impact of starting new teams as the results of FIRST programs is literally right in front of them. Other activities, especially those that are a unique idea by the team, will likely require more details on the results.

1 Like
20

Unanswerable thought exercise. Which is the Better Thing: 200 kids in a suburban middle school who learn a bit through FLL, or just one of those Afghan girls who becomes an engineer and hopefully returns home in happier times? ( fill in what ever else you want, a student learning English and Java at the same time, a kid in the alternate school who goes to college vs a much less happy path, etc…)

1 Like