Pit Judging Sucks

Admittedly click-baity title, but hear me out.

Pit judging, especially with a smaller team, is a crap shoot:

  • What students will happen to be in the pit when the judges happen to come by?
  • Will the judging session get cut short because we need to leave for a match?
  • Will we be in the middle of a critical repair while the judges are talking to me and not able to fully engage?
  • What questions will the judges happen to ask? Will you actually have the opportunity to tell them everything you need about your team?

But pit judging is also important. There’s the obvious reason, it determines two Championship qualifying awards (Engineering Inspiration, Rookie All-Star) and district points. There are also the less obvious, but arguably more important reasons. Judging helps develop the important life skill of talking about your work and yourself, and judging celebrate important aspects of a team beyond on-field performance.

I’ve accepted these as inherant issues in judging, until the Wisconsin Regional this year, where my team, along with multiple other teams, were not asked any questions related to team attribute awards, only robot related questions. Effectively, this meant that we were eliminated from contentention for a Championship qualifying award because of the questions we happened to be asked in judging.

So what can we do about this?

There are some things that could be done easily, tomorrow:

  • Begin initial pit interviews with “Can you tell me about your robot?” or “Can you tell me about what your team does to promote engineering?” and then let the students talk, uninterrupted, for some period of time (say 3 minutes). Prior to the event, communicate to teams that this is going to happen, so teams can plan how they will respond. This accomplishes two things: it helps the team ensure they can tell the judges all of the most important things they’d like them to know, and it helps the judges know what the team wants to talk about, so they can utilize their time effectively. It also encourages teams to spend time talking about awards and working with students on their communication skills. By doing just this, you can improve the quality and consistency of pit interviews by leaps and bounds!
  • After a team’s initial pit interview(s), put a colored sticker on their pit sign. Prior to the event, communicate what these stickers mean and by what time you should have one.
  • At the beginning of initial pit interviews, have the judges say if they’re coming for robot awards, team attribute awards, or both. This becomes moot if the first bullet point is implemented, but helps the students know what type of information the judges are looking for, and helps to ensure they are able to present that information before the judges leave.
  • Have judge advisors train their judges to ask at least a set of standard questions to each team, to ensure they are being considered for each award (again, moot if the first bullet point is implemented).

There are also some harder things that could be implemented in future seasons. Specifically, to give teams scheduled interview times. These times could be either in the pit or in a separate room (preferred for neurodiverse students who may not be able to participate fully in a noisy pit area). The times could also be before matches, or in a gap between the team’s matches. While there are logistical challenges to scheduling interviews, this solves the issues of the students being at the judging interview being determined by random chance, and the issue of a team needing to leave for a match.

56 Likes

Just do away with pit interviews all together.

One of the few good things to come out of the 2021 season during the pandemic was the remote technical award interviews conducted via MS Teams. The students having a true opportunity to prepare what they actually wanted to talk about, hone their message, and present it on their terms was incredibly beneficial. It saddened me that we went back to such a flawed system of the pit interviews following the resumption of in-person events.

At the very least, have dedicated time slots for team’s to “present” to the judges, whether it be remotely or in person. Then follow that up with on-field observation or further interaction at the event.

44 Likes

My frustrations from wearing a blue shirt for 2 years convinced me to start mentoring.

15 Likes

Another issue which happens all the time is the judges come by during a match. This means 2 things:

  1. Everyone that has the most knowledge about the robot is either on the field or in the stands
  2. There’s only 1 or 2 people in the pit, so there is limited knowledge that can be shared.

I like the idea of having a scheduled interview that doesn’t interfere with robot operations or matches. Plus teams can say everything that they want the judges to know.

It feels like winning awards has a lot of luck to it, which comes from the judges asking the questions that prompt the answers that will win you an award.

18 Likes

Who are you and what have you done with Sean? I agree with 100% of this post.

And what’s more, their mentors could observe and support them.

I would go one step further and I would tell the students who their judges would be, send em LinkedIn accounts or a quick bio. Part of the judge role is to be a STEM Ambassador so let the teams prepare questions to ask, let there be a two way discussion.

There’s downsides to remote judging but in many years of judging I will say the best experience I had with teams was during the At Home judging in 2021. The students were able to be more comfortable, more ready, and happier. The only suggestion I’d have would be giving them a dedicated 5 minutes to ask questions to the judges.

19 Likes

I’m not sure the value proposition is there for the judges if they aren’t physically present. They’re volunteers; many were enticed by the event and its atmosphere. Turning it into a teleconference where people make practiced presentations sounds a lot less compelling.

I’m sure that it would better for some judges and some teams. But students speaking extemporaneously is also valuable and worth rewarding, and questions in the pits can readily be answered with a demonstration.

Nothing particularly wrong with having a calendar of judging times—and maybe offering some remotely vs. in the pits, to gauge preferences.

9 Likes

On FIRST’s website it says this about RAS and EI.

**An interview room may be used for this award at the District Championships.Teams should not prepare a presentation for this interview. The purpose of this interview is to replace the traditional pit interview. Please consult your district leadership with any questions.

Is this implemented at all?

What we should do is what FTC does. Before matches start, at the same time as inspection, there are scheduled judges interviews. 5 minutes of uninterrupted talking, and then 5-10 minutes of questions from the judges.

30 Likes

Some of the judging depends not only what we see from the pit but also on how the robot performs on the field so it would be hard to do remote with that. Also many times we have to come back to ask additional questions.

I’ll admit there is not some standardized way pit judging happens. But we (at least the judges I know) always ask if you have time to talk to us. If you’re in the middle of a big repair, we shouldn’t come by.

It’s often up to the judge advisors how some of the judging happens. But at least I know my groups usually start with questions like “can you tell me about your robot?” Or “can you tell me about your team?”

8 Likes

We have had success simply asking judges to come back later.

At the recent Glacier Peak district event I had this interaction:

Judge: “Hello! is anyone available to talk about you outreach efforts?”
Me: turns around to see 6 mech kids circled around the robot busy fixing something “Sorry we only have mech students here at the moment, can you give me 5 minutes to find our team captain and come back?”
Judge: “Alright we will come back later”

The exact situation also happened but reversed when some judges wanted to know about the robot, but our pit crew was at lunch.

I’m sure that judging probably varies slightly by location and whether it is a district or regional. Glacier peak seemed to have a disproportionatly large number of judges. I think I saw 5 groups of 2 and a group of 3. I believe 2 groups came back for more info, so in total we had something like 8 judge visits.

Asking them to come back later didn’t seem to be a problem since we managed to get our first ever Engineering Inspiration award.

30 Likes

I by no means think judging is perfect, but I think your rhetorical questions actually all have reasonable answers. As with anything in a decentralized program, your mileage may vary a bit from event to event, but it shouldn’t be dramatically different.

Generally, most teams will be seen by roughly four sets of judges at a minimum: two machine-focused and two team attributes-focused. This varies a bit by event, but is the standard. Sometimes the much of the draw will have the wrong students and the wrong time, but generally this approach tends to level things off.

Also, I’m a firm believer that every student should have a rudimentary understanding of every aspect of the team. And it’s okay to say “I’m the electrical lead, so I can tell you about my involvement with our outreach, but you may also want to hear from our outreach lead who can provide a lot more color” (or vice versa).

If judges don’t feel like they had enough time, they will find a way to continue that conversation, whether that means following you to queue, coming back later, or sending another judging pair your way with specific questions.

“My apologies, but we have a critical issue to fix and we’re up again in 45 minutes. Do you think you’d be able to come back later? If not, maybe you can see some of our other students who are scouting in the stands right now” or whatever. Judges have tight schedules, but if you’re flexible, they will tend to be flexible, too.

You may wish to check out the FRC Awards Workbook which is actually part of the Judging Manual and gives judges suggestions for what to ask. Answering the question you wanted to be asked, rather than the question you were actually asked, is a skill worth building, too.

26 Likes

(As I wrote this, I realized it’s a very long response. Sorry!)

As a former FRC and FRC judge, I know the pros/cons of both judging in pits and in a room. It is easier to focus in a presentation room. But it’s pretty cool to see the robot up close and see the pits and talk to teams there too. In FTC you can carry a robot into a presentation room. FRC you can’t (or if you can it’s a real pain). So that might be part of it.

At the FTC events in my area, it’s usually 20-30 teams. At the regional FRC event, it’s 50 teams. So about double. To plan the time for 50 teams to talk is definitely a larger time suck for larger events. It could maybe be done (do most of the interviews on Thursday, some on Friday or something). The logistics are just harder.

To OP’s point, I definitely think you should prep students however you can. I try to hold a “Ask A Judge” workshop at our local alliance of 15ish FRC teams and just explain what we as judges do, what to look for, how to best talk to judges in the pits, etc. I’ve even thought I should record things and post on YouTube or something to help rookie teams too. (Maybe this summer?) As for “have judge advisors train their judges to ask at least a set of standard questions to each team”, they kind of do. It’s not required, but there’s materials that judges receive that suggest things to ask. I try to let students guide lead the conversations how they want, and I just guide it gently to get the answers I need. But answers lead to more questions sometimes, and it’s hard to stick to an exact list. I’d mostly say students should focus answers on the type of award(s) they want to get or think they’re qualified for.

Interestingly, your idea of using colored stickers on pit signs is something our regional has done in the past. I’ve also explained what the stickers mean in my workshop.

As for actual talking to judges in pits, I don’t usually say what I’m coming there to talk about as I want the conversation to happen more naturally, and I try to get students talking about the team and robots in the ways they want to and I gently guide the conversation where I need it to go. I write down the summary of thoughts. They don’t have to have memorized speeches or anything, but they should know how to “tell their story” (as I usually say). What’s your bot do? Why does it do that? How did it get to that point? What did you learn along the way? Tell me about your team. Why is it like that? What’s the advantages of how you have it set up? How is work divided up? How do you ensure success? How do you keep the program going? These don’t have to be perfectly rehearsed, but these are the types of things I try to find out and I write notes about to be able to defend why I think a team should get another round of interviews, or an award later.

Is the timing bad? That’s ok… be up front. “Can you come back later? We need to repair our robot” and/or “we have more people that could speak on that topic” and/or “we’re going to a match in a few minutes”, etc. I’m always happy to come back!

As for interview times… it’s SOOO hard to do that well. I could get a match schedule and align teams I need to talk to with when they’re not competing in the next two rounds but a) times shift around a lot, b) one team may have no robot repairs, another may have a ton between matches, c) if they make changes and have to reinspect, they’ll be gone from the pit anyway, and d) probably other things I can’t think of this late. Even if I gave teams a schedule of when I’d drop by, at least half would still ask me to come later or couldn’t really give good presentations at the time. It’d only work if it was dedicated time in a separate room like at FTC, but already mentioned why that’s hard in FRC.

I hear you on the process. Pit judging isn’t perfect. I don’t know a better answer (as a former judge OR a volunteer coordinator). But I’m happy to talk out what I know and share all the stuff I can!

(Also I see Jared wrote a response as I did. He’s helpful too!)

16 Likes

Totally agree with OP on this issue. For awards beyond the scope of the robot, like Engineering Inspiration or RAS, there is currently a very fundamental problem:

The people who understand the most about our community outreach should definitely not be in the pit!

We want to keep the people in the pits to an absolute minimum: a software member for software problems and a few knowledgeable mech students for mech issues.

Though there’s certainly some overlap, the people who are most passionate and knowledgeable about our big-picture outreach are on our impact team, our executive leadership, or are typically running other aspects of team logistics at a competition. Plenty of our mech members contribute to certain outreach projects, but expecting all of them to be able to “tell the story” of our team’s community impact on a wider scale at a moments notice is not reasonable.

Now to solve this problem us (and many other teams) put our most knowledgeable people in our pit as “ambassadors” to solve this problem. But this solution is FAR from ideal. That means:

  • More students crammed into the pit, which is always a negative
  • These students, who often could have other roles, have to always be standing there, waiting with nothing else to do, because they have no way of knowing when the judges will come
  • Since we want to keep people in the pit to a minimum, we usually only have one ambassador. We could explain our outreach much more effectively if we could choose 2-3 people from the team to give a short presentation

Please FIRST, can we limit pit judging to technical awards, or eliminate it altogether? Let our ambassadors present the rest of our team in a more reasonable environment.

Impact judging works fine as it is, but as a new team we are really making a run at EI. The way it’s currently set up, it feels like FIRST just doesn’t care the same way about judging for these other awards - despite the fact that they are the best shot at Houston for many teams that deserve to be there. I would love to see this change.

11 Likes

In-person pit judging > remote pit judging. I did both and were able to better judge teams in-person then remotely.

3 Likes

Pit judging can also sometimes lead to team’s getting skipped, at my team’s regional the outreach judges skipped over our pits despite us almost always being there. This lead to us not being able to compete for EI, an award we often win.

Whenever we have team attribute judges come by our pit, the mechanical members always ask if they can quickly grab the members of our team responsible for talking about outreach. When the judges say yes, we put a message in slack or send someone to the stands to grab the person/people. We haven’t had any judge tell us no, because they know that it’s best to have the most knowledgeable group of students on that topic there to answer their questions.

As for judging right before queue or during a critical fix, we normally just ask the judges if they can come back. They’re normally understanding and come back later. Sometimes we’ll start an interview without realizing that we’re close to a match, but again, the judges understand and either come back to our pit to resume the interview after the match or just finish it up as we walk to queue.

4 Likes

Something about the Wisconsin Regional, in particular, has always seemed off for judging to me. As a team we’ve attended this regional every year of its existence, I’ve personally attended for the last 13 years. Whether its due to the fact that the team order is very difficult to follow (alternating teams by seniority – but with exceptions), or other reasons, there have been years that we’ve seen very few to no judges for the entire event. Ironically, this year we saw a lot more judges than we were used to, which was a nice change. (I literally took pictures because I had never seen that many judges speaking to our students at one time)

The suggestion to tell judges to come back later sounds reasonable, but in past years when we’ve done this we have never seen a judge again. We now instruct our students to never tell judges to come back later because we have had such bad experiences with this.

Note, we usually do not see this issue at other regionals we’ve attended. At Midwest, the regionals in Duluth, St. Louis, and more we see judges much more consistently than at our home regional.

16 Likes

This never happened at events I judged. If you didn’t impress the first group you got, good luck.

I don’t know that I’ve ever seen this level of judging interaction at any FRC event I’ve attended as a student or mentor.

14 Likes

Yes, at least in New England.

I’ll try to pick my words wisely in responding to some of your questions and respond wearing my mentor hat since I can say more :slight_smile: I will say the judging process is much easier said then done… The BIGGEST piece of advice I have for you is to volunteer to BE a judge and get the inside scoop on how it all works!

My team’s student leadership sees the importance of EVERYONE knowing a little bit about everything, so if they do get caught in the pit, they can talk to the judges until the pit representatives are able to make it to the pit. To this point, one of the things they’ve done in the past is play a game so that everyone knows all the sponsors.

Most likely yes, but the judges can come back later if they have any follow up questions. I know at the event that my team was at (36 teams), everyone only had a few matches between when they left the field and when they needed to be back in queue, but the judges made it happen. I will say that they usually try to place experienced judges with more rookie judges, and the experienced judges can tease out some of the answers by looking through pit materials or at the robot.

If this happens, you can politely tell the judges this and ask if they can come back in a bit. We’re human, we understand! :slight_smile:

While the judging process is consistent, the timeline varies a bit based on variables like how many teams are at an event and how many judges are there. There’s 2 sets of judges that come by on the first day of qualification matches (regionals), one will be evaluating your team for MCI awards and the second will be evaluating your team for team attribute awards. Then on the second day of quals/elims judges will be re-interviewing teams for specific awards that they have been nominated for.
I will say the one question that we ask all teams is: “Is there anything else that you would like to tell us today that you haven’t told us/covered?”

This is what I usually do, and I would imagine that most other judges do it this way. Coming in, we know nothing about a team so we need them to provide us that information up front.

I would strongly recommend that you bring this up to your RD, who can reach out to the JA and ask them about this. Usually we have a sheet that we track teams that we’ve talked to and in the case of champs, they have a student initial that we talked to them (which I think is a great way of doing it).

TLDR:

  • Volunteer as a judge and learn the process! (Plus we always need judges!)
  • Have your team practice what they’re going to say to judges both technical and non-technical.
4 Likes