[FRC Blog] Chairman’s Award Feedback

Having served as a judge for an FTC qualifying event recently, I was heavily disappointed that I was not allowed to give constructive feedback to teams.

To sort of add to this discussion. Surprisingly some of the best feedback we have ever received came from judges at the two events where we won the Chairman’s Award. The feedback on the form was helpful, but in both cases members of the judging team talked directly with either myself or a few of the team members and gave us feedback. Some of it was just congratulatory but part of it was talking about things they liked and also a little about what distinguished our submission from the others at the event and advice on how to improve for Championships. This clearly can’t be done for every team but to me we should be trying to get closer to that level and not moving away from it.

Come up to Minnesota! We give feedback to all FLL and FTC teams here :slight_smile:

I’m in the middle of editing a much longer-form response in this thread, but I was just about to comment something similar with regards to FLL/FTC feedback: When I judged in Northern NY, we gave feedback forms in addition to comments.

So why are those programs not standardized? How is that fair to the teams? Why, if different regions can decide to do whatever they want for the younger programs, does FRC lock down any and all chances at team feedback?!

Something is seriously not right here.

(P.S. I sincerely hope that HQ doesn’t respond to this by taking feedback away from all programs. That’s not my intent in commenting about it. I’m much more on the side of feedback in as many ways as possible, but like I said - much longer post incoming later.)

I’m not saying this to pile on to anyone or anything, but other than the names and general award qualities/quantities, I’m not really convinced there is true, enforced standardization at all FRC events in the judging process, whether or not it’s intended or official.

It was disallowed at the HQ level.

This especially stuck out to me because I missed this on the Judge Certification (as did other FTC judges at that event with illustrious backgrounds in FRC)

Hang on hang on, there’s a Judge Cert for FTC, there’s not for FRC :\

It was optional.

What was your Judge Cert process like? Ours was a ~20 minute powerpoint review before opening ceremonies.

As noted from this tiny sidetrack on the main discussion, there are so many -other- things that should be corrected surrounding judging… I think this gigantic step in the wrong direction should not have been the first on the priority list.

(Again - long-form response coming later. I’ve got a day job now…)

I don’t suppose it’s in a format you could share? I’m curious.

It was an online quiz. I’m not comfortable sharing the link; I got the distinct feeling that it is not for public consumption.
I think the single sentence I quoted from the judge manual was stretching things as it is. As the manual isn’t posted publicly (to my knowledge) I’ll choose to keep it that way.
With the disclaimer that I believe that if HQ is indeed not allowing feedback, the judge manuals really ought to be available to teams. But that’s not my call to make, and they have their reasons.
FTC teams are encouraged to use the Self Reflection Sheet; I haven’t seen a version for FRC.

I know, fortunately both I and the MN partner disagrees with that.

Having been at a few meetings where feedback (for FTC) was discussed, there are a few partners that are very adamant that feedback is not given to teams because teams then complain to them (the partner) about it. We don’t experience that in Minnesota but other areas do. My theory is that it has to do with the different cultures in different areas of the US.

We do the same in Washington.

So with the admin update today, it looks like the mentor can be in addition to the 3 students. That should help some of the issues of not having feedback.

http://www.usfirst.org/sites/default/files/uploadedFiles/Robotics_Programs/FRC/Game_and_Season__Info/2015/TeamUpdateBundle0123.pdf

I realize that judge feedback is not always perfect, and can be upsetting. But good gracious, it is far better to be upset by something that you know, than to be upset with what your imagination is fabricating inside your skull. Doesn’t FIRST teach us that knowledge is power? I just don’t see how this decision is in the best interests of students trying to learn and improve. It seems like a decision that causes less trouble at an administrative level.

As a project judge for FLL, I can admit that providing fair, constructive feedback for every team was a challenge. It’s time consuming during and already time crunched day, and sometimes I’ve just been at a loss for words. But to a degree, it keeps you honest and fair, and gives you a solid place to “negotiate” with other judges. By taking away all feedback, FIRST is turning the Chairman’s Award into a blackbox that allows for an uncomfortably low level of transparency.

Normally I can at least see the merits of both sides of a controversial FIRST policy change. But the more I think about this and discuss it with other mentors, the more I’m convinced that it’s not in the best interests of the students.

This is correct. THIS makes me more okay with the change. Allowing the mentor in the room IN ADDITION TO THE 3 presenters is key. Having gone through the chairman’s interview process several times, both as a student, and now as a mentor, I have seen the highs and lows of the feedback forms. 99% of the time, they were NOT helpful at all. They never told the kids what could have been done or demonstrated better, and they were never constructive. And half the time, we were docked for things that were clearly demonstrated, communicated, and explained in the room, binder, video, essay, etc. Still docked for it. There were some events that we never actually got our feedback in the first place. Having the mentor in the room, and receiving comments from HoF, other teams, and other judges will help a lot.

We shall see. FIRST is trying something out. FIRST has changed their minds “after the fact” before due to overwhelming complaints from teams. It’s not unprecedented, and if this doesn’t work in the way FIRST hopes it will, it can be changed back.

As someone who presented Week 1, having the mentor in the room was somewhat helpful.

It’s better than not getting any feedback at all, but he wasn’t able to offer more than ‘they really liked you and were very interested in that thing’ because he can’t read their minds.

We won Chairman’s at that event, and we don’t really know what could be improved for District Champs, since we didn’t get any direct feedback from the judges. Sure, having the mentor in there is better than no feedback whatsoever, but it’s not very helpful in terms of improving the presentation for subsequent events. I think the feedback form should come back next year - a mentor, as wonderful as they are, doesn’t know what the judges were thinking, and it’s way harder to improve without it.

I can’t wait for this pilot year to be over. the lack of any feedback boggles my mind.

Hopefully the judges will at least let a mentor in for VA so if y’all don’t win you have something for Chesapeake. We don’t have squat :confused:

I think if it does come back, it needs to be overhauled from what it was before. You couldn’t really tell what the judges were thinking with the old system either. I think it should be more qualitative than quantitative. There should be a comment section for each “category” for the judges to explain WHY they think what they do. Before it was just a ranking from 1-10, but no way of knowing why a judge chose the number that they did. Because, again, we’ve had it happen several times where they dock us for something, even though it is explained in every part of the submission.

So, in closing. If it comes back, t should be mostly qualitative comments on why the judges chose the way that they did, and actually talk about things that could be improved, rather than an arbitrary numerical ranking that can completely mislead and can change from team to team as the day goes along, included with only a couple lines for random comments. It should be completely opposite.