![]() |
[FRC Blog] Chairman’s Award Feedback
Posted on the FRC Blog, 1/22/15: http://www.usfirst.org/roboticsprogr...award-feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Ok,
I am a bit interested in knowing how many teams have mentors in the presentation rooms. -Daniel |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
Still incredibly against the change of eliminating feedback. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
This doesn't make me any happier about getting rid of feedback. A mentor being in the room is a huge disadvantage, if you have to take away a presenter spot from a student. I've never heard of a chairman's award winning team that had a mentor in the room, has anyone else?
I'm glad they are trying to find ways for teams to continue to improve but these are all general things and not specific feedback. It takes someone going over your entire submission to really give you feedback on both what they liked and what they thought you could improve on. Looking at other winning teams' presentations and essays is a good idea but we also don't want every presentation and every team to look the same. To me that's what we are encouraging by not giving individual feedback but instead saying here look at these teams that did it right. This won't encourage innovation since you have no way to know if something worked unless you happen to win and even then you don't know if your innovative presentation style or essay writing was noticed by the judges. |
Re: [FRC Blog] Chairman’s Award Feedback
The blog first states,
"In order to provide teams with feedback that is effective and most useful, we will offer the following ways teams can get feedback for the 2015 season…" They then go on to only list one clear way to get any sort of feedback, and that's by having a mentor observe the presentation. This is something not many teams will probably do, considering the mentor would have to take the place of one of the three presenters. Even then, the mentor's feedback will most likely not help the team majorly improve their chances of winning Chairman's in the future, since they [the mentor] don't know what the judges are specifically looking for in teams. Since they only gave one, poorly elaborated upon reason as to why they removed the feedback forms, does anyone have any potential other reasons? This just seems like a largely unfavorable change. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
The feedback form had problems but eliminating it isn't the answer. If you want feedback back please contact FIRST and let them know, talking about it on CD only does so much. Tell Frank when you see him or the judges or anyone else involved in making this decision, not everyone reads CD and knows people are upset by this. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
I've had a lot of time to think about this decision to eliminate the Chairman's Feedback Form and I have to say I'm no more pleased with it than I was when it first came out. The feedback form was the only method for teams to get any sort of official and concrete information on the strength and weaknesses of a submission. I know for 1114, the feedback forms we received over the years helped lead us to new levels of accomplishment and were a critical to our path to the Hall of Fame. Not only did they help us improve our submission, but they also helped lead our program in new directions which allowed us to come closer to our goal of achieving culture change. Judges gave us concrete examples of things we could do to help strengthen our program and we took that feedback and ran with it. It saddens me that teams are now going to miss this opportunity for feedback and growth, or at least have it limited in some fashion or another. The feedback form provided a direct path of communication between our team and the judges that has now been eliminated. In 2012 our feedback form told us that we "need[ed] to show more examples of diversity". This was an easy change for us to make to our submission, but without the feedback form we never would have known this was something we were lacking or that the judges were stressing. There was no other way of knowing this.
At its core FIRST is an educational program for students. We want to see the students learn as much as possible through their participation in the program. In the classroom the feedback loop between teachers and students via the grading process is integral to the learning experience of the students. Especially in courses that focus on writing and presenting skills, students thrive on feedback from their teachers which allow them to develop and improve their skills over time. The Chairman’s process should be no different. We’ve all witnessed numerous students evolve by being part of a Chairman’s submission. They’ve become talented writers and presenters, and used these skills to earn scholarships and job offers at the top universities and companies across the world. But by eliminating the feedback form, we’ll be depriving these students of an incomparable experience. Where else other than FIRST do students have the opportunity to get feedback on their presenting skills from executives from Fortune 500 companies and leaders in our community? The Chairman’s feedback form represents a unique facet of the FIRST® program that we should not let go of. Consider the analogy to the classroom; imagine that students work for an entire semester on major paper and presentation. After everything has been submitted and evaluated, the teacher steps up to the front of the classroom and announces the name of the student with the best overall submission. No one else is given any grades, feedback, or comments on their work. This would leave the other students feeling slightly dumbfounded and definitely with no sense of how their work was received or what they should improve in the future. Quote:
Quote:
Quote:
Giving proper feedback takes time. I fully understand that our judges are volunteers who are extremely busy. With some events having upwards of 40 Chairman's Submissions, giving feedback to all teams based on their entire submission must be an incredibly time consuming and difficult process. It is definitely asking a lot of the judges to have to complete these forms. At the same time the Chairman's Award is FIRST's most prestigious and important award. If we're going to hold this award as high as we do, and expect teams to hundreds if not thousands of people-hours into their efforts and submissions, I don't think it's too much to ask that they still receive a short feedback form that summarizes their strengths and weaknesses. If the check box rubric is too rigid and quantitative for tastes of some, then do away with it. Go with free form text boxes for strengths and weaknesses. It's still essential that teams get back some comments from the judges to help them know how to improve, not just when it comes to Chairman's submissions, but also in their own personal development of their writing and presenting skills. If the workload is the issue, perhaps we can remove the expectation that feedback forms be handed back at the end of the event. Perhaps they can be emailed to the teams a few days after the event. Maybe we could have one person just do feedback on the essay (which could be done long before the event and only done once a season), and one person do feedback on the presentation. Thereby splitting and lessening the load on the feedback writers. I'm sure as a community we could come up with numerous ideas to streamline this process. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
One thing that crossed my mind - a "court reporter" type position that can take the judges' quick notes on a team's essay and presentation and type it up on behalf of the judges. This could be available after the event for teams that want to retrieve it. I see it working this way: The reporter is not a judge but merely taking notes. After a team's presentation, the judges can discuss quickly among each other and communicate a few small things about the team's presentation, and pass that along to the reporter (or the reporter can be present while they are talking). The reporter types this up in a standard feedback form, and when finished, these forms are provided to Pit Admin in an envelope for teams to pick up. There are probably a few issues with this idea. Privacy is one that immediately comes to mind. But perhaps this volunteer doesn't necessarily even need to be in the room during the presentations. If the judges keep track of the order in which teams present, the reporter can assign a letter or number on the feedback pages, and the Judge Advisor can stuff the appropriate team's feedback into their already-numbered envelope. The reporter never sees more than an A-B-C letter. Another issue is that this is a somewhat trusted and difficult position to fill. You need someone that is professionally literate and able to communicate the judges' feedback - but maybe this would be a great entry position for a corporate volunteer with little FIRST experience. Someone that types quickly and would be able to meet that strict deadline is also helpful. :) This feedback might not be more than a couple sentences, but it is something. Anyway. A quick idea, perhaps not the best, but the first thing that came to mind. Feel free to expand or poke holes through it. What else can we come up with, CD? |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
http://www.chiefdelphi.com/media/papers/3095 |
Re: [FRC Blog] Chairman’s Award Feedback
I wanted to just let this go, but since the soapbox seal has been justly broken and I still find the decision... perplexing... I'll fire up my old soapbox for the first time in a while.
In the sixth grade, I had a science teacher that had a lesson the first day to help refresh the idea of the scientific method in our young minds. The tool to help us was a menacing black wooden crate casting a shadow from the front table onto the floor below, affectionately called "Die Wunderbox". The question: what would happen if he poured clear water through a funnel in the box? We took time observing the box, and due to its large nature, its rusting MasterLock keeping its contents hidden from us, and the teacher's general wackiness, we hypothesized that the squirrely tube at the base of the box would output something different into a glass beaker on the floor, transforming the clear water he would pour into a funnel on the top. Our hypothesis was supported by the test. Clear water went in, and a lot of dyed water came out. Through this single input into the black box, we received a different output. How did the box change the input on the way to its output? He promised he would show us on the last day of eighth grade. Almost 3 years later, June rolls around and the whole 8th grade class of around 100 students crowds Mr. Longworth's room to see him reveal to us the inner workings of the mystery black box. For years we wanted to see a peek behind the curtain, a hint of what it could be. He pulls the key out of his pocket, jiggles it into the lock, and turns all the tumblers. The air is still in the room as the door swings open. Then we finally all see the secret... It was a bright haired, wrinkly plastic troll doll, grasping a sign reading "Nice try, Suckers!" After all the anticipation, after youthful curiosity had reached its zenith, we literally got trolled. He took out the mechanism that gave us our output, for no reason other than a couple laughs at anguished students at the height of puberty. FRC judging is kind of like the menacing, splintery black box. Teams spend all year preparing for a couple sessions where they can provide input to the judges. For all judged awards except one, the inputs of time and effort into executing robot design and growing the team's outreach and reaching organizational milestones were funneled into the black box all at once, and out came all the awards winners. All except for one. The "black box" for the Chairman's Award was never fully clear, but it was a far cry from the black box. Through its fuzzy, translucent shell one could reasonably see shadows of how their inputs of the written submission, presentation, and Q&A would generate a slightly clearer output than one you would get from the splintery, opaque box used for all the other awards. 422 was able to take the output of feedback from the judges over a 3 year period and turn a total dud of a team into a good team with a total dud of a submission, into a team that took home the award for the first time in its 15 years. Still, we kept trying to pry open the box; cut the lock, break the glass, just see what was really happening inside the Wonder Box of Chairman's decision making. Instead, we come back later to see the box ajar, ready to have its lid peeled open and reveal... a troll... a bright haired, wrinkly plastic troll doll, grasping a sign reading "Nice try, Suckers!" |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
From Franks section of the blog post..."We have received some feedback about these changes from individual teams, as well as our Hall of Fame teams, expressing concern about the lack of written feedback for teams this year"
From the judges note in the blog post...."Teams learn best from each other. (Your team can learn so much from the Hall of Fame teams)" Maybe we should listen to the Hall of Fame teams then. Like Karthik and Amanda showed, there are solutions. Give the HoF teams a chance to have some input on the matter. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
I definitely think the option of a silent mentor attending should not eliminate a presenter. It kinda sorta made sense before when it was introduced, but removing the feedback form means we have to make other changes as well, and this would be one I would expect. I would also like to see this optional mentor position be opened up to more than just a team mentor. Allow any option silent observer.
This would allow teams (hall of fame, or teams that seem to always win a regional chairman's award, etc) to offer up their services to others at their regional to provide feedback. I think even more important than feedback from the judges is feedback from the teams who have already achieved what your striving for. They know how much work it is, they know what they did and probably have a good idea of what you do (how many judges follow other team's Twitter and Facebook accounts to keep up to date on local activities?), and they probably know and understand any limitations your team is dealing with. They can tell you what areas you are weak in, and highlight items that you may have done that didn't come across in the presentation. Unlike judges, they can be around to help you team practice and improve in the off season, or able to sit in before the event next year to provide extra feedback. And on the plus side, they haven't already seen the presentation a half hundred times! |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
I too find this removal of feedback disturbing... and their justifications sadly lacking. I also concur that the ideas mentioned so far regarding (1) feedback on the written submission which could be done once before competition as well as the idea of (2) assigning someone to serve as a "court reporter" to type up individual team feedback. These are really good and helpful ideas and if we continue working together on this topic I suspect we can come up with more.
I do have one one question that I have not heard yet... Having judged at many FLL tournaments both local and state I am intimately familiar with their scoring rubrics and the judging process used at that level. And through that process I have learned that the perception of individuals viewing the same materials and hearing the same facts can vary widely based on their personal experience and area of expertise. Don't the FRC Chairman's judges use some sort of objective scoring system (eg. a rubric of sorts) to narrow down the field for discussion during their deliberation process? It seems to me that providing access to that score sheet at least provides some specific feedback to the team on how well their message was communicated and therefore if their presentation/submission was received as intended. Providing at least this portion of feedback seems like it would require almost no effort since these scores are being accumulated anyway. Maybe if we could combine this "low cost" rubric-like numeric scoring feedback with the somewhat off-loaded "court reporter" free-form subjective feedback, it seems like we would be doing everyone in the community a service. Providing no feedback what so ever creates a process that lacks transparency. This can breed the appearance of favoritism and unfairness because there is no evidence refute it. This is not the culture change we are all striving for. |
Re: [FRC Blog] Chairman’s Award Feedback
Having served as a judge for an FTC qualifying event recently, I was heavily disappointed that I was not allowed to give constructive feedback to teams.
|
Re: [FRC Blog] Chairman’s Award Feedback
To sort of add to this discussion. Surprisingly some of the best feedback we have ever received came from judges at the two events where we won the Chairman's Award. The feedback on the form was helpful, but in both cases members of the judging team talked directly with either myself or a few of the team members and gave us feedback. Some of it was just congratulatory but part of it was talking about things they liked and also a little about what distinguished our submission from the others at the event and advice on how to improve for Championships. This clearly can't be done for every team but to me we should be trying to get closer to that level and not moving away from it.
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
So why are those programs not standardized? How is that fair to the teams? Why, if different regions can decide to do whatever they want for the younger programs, does FRC lock down any and all chances at team feedback?! Something is seriously not right here. (P.S. I sincerely hope that HQ doesn't respond to this by taking feedback away from all programs. That's not my intent in commenting about it. I'm much more on the side of feedback in as many ways as possible, but like I said - much longer post incoming later.) |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
As noted from this tiny sidetrack on the main discussion, there are so many -other- things that should be corrected surrounding judging... I think this gigantic step in the wrong direction should not have been the first on the priority list. (Again - long-form response coming later. I've got a day job now...) |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
Quote:
I think the single sentence I quoted from the judge manual was stretching things as it is. As the manual isn't posted publicly (to my knowledge) I'll choose to keep it that way. With the disclaimer that I believe that if HQ is indeed not allowing feedback, the judge manuals really ought to be available to teams. But that's not my call to make, and they have their reasons. FTC teams are encouraged to use the Self Reflection Sheet; I haven't seen a version for FRC. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
So with the admin update today, it looks like the mentor can be in addition to the 3 students. That should help some of the issues of not having feedback.
http://www.usfirst.org/sites/default...Bundle0123.pdf |
Re: [FRC Blog] Chairman’s Award Feedback
I realize that judge feedback is not always perfect, and can be upsetting. But good gracious, it is far better to be upset by something that you know, than to be upset with what your imagination is fabricating inside your skull. Doesn't FIRST teach us that knowledge is power? I just don’t see how this decision is in the best interests of students trying to learn and improve. It seems like a decision that causes less trouble at an administrative level.
As a project judge for FLL, I can admit that providing fair, constructive feedback for every team was a challenge. It’s time consuming during and already time crunched day, and sometimes I've just been at a loss for words. But to a degree, it keeps you honest and fair, and gives you a solid place to “negotiate” with other judges. By taking away all feedback, FIRST is turning the Chairman’s Award into a blackbox that allows for an uncomfortably low level of transparency. Normally I can at least see the merits of both sides of a controversial FIRST policy change. But the more I think about this and discuss it with other mentors, the more I’m convinced that it’s not in the best interests of the students. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
We shall see. FIRST is trying something out. FIRST has changed their minds "after the fact" before due to overwhelming complaints from teams. It's not unprecedented, and if this doesn't work in the way FIRST hopes it will, it can be changed back. |
Re: [FRC Blog] Chairman’s Award Feedback
As someone who presented Week 1, having the mentor in the room was somewhat helpful.
It's better than not getting any feedback at all, but he wasn't able to offer more than 'they really liked you and were very interested in that thing' because he can't read their minds. We won Chairman's at that event, and we don't really know what could be improved for District Champs, since we didn't get any direct feedback from the judges. Sure, having the mentor in there is better than no feedback whatsoever, but it's not very helpful in terms of improving the presentation for subsequent events. I think the feedback form should come back next year - a mentor, as wonderful as they are, doesn't know what the judges were thinking, and it's way harder to improve without it. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
So, in closing. If it comes back, t should be mostly qualitative comments on why the judges chose the way that they did, and actually talk about things that could be improved, rather than an arbitrary numerical ranking that can completely mislead and can change from team to team as the day goes along, included with only a couple lines for random comments. It should be completely opposite. |
Re: [FRC Blog] Chairman’s Award Feedback
Also, I really look forward to hearing from other teams with this new presentation video we are supposed to submit, and more importantly I look forward to seeing all of the other winning presentations from this year. Honestly seeing what all the other presentations look like will be more valuable than any judge feedback form would ever be. The feedback forms usually left us more frustrated and confused than enlightened. By actually seeing what the other teams did will be inspiring and educational. So whatever happens, THAT needs to stay. The presentation video is a change that I welcome wholeheartedly.
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
http://www.usfirst.org/roboticsprogr...teer-resources |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
Take a look at the feedback form I posted earlier (http://www.chiefdelphi.com/media/papers/3095). Having judged feedback is nice but I think having a third party (perhaps an English teacher at your school) fill out some sort of feedback (to avoid bias) will likely help you iterate. And I still agree with you, lack of feedback is terrible. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Interesting. I strongly dislike how having a mentor takes away from having 3 student presenters. In this case, I feel that it is best to have 3 student presenters. Your students are ideally what have made you worthy for the CA, and therefore you should have as many students as possible to showcase your accomplishments.
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
Something interesting happened at Mt. Olive this weekend - the JA invited one mentor and one Chairman's presenter for an all-team 'meeting'. It wasn't exactly a feedback session, but they handed back everyone's USB drive & forms, and then shared a few tips that they noticed from this round of presentations. (Things like, remember that setup time is included in your minutes, etc.) Their first one: Make sure your mentor-observer has a good poker face. It'll totally throw off your students, and could (potentially) be seen as coaching if your mentor is actively reacting to the things your students say. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
This is what happened at Palmetto and it was very unfortunate. As our first time presenting for the Chairmans award it would have been really nice to have some sort of feedback as to how it went. We think we have a lot to improve on, but without anyone taking notes in the room, we really can't be sure of details. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
We didn't know there was an issue until we actually went in to present. It became public knowledge by the end of the day, but there was little more that we could do than to protest to the Administrators. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
As a coach of a FTC team, I also want feedback. It helps me tell the students what they did, where they can improve and what they should change. It also helps the students realize this for themselves. As someone that volunteers at every single FTC event in Minnesota, I don't spend events with my team and if it were not for the feedback I would be clueless as to how my team did in judging. It blows my mind that feedback isn't allowed to be provided at both the FTC and FRC level. Disclaimer: Giving feedback to teams at events in MN is not up to me, I don't make that decision, I am just an incredibly strong supporter of feedback. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
Going through this process for the first time, we have a lot to work on so it's easy to identify areas to improve on. I expect as we mature, having the feedback of how the judges interpreted our presentation would be much better if received it from them vs. trying to figure it out on our own. |
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
Re: [FRC Blog] Chairman’s Award Feedback
Feedback is a multi-faceted topic. I want to mention an excellent read - "Thanks for the Feedback - the science and art of receiving feedback well" by Douglas Stone and Sheila Heen. I've had quite a bit of training on feedback through my job the past couple of years and see the pros and esp. the cons if it is not done well and done appropriately for the circumstances. It is a topic that is much mis-understood.
|
Re: [FRC Blog] Chairman’s Award Feedback
Quote:
|
| All times are GMT -5. The time now is 17:42. |
Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi