|
|
|
![]() |
|
|||||||
|
||||||||
![]() |
|
|
Thread Tools | Rate Thread | Display Modes |
|
|
|
#1
|
||||
|
||||
|
Re: Judge Consistency Between Events
First of all, great job of keeping the tone of your post constructive and respectful, even while describing events that you, naturally, are a little let down about. I don't have much experience with that particular award, but I can give some insight for Judges feedback concerning Chairman's.
A couple years ago, FIRST had the judges fill out feedback sheets, essentially scoring teams on the Chairman's Award. Overall, it seemed teams found them helpful in deciding where to expand and dedicate more outreach to. A year ago or so, FIRST discontinued use of those sheets as they (to my knowledge) felt they made the Chairman's Award seemed more like an award to be won than to be earned. Essentially, they wanted it to be based on individual teams and not a checklist. This was met with a mixed response, as teams understood FIRST in their rationale behind their decision but also appreciated the insight they would receive with each feedback sheet. After that, not all teams got feedback, but some still did if they were well known or knew the right people. In this aspect, it is a little unfair to the rest of the teams who don't have these connections. Additionally, throughout the course of time, it appears there may be some consistent inconsistency or trends. For instance, FIRST wants to award as many teams as possible, so at any given regional it is unlikely one team will win more than one award that isn't drastically different than the first one. Look at the Gracious Professionalism Award and the Chairman's Award. Both are honoring similar characteristics, so it is rare for one team to win both at the same regional. I've observed this a few times, that when talking with judges they make their decisions loosely based on previous awards won and awards they will win that regional. Overall, it's not a bad practice––more teams get more awards and the awards diversity goes up. The downside, of course, is that there will be some inconsistency and confusion in feedback. In regards to them just not bringing up your name, that seems a bit strange. Given that not many rookie teams will attend the same regional, it appears off that they would completely overlook your team in that area. I would say better luck next year, but you won't be rookies then! It's good to see how you're congratulatory of the award winners and already looking forward, though. Who knows, it sounds like you have a good shot at qualifying for Worlds next year if the cards fall in the right place. |
|
#2
|
|||
|
|||
|
Re: Judge Consistency Between Events
Quote:
Second - the practice you're talking about is known as spreading the wealth. It makes sense from a team experience perspective. Third - you're right there is a fair bit of overlap in awards, a lot of the tech awards boil down to articulating design process and intent. Fourth - not bringing up name can mean a lot of things in a lot of contexts. But I have some slightly bigger concerns - Rule 1 in the Judge Room is "what happens in the Judge Room stays there" There's a lot of reasons for this, but this thread is exactly one of them. |
|
#3
|
|||||
|
|||||
|
Re: Judge Consistency Between Events
Having been a judge in the past for various levels of FIRST, I can attest this was very strongly emphasized.
|
|
#4
|
||||
|
||||
|
Re: Judge Consistency Between Events
Quote:
To your point about making sure the kids learn and work hard, that's the main principle of our team. We did a lot of teaching this year and the kids learned a lot. They worked hard on their own and I think that's because of the FIRST community and the teams they interacted with. Seeing what those teams and other teams accomplished, they knew they could attempt to do the same thing and it really showed. They were proud of the success they made and everyone of my students said they enjoyed it all. They are excited about next year and are already looking forward to off season events. Also, I would love to judge. I just love mentoring too much. Quote:
Last edited by Xavbro : 10-04-2016 at 22:10. |
|
#5
|
||||
|
||||
|
Re: Judge Consistency Between Events
[Sorry, this post is very all over the place but I have a lot of thoughts on this topic]
I think the main issue with judging, and I have thought about this a lot, is just how subjective it is, and then, with that, how can it be less subjective but still effective? We know that checklists/rubrics aren't really the answer, but how do we find a happy medium? One thing that I had never seen at a regional before, but saw this year at one event, was two Chairman's judging rooms. I've always been bummed about multiple judging rooms (there are always multiple at Champs), but there is really nothing to do about it since there are just so many teams. The event where there were two judging panels had ~30 teams applying for the award. The problem is that this makes the process a lot muddier. Not only does the Chairman's team have to advocate that they are the best role model to one set of judges, but they have to connect with those judges so well that their judging panel, in turn, then has to advocate for the team. It adds a layer of subjectivity to the whole situation; something, like starting a robotics camp for special needs students, for example, may connect better with a judge who is a special needs teacher than a aerospace engineer (totally just an example I made up). In my time in FIRST I have been a FRC Judge Assistant, FLL Project Judge, FTC Dean's List Judge, and general FTC Judge. Like many of you have mentioned, it is really tough, and there are issues like the ones that have been mentioned in all levels of FIRST and in many other competitions outside of FIRST. As a team, I think the only thing we can really do about it (besides mentioning it in the post-event survey*) is to make yourselves so good that subjectivity is a non-issue. As a mentor, I think the best thing we can do is encourage our students to understand that there is a difference between not winning an award and "losing" an award and to utilize moments like these to become more motivated. A final thought, as your team continues its Chairman's journey, is that the Chairman's Award was put in place to encourage teams to do Chairman's-like activities. When you are signing up for outreach events, running tournaments, starting teams, etc., it is important to understand what the truly positive and life-changing results will be on the team, school, community, and larger world. And, the truth of the matter is that only one team at each event can earn the Chairman's Award, regardless of how many outstanding teams are applying, but each team still has that outstanding work they have done behind the scenes that is truly making a difference!! *I don't mean that any disagreement with the judges should go in the survey, just any evidence-based concerns that could negatively effect you or other teams FIRST experience. |
|
#6
|
||||
|
||||
|
Re: Judge Consistency Between Events
Quote:
![]() Quote:
Now for the meat: Quote:
I understand what you're trying to get at, but the fundamental problem is you're seeking to make a subjective process more objective. It's a hard transition. The way FRC controls this is by training. Judge Advisors are all trained by HQ and Judges are all trained by HQ-trained-Judge-Advisors. This discussion becomes really difficult at this point, neither of us know what training involves. We haven't been through the process of FRC judging. Frankly we don't even know what the judges at each event you went to thought. We're going to get down to anectodal evidence of one perspective. If we were to have an honest and complete discussion, I think we really owe it to figure out all perspectives before trying to change how things are done. Again, I have to implore that you have to try being a judge to really see what the process is. With the expansion of districts, it's also something I have to try to be honest. Do you have any specific proposals in mind? I wish I could even say which were adopted and which were not. But if you do have any, I'd propose sending them to frcteams@usfirst.org or perhaps if you can get in touch with your local planning committee and the Judge Advisor there. |
|
#7
|
|||
|
|||
|
Re: Judge Consistency Between Events
Quote:
Bayou: http://www.thebluealliance.com/event/2016lake Lone Star: http://www.thebluealliance.com/event/2016txho We can look at a variety of stats here to see the two events were a bit different. The highest OPR at Bayou would have been valid only for 4th at Lone Star. 233, one of the winning teams at Bayou, increased their OPR by 10% between these two events. This helps to show the teams you're competing against have also improved during this time. There were 10 more teams at Lone Star. They were different. Even "consistent" judging will appear different when the thing being judged changes. What you're asking for isn't consistent. "We received X award at this event and didn't win at the following event" doesn't show a lack of consistency. You need to step back and evaluate the rest of the events to see if the decisions were really all that different. |
|
#8
|
|||
|
|||
|
Re: Judge Consistency Between Events
In addition to this, you need to know that the judges take the process very seriously. There are many checks and balances in place to ensure a fair event.
|
|
#9
|
|||||
|
|||||
|
Re: Judge Consistency Between Events
Especially with a rookie award how hard is it to go down a list of all of the rookies in attendance. There's never too many of them. That way stuff like "you weren't mentioned" doesn't happen.
One thing I recommend is to specifically try not to be forgotten. Say something that's wows them and leaves them thinking of you. |
|
#10
|
||||||||
|
||||||||
|
Re: Judge Consistency Between Events
Quote:
The "Robot Design Judging Pre-Tournament Preparation Pack" I received this year did not contain the word "advocate" or any phrasing like it. It had plenty of information about the mechanics of the judging process but only had two sentences about the deliberation process and some boxes that say "Determine Top Teams Seen by Each Pair" and "Review and Discuss Top Teams" that only imply that the Judges should advocate for the teams that they saw. Perhaps a short video might be appropriate since the document I received was already 34 pages long. It can also cover concepts such as "Gracious Professionalism" and "the kids do all the work (in FLL)" that may not be familiar to the volunteers who are sometimes being trained on the day of the competition and have not had time to have seen these concepts before. Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
For full disclosure, the mentors for the OP's team left our team, on good terms, to help a local school start a new team. They (and 3847) very graciously allowed us to work at their facilities when the school we normally work at was closed for Spring Break. We celebrated Bag and Tag with them. You see their facility in our reveal video and the reveal video for another well established Houston area team. It is very likely that we would not have done as well this season if they had not allowed us to test our robot at their facility. I do not feel that our relationship is coloring my views about the inconsistencies in judging at these events. There is now another thread in a similar vein. Last edited by philso : 11-04-2016 at 15:50. |
|
#11
|
|||
|
|||
|
Re: Judge Consistency Between Events
Quote:
■ This team seems like a “Chairman’s Award team in the making.” (Community activities, leadership, vision, spirit, etc.) ■ The team is a true partnership between school or organization and sponsors. ■ The team understands what FIRST is really trying to accomplish – realizes that technical stuff is fun, challenging, and offers a future. ■ This team has built a robot appropriate to the game’s challenges. What you are describing, to me, is the Highest Rookie Seed Award, which I assume they won if they were the highest ranked rookie. The Rookie All-Star, from my understanding (and the Awards section of the manual), is more of a rookie version of the Chairman's Award. (I wasn't at the event, obviously; I just looked this up to find out whether Rookie All-Star was that different at a regional than at our district events.) |
|
#12
|
||||
|
||||
|
Re: Judge Consistency Between Events
Quote:
Quote:
Quote:
I agree and with there being 11 rookies at Lone Star, we accounted for that and thought we would be able to stand out. The rookies in attendance also did their best to stand out too so I'm sure the judges didn't have an easy job either. Quote:
Rookie Inspiration - 6133 Rookie All-Star - 5892 Last edited by Xavbro : 11-04-2016 at 17:08. Reason: typos suck |
|
#13
|
||||
|
||||
|
Re: Judge Consistency Between Events
First congratulations to you as a team. You are an up and coming star and with the enthusiasm that you are displaying you should be very successful.
After 10 years I have come to one conclusion. If you base the success of your team on winning awards, you will have a tough time. If you base the success of your team on meeting your team goals (as long as the goals are not we want to win award "x") you will be far better off. Those goals should include inspiring your students and giving them pride in what they accomplish. Our team has a 10 year history and I can honestly say that having inspired students and active and committed mentors lead to the best rewards. That is what being in FIRST is about. Blue banners and trophies are pretty cold, teamwork, camaraderie, loyalty, gracious professionalism are the marks of success and those lessons last forever. I know this may sound corny but that is what it is really about. We love to compete... we love to do our best... we love to celebrate each other's accomplishments.... that is why students keep coming back and that is why this endeavor will change lives. Keep at it... you guys are already an All Star.... you don't need a trophy or anyone else's opinion when you know who you are. Woodie Flowers once said: "I'm not going to tell you all that you all are winners. At this point you are smart enough to know whether you are or you aren't." Woodie is smart, listen to him....:0) Good luck and we hope to see you on the field sometime. Last edited by Bob Steele : 11-04-2016 at 17:03. |
|
#14
|
|||
|
|||
|
Re: Judge Consistency Between Events
Quote:
Highest Rookie Seed is described as "Celebrates the highest-seeded rookie team at the conclusion of the qualifying rounds" and is selected by "Robot Performance", not the Judges. This is a non-subjective award and the OP's team could not have won this award at Lone Star since they were ranked below the rookie team that did win that award. Probably, the only thing the Judges have to do with this award is high-fiving the winners and possibly helping to write the script for introducing them. What you have quoted is from Section 6.14.4 which is specifically for the Rookie All-Star Award. I noticed that the section of the Admin Manual on Non-Submitted Judged Awards (6.11) does not really say how the Judges are supposed to collect the information on the teams. It also does not say that the Judges have to collect information on all the teams. I would hope that this is in the training material that the Judges are given since there is no schedule for the teams to meet with the Judges that I am aware of. If it isn't clear how and when teams are supposed to communicate their efforts and accomplishments to the Judges, it will lead to a lot of frustration. |
|
#15
|
|||
|
|||
|
Re: Judge Consistency Between Events
Quote:
Last edited by angelah : 11-04-2016 at 23:57. |
![]() |
| Thread Tools | |
| Display Modes | Rate This Thread |
|
|