Lesson Learned 2016 - The Negative

[Chris is me]

Originally Posted by jweston

Walking the line between "mentor built" and "mentor enabled" can be a fine one, especially for teams that are deficient in certain student skillsets. "Mentor built" often means different things to different people.

It can be a tough call when you have, for instance, advanced students on your mechanical and electrical team but only basic skilled students on your programming team. Do you tell the mechanical and electrical teams they can't build certain things because the programmers won't be able to support it? If you do allow mechanical and electrical to build to their potential, how much and what kind of support as a mentor do you provide to the overwhelmed programmers? How do you balance success to encourage pursuit of STEM with student participation? It's not easy. The answers will be a little different for each team in any given year.

Getting back to judges, another worry I have is once in a while a judge will mistake one of our students as a mentor. This usually happens to taller, mature, highly knowledgable seniors. Sometimes I get the feeling that we weren't believed when we correct this misperception.

Quite honestly, though, while the amount of mentor involvement is a very important personal debate for teams to have internally, it shouldn't have the slightest bit of relevance to judged awards at all. And what a judge perceives as "student built" or "mentor built" shouldn't matter. If an inspired student can professionally explain a quality mechanism within the criteria of an award, they should be eligible for the tech award. If a team of students manages to inspire not only themselves but the community at large through promoting STEM awareness, then they should be eligible for EI. And no student should have to convince any judge that they are the sole entity who built a quality, professional looking mechanism in order for the team to deserve recognition for it.

[Oblarg]

Originally Posted by Chris is me

Quite honestly, though, while the amount of mentor involvement is a very important personal debate for teams to have internally, it shouldn't have the slightest bit of relevance to judged awards at all. And what a judge perceives as "student built" or "mentor built" shouldn't matter.

Respectfully, I think it's a bit too simplistic to proclaim this as an absolute. Student involvement is important. Let's consider an extreme case: if a team showed up to competition that consisted of a handful of students who drove the robot, with literally everything else handled by the mentors, do you think that ought to have "not the slightest bit of relevance" to judged awards at all? There's a reason that the judges talk to students, not to mentors.

Now, I don't mean to imply that such a case is representative of any actual teams - but I think it illustrates, as a principle, why we can't just discard the notion that student involvement is important to whether or not a team deserves an award. I obviously can't speak to the questions that the judges you observed were actually asking, or to whether or not their judgment in the matter was reasonable - but I don't think the concept itself is necessarily wrong. It's all a matter of extent.

If an inspired student can professionally explain a quality mechanism within the criteria of an award, they should be eligible for the tech award. If a team of students manages to inspire not only themselves but the community at large through promoting STEM awareness, then they should be eligible for EI. And no student should have to convince any judge that they are the sole entity who built a quality, professional looking mechanism in order for the team to deserve recognition for it.

Now here, I agree entirely. But if a student is able to professionally explain a mechanism (both in terms of operation and manufacture), what reason would the judges have to believe that the mentors did all the work?

[Andrew Schreiber]

Originally Posted by Oblarg

Respectfully, I think it's a bit too simplistic to proclaim this as an absolute. Student involvement is important. Let's consider an extreme case: if a team showed up to competition that consisted of a handful of students who drove the robot, with literally everything else handled by the mentors, do you think that ought to have "not the slightest bit of relevance" to judged awards at all? There's a reason that the judges talk to students, not to mentors.

Now, I don't mean to imply that such a case is representative of any actual teams - but I think it illustrates, as a principle, why we can't just discard the notion that student involvement is important to whether or not a team deserves an award. I obviously can't speak to the questions that the judges you observed were actually asking, or to whether or not their judgment in the matter was reasonable - but I don't think the concept itself is necessarily wrong. It's all a matter of extent.



Now here, I agree entirely. But a student being able to professionally explain a mechanism (both in terms of operation and manufacture), what reason would the judges have to believe that the mentors did all the work?

It's not in the criteria for the awards. It's not relevant. When judges go off criteria it's always a problem because then teams don't know what they are being judged on... it's a mess.


And the reason is "because it looks too professional to be done by students"

[Tim Sharp]

Originally Posted by Andrew Schreiber

And the reason is "because it looks too professional to be done by students"

IMO the main idea behind FIRST is for the mentors and students to work together to produce a machine that the students could never have produced on their own.

[GreyingJay]

Originally Posted by Tim Sharp

IMO the main idea behind FIRST is for the mentors and students to work together to produce a machine that the students could never have produced on their own.

For that matter, a machine that mentors couldn't produce on their own, either. Contributions matter on both sides.

[Oblarg]

Originally Posted by Andrew Schreiber

It's not in the criteria for the awards. It's not relevant. When judges go off criteria it's always a problem because then teams don't know what they are being judged on... it's a mess.

I don't think we can rightly expect the one or two-sentence award descriptions to be exhaustive lists of all necessary-and-sufficient criteria for award eligibility.

That the awards are judged by human judges who interview teams on the spot in the pits is more or less a guarantee that awards will be determined by a huge number of factors that are not explicitly in the descriptions. Are some of these undesirable? Of course, we humans are highly imperfect creatures - like it or not, teams are probably judged, to some extent, on whether they were interviewed before or after lunch.

However, I'd contend that along with the bad comes a fair bit of good. If a judge sees members of a certain team behaving ungraciously, that judge is probably going to be less-likely to give that team an award. I think this is probably a good thing, even though plenty of the awards specify nothing about standards of team behavior.

Originally Posted by Chris is me

That is why I followed with an inspired student explaining the mechanism in detail, which is basically the criteria for the award. If a student understands and can explain it, who cares who built it? (Though this is an academic debate mostly; it's not like my team's robots have no student involvement)

I still think that's a bit more strongly than I'd put it. I do think that student involvement in the design and manufacture process counts for something - personally, the vast majority of the value I got from FIRST was not technical knowledge of robot parts but the realities of working on a difficult challenge under a deadline with limited resources, having to learn to troubleshoot, to figure out what you need to know, what you don't know, and fill in the gaps.

That kind of meta-learning is something that I honestly don't think students can get just by watching, and so I do feel that FIRST has a strong reason to incentivize teams to actively involve the students.

They wouldn't, which is why technical judging went off without a hitch for us. The only reason they would have this suspicion is if they sent culture judges to your pit asking them 18 different "gotcha questions", then jumped on your kids for saying a sponsor EDMed a single part of the robot. The witch hunt has to stop. It's harassment, and it's casting broad judgments on entire teams based on preconceived notions.

If that's the nature of the judging in question, then yeah, that sounds pretty out-of-line.

[CalTran]

Originally Posted by Oblarg

I don't think we can rightly expect the one or two-sentence award descriptions to be exhaustive lists of all necessary-and-sufficient criteria for award eligibility.

While I agree that it wouldn't be remotely feasible, it is reasonable to believe that technical awards are based solely on the technical, rather than whether my students can maneuver through a Q&A without mentioning adult involvement.

[PayneTrain]

Originally Posted by Oblarg

I don't think we can rightly expect the one or two-sentence award descriptions to be exhaustive lists of all necessary-and-sufficient criteria for award eligibility.

That the awards are judged by human judges who interview teams on the spot in the pits is more or less a guarantee that awards will be determined by a huge number of factors that are not explicitly in the descriptions. Are some of these undesirable? Of course, we humans are highly imperfect creatures - like it or not, teams are probably judged, to some extent, on whether they were interviewed before or after lunch.

However, I'd contend that along with the bad comes a fair bit of good. If a judge sees members of a certain team behaving ungraciously, that judge is probably going to be less-likely to give that team an award. I think this is probably a good thing, even though plenty of the awards specify nothing about standards of team behavior.

For what it's worth, I think Andrew has been in a blue shirt at more events than I have been to events in any capacity. Opinions on award criteria availability can be discussed in this thread (and probably should) but Andrew has to train judges on criteria so I'm sure he knows a fair bit of it.

[Andrew Schreiber]

Originally Posted by PayneTrain

For what it's worth, I think Andrew has been in a blue shirt at more events than I have been to events in any capacity. Opinions on award criteria availability can be discussed in this thread (and probably should) but Andrew has to train judges on criteria so I'm sure he knows a fair bit of it.

Lol, I'm not THAT old.


No, Oblarg raises a good point - the criteria are relatively open and by design allow interpretation. But, it has been my experience that the best way to settle a disagreement over who gets awards [1] is to work section by section through the award criteria. It provides a common framework for discussion. Does student involvement factor in? We are human, if a student just seems overly enthusiastic and knowledgeable or even just incredibly personable, that's a distinct advantage.

Really what I was getting at is that judges should absolutely NOT be grilling students to find if the mentors or the students did the work. The award criteria includes that students must be able to describe the stuff [2]. So, if you can't describe it, you don't get an award. Who cares who did it from the award criteria perspective. NOW if a student is more involved they are likely going to be both more informed and more passionate.




[1] OMG SPOILERS judges want to give awards to other teams! This actually isn't about judging in FRC, more a decent conflict resolution skill I suggest folks pick up.

[2] Ok, I think if you read close, it says "team representative" which can TECHNICALLY include mentors.

[Chris is me]

Originally Posted by Oblarg

Respectfully, I think it's a bit too simplistic to proclaim this as an absolute. Student involvement is important. Let's consider an extreme case: if a team showed up to competition that consisted of a handful of students who drove the robot, with literally everything else handled by the mentors, do you think that ought to have "not the slightest bit of relevance" to judged awards at all? There's a reason that the judges talk to students, not to mentors.

That is why I followed with an inspired student explaining the mechanism in detail, which is basically the criteria for the award. If a student understands and can explain it, who cares who built it? (Though this is an academic debate mostly; it's not like my team's robots have no student involvement)

Now here, I agree entirely. But if a student is able to professionally explain a mechanism (both in terms of operation and manufacture), what reason would the judges have to believe that the mentors did all the work?

They wouldn't, which is why technical judging went off without a hitch for us. The only reason they would have this suspicion is if they sent culture judges to your pit asking them 18 different "gotcha questions", then jumped on your kids for saying a sponsor EDMed a single part of the robot. The witch hunt has to stop. It's harassment, and it's casting broad judgments on entire teams based on preconceived notions.