Chairman's Submission Inconsistencies
 

Normally I'd post this in the Chairman's Award section, and I completely understand if it gets moved there, but I feel like there's a very important discussion that needs to be had now that winning Chairman's submissions are public, and that question is this:

How do you proceed if you read through a Chairman's essay and notice things that are inconsistent or obviously false?

This question came to me as I was finishing up reading through all the winning regional Chairman's submissions. I'm not saying that I noticed any particular team that had this issue, but to my knowledge, there are no FIRST-provided guidelines for dealing with this sticky situation, which means it's up to us as a community to figure it out until such a time as FIRST gives us guidelines.

How would you react? Who would you contact-- the team, or FIRST, or both??

Re: Chairman's Submission Inconsistencies
 

I think it all depends on the severity and (perceived) intent.

I know every year no matter how hard we proofread, there are always inconsistencies that get through to our pit boards (which means we re-print them for every competition).

I get really nervous about it -- In fact, one year that we were awarded a RCA the "judge snippet" listed at least one thing (not major, but part of the list-off) that we did not do and I am 100% sure (and confirmed) that none of our materials or students told or implied that thing to judges. I'm sure some wires got crossed in the judge room, but it was really concerning at the time.

I think in the end it's just like Stop Build Day, unbag windows, withholding allowance, pre-build restrictions, etc. We all operate under a giant honor system, and trust each other not to break it.

I know sometimes my students will write things that start to stretch the truth, but we bring it back. I find that the problems usually come from things 3+ years ago that current students have had little to no intaction with, which is why we are leading an effort this summer to create a "comprehensive" team encyclopedia so that we can be sure all our statistics are as accurately reported as possible.

Re: Chairman's Submission Inconsistencies
 

I have witnessed this multiple times before, unfortunately. But I choose to leave things alone. Pardon me if this seems too political, but any time any of us calls someone else out on something like that, it only looks bad on us, especially if it cannot be proven. Not to mention that any attempt to do so will be considered "Un-GP" by many.

I do think, though, that we should have some kind of guideline in place. More extensive "background checks" on the teams might help, but you're cutting into more volunteer hours and it may just not be worth it. But like I said, I've seen things this multiple times, and ultimately the best thing to do on our end is just be adults about it and move on.

It's really unfortunate that this happens.

Re: Chairman's Submission Inconsistencies
 

This is definitely a discussion I saw bubbling up publicly when FIRST said they would be posting complete wining submissions online. It will be interesting to see how teams with less-than-genuine information in a winning submission are dealt with by the community near them and abroad. Regardless of feedback changes for the 2016 submissions I am hoping for, I do hope they continue to post winning submissions. It could be difficult for a mentor to explain to a team why a submission that looks weaker by a lot of measures won over theirs, but looking at essays and videos of some Hall of Fame probables with such ease is an invaluable resource.

Re: Chairman's Submission Inconsistencies
 

You know, I consider myself fairly knowledgeable about the local teams here. I know and am friendly with most of the mentors, I follow teams on Twitter and Facebook, I see posts by them on CD... Yet I wouldn't feel comfortable saying that anything in any chairman's essay from those teams is false. I just don't know everything about those teams, and can't know unless I'm actually part of the team and privy to their internal discussions and calendars.

If you come across something that strikes you as improbable, it's better to find a way to ask about it in an interested manor, than an accusatory one. Something like "wow, how'd you start up 30 FTC teams? That sounds like a huge amount of work!", versus " there's no way you can meaningfully contribute to mentoring 30 FTC with the 10 students your team has!". What you might find out, by approaching the team in a better way, is that they're doing something really useful, creative, and ultimately something that you may be able to adopt in the future!. And you know, if they are BSing about it... Then simply by drawing some attention to it by asking about it may help bring them back in line.

Re: Chairman's Submission Inconsistencies
 

I have definitely noticed some teams have stretched the truth a little bit or a whole lot. But I agree, it isn't very GP to point out those teams very specifically. It is a very unfortunate thing that happens. I think it is up to the judges to crosscheck any gray areas. I do realize it that such things that will take time from our fellow volunteers and judges. But it is also up to the teams to be honest and make that choice to be GP. But for now, I think it's best to just let things go and hope that teams are more honest in the future.

Re: Chairman's Submission Inconsistencies
 

I think a community sourced "Chairman's Audit Protocol" could be a good tool here. Maybe it could be referenced officially eventually. Setting a community wide standard for evidence of completed tasks would solve this issue provided judges or the community checked it at or before competition. I think the concept that teams do not disclose their submission, publicly, ahead of time also may make this concept worse.

This is similar to the vague financial reporting standards defined in GAAP accounting or the standards organizations like Charity Navigator check. Unfortunately Charity Navigator only audits organizations with over $500,000 in public support. If we got a few hall of fame teams to buy into this concept I think it would go far.

Re: Chairman's Submission Inconsistencies
 

Where are these posted?

Re: Chairman's Submission Inconsistencies
 

Not surprisingly, it's somewhat hard to find if you don't know where to look.

http://www.usfirst.org/roboticsprogr...ard-winners%20

I think FIRST is heading that direction, with the winners being required to record their presentation, and FIRST publically displaying essays, videos, and presentations for everyone to see. Granted, this is just the winners, but a step in that direction.
Also, I've noticed over the past couple years a change in the wording of the award criteria. IT has been hugely focused on "significant measurable impact" more and more. They want to see numbers and data more now than ever, and they will grill you for that information. I think FIRST is heading there, and I think there is more accountability now with the increased emphasis on numbers and also the increased emphasis on "the previous 3-5 years" instead of your team's whole history. I.e. we can't ride on stuff team members did many years ago.

Re: Chairman's Submission Inconsistencies
 

I agree! Its gotten better, but as you pointed out, this happens retroactively it doesn't help filter and select the winners. Generally if you have a quality judge advisor they help make sure the best teams win and they challenge the judges to do follow up research as necessary.

I also think a similar process could be put into place for other awards like EI too.

True independent evidence of the activities completed would be a big plus, referencing local newspapers, or hard evidence such as thank you letters from organizations teams help out would be an asset to an audit. You can always stage a chairman's video or pictures.

Re: Chairman's Submission Inconsistencies
 

Over the years, I’ve run into what I perceive as inconsistent or falsified claims by teams. This ranges from content of Chairman’s essays, to what students are verbally saying to judges, to materials available in the pits or team websites. I really don’t think that it’s 100% intentional 100% of the time, especially when there is a lot of student or mentor turnover on a team in a short time period.

Just as it isn’t my job to police other team’s bag-and-tag procedures, it’s not my job to police their Chairman’s submissions. If I encounter something that’s blatantly false, then I’ll probably ask some questions to find out more. A lot of the time, I’ve misunderstood what they were trying to say, or it needed to be phrased differently. It’s irritating when I have a gut feeling that a team is “padding” their submission materials, but it’s really not my place to say anything.

In fact, it’s the judge’s job to dig sufficiently deep into a team’s materials that they feel comfortable with the information. For this reason, I have my students sit down, and go line by line in the essay and any other submission materials, and “cite” where the information is coming from. We build an internal bibliography of sorts, so that if we’re ever questioned on a claim or statistic, we can just pull the source from our evidence book. Having all of your numbers/achievements thoroughly documented saves a ton of time – it’s easily retrievable not only for the judges, but anybody else that may be questioning the integrity of our team’s work.

If teams are getting away with exaggerated claims or flat out lies, then it’s a problem with the system, not necessarily a problem with individual teams. The Chairman’s Award is supposed to represent the pinnacle of what it means to be a FRC team – you are a shining example to the rest of FIRST, and should be above reproach when it comes to your facts. I would certainly be in favor of a more rigorous fact-checking process, because I think it’s important to prioritize accountability. I think having an anonymous “tip” submission could go a long way towards helping this problem, as many teams don’t want to make themselves look bad by pointing fingers at somebody else and stay silent as a result. If the concerns could be handled at an organizational level and addressed through the judges, I wouldn't mind it. I would be concerned, however, about individuals taking advantage of an anonymous system with malicious intent to “ruin” or “sabotage” somebody’s chances. It’s certainly something that FIRST should discuss as a community, and maybe we can come up with some satisfactory solutions that would put most people’s worries to rest.

Re: Chairman's Submission Inconsistencies
 

Thank you so much for all the responses so far!

I've always been an advocate of assuming best intentions, especially in situations like this where all the facts aren't known by one (or sometimes both) parties. Outside of this year, I've been on both sides of this situation-- both the one someone asked about something they thought was wrong in our essay and someone who reached out to another team with questions about their essay. I know that I appreciated being asked directly about our submission, and the opportunity to both clarify the events in question with the person who asked and to make our submission clearer in intent, which is why I personally would lean towards directly asking the team about whatever is in question.

Re: Chairman's Submission Inconsistencies
 

My proposal would be that you ask the team how they accomplished X task that seems improbable. And if there's a reasonable explanation or crossed wires somewhere, that allows them to explain or uncross. I would say that 99 times out of 100, that's the whole problem.

But if there's something really, really, impossible to explain/ignore (I've heard rumors of teams barely doing something and winning, while the team that set that thing up didn't win and was told they "copied" the other team!), then I suspect that FIRST needs to have one minor note added to the Manual. You'll notice that the Judge Advisor can be called on to answer process questions. My proposal on that "minor note" would be this: A team noticing major inconsistencies (I don't mean spelling/grammar, I mean very large exaggeration or situations like the above rumor, particularly if the team is unable to get a reasonable explanation from the team in question) in the "culture change" awards (RCA, EI) may leave a message for the JA at Pit Admin with a description of the inconsistencies, the team with the inconsistencies, and the team reporting. The JA would then presumably have the judges in question look into the report, and take appropriate action--which may go against the reporting team.

Re: Chairman's Submission Inconsistencies
 

I've run into this a little bit, and I completely agree with what's been said so far - ask the team in a nonjudgmental way to tell you more about it, and it'll become pretty obvious if it was stretched or not.

Also, as someone who has put more time that he should into multiple Chairman's essays, I'll say that this is a tricky thing when you're writing the essay. The line between 'casting something in the best possible light' and 'stretching it such that it isn't true' can be a very fuzzy line, and it's not always immediately apparent when you're working on the essay. As a result, I'd encourage everyone to assume the best: I'd bet that 9 times out of 10, it wasn't intentional deception - just trying to make something sound good.

Re: Chairman's Submission Inconsistencies
 

This. Much to presenter's chagrin, we are usually continuously tweaking lines in our scripts for weeks after the draft is "finished" based on how it comes across in practice!

Re: Chairman's Submission Inconsistencies
 

Perhaps if the judges get some training in "job interview techniques" they can more easily detect when the truth has been stretched beyond the line.

I have also seen teams who undersell themselves. An FLL team we have been prepare for the World Festival was practicing their Research Project and Core Values and did not mention many of their amazing accomplishments such as getting over 1000 students to participate in a survey and providing instructional materials to teachers all over the world AND receiving videos of the teachers actually using the materials they provided. It seems that they were so focused on implementing the mission of FIRST that they forgot that the judging sessions were their opportunity to justifiably brag about their accomplishments.

Re: Chairman's Submission Inconsistencies
 

What we always told our Chairman's team was that you had to treat the Chairman's submission process as if you were going to court.

For every claim you make, you must have EVIDENCE. Photos, video, quotations, clippings, or independent statistics. If we did something amazing but did not have evidence to back it up, we did not put it in our Chairman's Award. If we were making a good faith estimate, we would always go with the lower estimate and then cut it back another 10% for safety sake until it was clearly reasonable to all. It was better to go with a low number, than to be perceived as padding your submission. If a cool event was on the schedule but had not happened yet, we did not include it until it actually happened.

It was a high standard that was sometimes frustrating for the kids. Sometimes they did something really cool but had no record of it or they had an event planned that had not happened yet. But we would not include any claims unless there was clear evidence to back it up.

Teams need to find a standard that works for them, but I think that evidence is the key to credibility.
If I were a judge, I would politely ask teams for this evidence.

Re: Chairman's Submission Inconsistencies
 

That's pretty inspiring in and of itself.

Re: Chairman's Submission Inconsistencies
 

I agree with Oz above. I believe that teams that are attempting Chairman's have a different way of assessing the actions of their team. They make decisions based on what Chairman's team are expected to do. (of course we we will help, that is what a Chairman's team does. No, we would not do that, a Chairman's team doesn't do those things.)
I cannot tell you how often someone has come to me and told me they know a Team X is doing something or lying about this or that. When I investigate, 100% of the time, the individual has been wrong. They either heard it second hand, out of context or the misunderstood what was being said. Often individuals will hear something one way that was not close to the intent of the person delivering the information. For instance, a student walking by a pit hears "I use a 50 amp charger" and assumes the team is using that for robot batteries. I go and investigate to find that mentor was talking about his boat.
A Chairman's team is truthful, sometimes to a fault. We once turned down a Qaulity Award because the judge's description was for another team. So my default standard for a Chairman's team is to believe what they are saying. If it sounds far fetched, I would ask them to explain. If starting 20 teams with only 10 students sounds like a lot, maybe they are totaling the number of teams they have started since their first season, 15 years ago. That sounds a lot more doable doesn't it. I guess what I am saying is don't listen with a closed mind. What you hear is not always what was said.

Re: Chairman's Submission Inconsistencies
 

It could also just be what they focus on. We've started 24 FIRST teams in two years with a team of less than a dozen kids (and only about 6 of them have helped with it.) We have had direct training and support for those teams, too, not just "Hey, you should start a team, good luck with that." We know it sounds far fetched, so our Chairman's video includes superintendents and principals speaking on our behalf. Our team has two main purposes - spreading FIRST teams and training/inspiring the students on the team - so it's just that we put a lot of time into those areas.

When we won Chairman's at a district this year, the judges came to our pit on Saturday morning for clarification, and we were able to show them pictures to prove our facts. I didn't mind at all, and the students thought it was great they had done something "unbelievable." We never expected a Chairman's win, because we don't do things to win it; we have the students present for it because it is good for them.

Re: Chairman's Submission Inconsistencies
 

Wow,

Four things struck me as I read the posts here, the foruth is probably the most important.

1) The notion that I need a community to tell me what to do in situations like this is completely foreign to me. Like the cliche says, "There are two kinds of people in the world, those that ... and those that don't."

2) People saying that it's not Graciously Professional to expose a fraud, if they are aware one has been perpetrated. My reply, "Poppycock!"

3) People saying that it's not their place to expose a fraud, if they are aware one has been committed. My reply, "Of course you have a duty to expose frauds. Do not turn a blind eye when something harmful occurs in your community."

4) The fourth and most important point is to be biased toward believing, to investigate with a positive attitude, to talk to people with different roles and responsibilities to check facts, and then if you are fully certain that a problem exists, bring it up discretely (especially if you are young, and perhaps have a hard time imagining just how little of the world you have experienced so far).

Blake

Re: Chairman's Submission Inconsistencies
 

Good Thoughts.