The Fraud of FTC Worlds - How FTC & FIRST have failed me forever.

[cadandcookies]

Originally Posted by Alan Anderson

Asking to replay a match based on a disputed call shouldn't get a favorable reaction in any case. The rules do not -- and in my opinion should not -- provide for such a thing. If the referees stand by the call, then accept it and move on. If the call is deemed wrong, a replay is not necessary. Just correct it.

Agreed. In my opinion the only reasons for a replay is a field fault. If a ref missed a call, but realizes it, then it's not a valid reason to replay. Fix the score and move on. Replaying the match is a waste of everyone's time if we know what the outcome should have been.

[gblake]

Originally Posted by cadandcookies

Agreed. In my opinion the only reasons for a replay is a field fault. If a ref missed a call, but realizes it, then it's not a valid reason to replay. Fix the score and move on. Replaying the match is a waste of everyone's time if we know what the outcome should have been.

I'm not disagreeing with you. Instead, I want to summarize what was discussed earlier in this thread.

For the 2105 season, whether to conduct a replay in the situation being discussed wasn't a matter of opinion. It was a matter governed by the rules.

You and the authors of the rules essentially agree about this.

[cadandcookies]

Originally Posted by gblake

I'm not disagreeing with you. Instead, I want to summarize what was discussed earlier in this thread.

For the 2015 season, whether to conduct a replay in the situation being discussed wasn't a matter of opinion. It was a matter governed by the rules.

You and the authors of the rules essentially agree about this.

Yup. We do, on this partiular matter. I'm still saying the same thing I was six pages ago. That's part of why this thread has rubbed me wrong this entire time, and why that petition floating around irritates me, even if it has some good fragments of points.

[gblake]

Originally Posted by cadandcookies

... that petition floating around ...

Scooby Doo says, "Ruh??? Rhat Rhetition?"

[Kpchem]

Originally Posted by Conor Ryan

Would would be the ethics of calling this a Arena Fault and replaying? Especially if teams and at least one volunteer agree

Originally Posted by 2015 FTC Manual

<G14> Matches are replayed at the discretion of the Head Referee and only under the following circumstances:
a. Failure of a Field Element that was likely to have impacted which Alliance won the Match.
b. Loss of control of a Robot due to a VERIFIABLE failure of the tournament-supplied FCS computer, FCS software, USB Hub, or Gamepad that was likely to have impacted which Alliance won the Match.
c. Loss of control of all four Robots due to a failure of the Field’s wireless router that was likely to have impacted which Alliance won the Match.

G14 clearly lays out the conditions for a replay. The term "Arena Fault" is not used. There is no clause under the rules to replay a match for a wrong call. The call is simply corrected if the referees agree that the initial call was wrong.

Originally Posted by nrgy_blast

This is where this discussion has headed for me: change the manual! This has gotten ridiculous enough, and FIRST can do better in the future than they have in the past.

What would you change in the manual? Simply removing the clause the prohibits video, images, and media from being used in discussions with the referee?

The problem I have with this is the amount of overhead it adds. Ignoring the costs that official review systems bring upon organizations because that has already been addressed earlier in this thread (and because some could argue that having the teams do the video reduces the cost), the amount of time overhead could quickly become a huge problem. How other major organizations solve this problem is by only giving teams a certain number challenges (the NFL gives each team 2, in the MLB it varies between the game type but it's generally 1-2 per team per game). However at an FTC event, there are many, many teams. I'm going to run through a hypothetical scenario for a 40-team event.

Let's say that, under this system, each team gets 1 "challenge" where they can use video evidence in their arguments with the referee. So the team very calmly approaches the question box and talks to the head referee raising their points. Because of the nature of video evidence, the head referee will need to spend ~3 minutes looking at it, examining it, replaying the video while consulting with the referees involved with the call, and then making a decision. Now let's assume that every single team has an issue with a call made by the referees at some point during the event. 40 teams * 3 minutes per review adds an extra 2 hours to the event. 2 hours that aren't built into the schedule, and cannot be planned for. That's a huge amount of time and if it had to be built into the budget would severly cut into match playing time for teams, because events should not run 2 hours behind schedule.

On top of that, referees catch enough flak as it is for calls that they make, and opening them up to video review would likely only make this worse.

I understand that FIRST is for the teams and that it is important to get the calls right, but a system like this is incredibly hard to implement successfully.

Originally Posted by Nemo

I had a chance to referee an FLL event this year and thought the conferences at the end of the matches were great. The teams and referees go through the scoring chart and verify that it's right at the end of each match. Teams walked away from the matches knowing what deductions were made and what they got credit for, and the students corrected my inevitable mistakes.
...

I know this is done at some events, and I think implementing it as a standard across FIRST would do a lot towards preventing situations/misunderstanings like this from occurring again in the future.

[FTC5110]

Originally Posted by Kpchem

The problem I have with this is the amount of overhead it adds.

Granted it would be unresonable to add some kind of review mechanism during qualification matches. However with FTC only 4 alliances go through to eliminations and there are a maximum (tied matches ignored) of 9 matches to determine the division winning alliance. It's hard work making it to eliminations and teams really deserve a fair shot without being cut down by errors.

[Kpchem]

Originally Posted by FTC5110

Granted it would be unresonable to add some kind of review mechanism during qualification matches. However with FTC only 4 alliances go through to eliminations and there are a maximum (tied matches ignored) of 9 matches to determine the division winning alliance. It's hard work making it to eliminations and teams really deserve a fair shot without being cut down by errors.

I would agree that possibly giving each alliance a "review" card in addition to their timeout cards is a possible solution to the the time issue, and honestly one I didn't initially consider. It still doesn't solve the problem of the referees being under more scrutiny from the parents and teams, but I'm not sure if that issue can be solved while at the same time introducing a formal review process.

[Foster]

I'd like to thank all the people that responded to this thread that are not presently FTC or VEX referees. We have a huge need in these programs and your willingness to do these jobs is a blessing.

Make sure you contact your nearest FTC or VEX team, there are lots of summer events that you can go to in order to practice your skills.

[EricH]

Originally Posted by Kpchem

What would you change in the manual? Simply removing the clause the prohibits video, images, and media from being used in discussions with the referee?

I really don't think that's where he was heading. I suspect that he was aiming at including "challenged score" as a reason for a replay, or something along those lines.


Incidentally, FRC's replay rules were updated for 2015. They now declare that not only does there have to be a field fault for a replay, but it has to have affected the outcome and one or more affected teams have to ask. Though I suspect that "scoring error" could theoretically be counted as a field fault...

[Kpchem]

Originally Posted by EricH

I really don't think that's where he was heading. I suspect that he was aiming at including "challenged score" as a reason for a replay, or something along those lines.

Fair enough, he could have been referring to that as well. That was my misunderstanding.

Incidentally, FRC's replay rules were updated for 2015. They now declare that not only does there have to be a field fault for a replay, but it has to have affected the outcome and one or more affected teams have to ask. Though I suspect that "scoring error" could theoretically be counted as a field fault...

FTC has a similar clause stating that it has to affect the outcome of the match. But a scoring error does not fall under any of the clauses stated in G14 (failure of a field element, failure of the FCS, or failure of the field router) and therefore can't be used for a replay.

[MichaelMcQuinn]

Originally Posted by Nemo

I had a chance to referee an FLL event this year and thought the conferences at the end of the matches were great. The teams and referees go through the scoring chart and verify that it's right at the end of each match. Teams walked away from the matches knowing what deductions were made and what they got credit for, and the students corrected my inevitable mistakes.

I do think it would be worth exploring options for correcting human errors that are bound to occur. I would entertain the option of an FLL style conference at the end of a match to let the teams verify the scoring. That could correct simple numerical errors such as, for example, assigning points to the wrong alliance for end game. Something like that swings a match, and with the current system there's no recourse if mistakes like that happen. It's unnecessary for it to be that way. For some types of errors, it would be an easy discussion and both sides would agree that the points went to the wrong alliance. In some scenarios, I could picture a team clarifying a penalty and admitting that "actually we did XYZ, it wasn't the other alliance." And realistically, sometimes the alliances will disagree and they'll have to just accept what the head referee says like we have now. At least they'd hear the reasoning straight from the ref. "I called a G99 because you did ABC." I'm not proposing a solution to those judgment situations, because human judgment by referees is going to have to take care of those. BUT, I do think a quick conference could help with an error that everybody agrees was an error, and then it can just be resolved quickly and without a bunch of bad feelings.

I am not offering any opinion on whether the scenario in this year's controversial match was an obvious error or not. I am just musing on a process change that I think could help to catch certain human errors and improve the integrity and positive experience of the competition.

It would be a tricky balance, because you'd have some people pushing the limits of grace and professionalism while other people would find a decent balance between competitiveness and respecting the volunteers and the process.

Maybe I'm wrong and this would put undue stress on volunteers. There seems to be an idea that revealing the scoring breakdowns would be bad for volunteers. But I think that would be a welcome bit of transparency, and I think it's unfortunate that currently there's no mechanism for identifying and correcting mistakes that naturally happen occasionally.

Could you expand more on how you would take a FRC Game like Aerial Assist, which is scored on the dot, and have a conference with all the teams like FLL? How does it work in FLL?

[EricH]

Originally Posted by MichaelMcQuinn

Could you expand more on how you would take a FRC Game like Aerial Assist, which is scored on the dot, and have a conference with all the teams like FLL? How does it work in FLL?

As far as how it works in FLL... It's been a while since I reffed a couple FLL tournaments, but in essence the kids and the refs check over the field and see what's been placed (or displaced) for how many points.


For Aerial Assist (or Ultimate Ascent, or Rebound Rumble), the only way you'd be able to do something like that would be to have one extra display on somebody's screen, something like X scored Y in manner Z for ABCD points, broken down for the entire alliance (I wouldn't have both alliances on the same screen as a default). Or, more likely: Y was scored in manner Z. To pick on AA, I would probably use the LAST screen from each cycle for each alliance--the one right before "submit"--and have that in miniature times however many cycles were scored.

For Recycle Rush (or Logomotion, Rack 'n' Roll, or Triple Play), it's a lot easier...

[Nemo]

Originally Posted by MichaelMcQuinn

Could you expand more on how you would take a FRC Game like Aerial Assist, which is scored on the dot, and have a conference with all the teams like FLL? How does it work in FLL?

Certainly Aerial Assist would pose a set of issues. FTC hasn't really had games like that, though. Quite a bit of the information necessary to calculate the score is sitting on the field at the end of the match in FTC games.

I've only done the one FLL event, but we simply talked to the teams and explained the scoring at the end of each match. We had a checklist that listed each mission, and I told them how many points they got for which things, then we all signed the paper. If they had missed some points that they thought they had, I explained why. In a couple of cases the students pointed out a mistake that I made, I agreed with them, and I changed it.

[GaryVoshol]

Originally Posted by Nemo

I've only done the one FLL event, but we simply talked to the teams and explained the scoring at the end of each match. We had a checklist that listed each mission, and I told them how many points they got for which things, then we all signed the paper. If they had missed some points that they thought they had, I explained why. In a couple of cases the students pointed out a mistake that I made, I agreed with them, and I changed it.

FLL has paper scoring as described above. Almost all points are determined based on the condition of the field at the end of the game. There's an occasional exception for a mission that is scored as it is completed.

FRC has been scoring near-real-time for several years now, either automatically or by entering it on tablets. Scores can be corrected post-match if needed - if something was missed before the buzzer, or if something was entered incorrectly.

When I made my comments about considering a replay in FTC (way upthread), I mentioned that I didn't know the FTC game. When it was posted that there was a scoring error, I wrongly assumed that there was some kind of automatic scoring that had failed. That would have been the basis for a replay. But it now sounds to me that the scoring was manually done based on field conditions. In that is the case, there could be some kind of team review of the scores like FLL. But remember, in FLL there's only one ref marking the score sheet. If the FTC scores are based on the observations of more than one ref, then the GDC probably figured that was the accuracy check and a separate team sign-off wasn't needed.