Instant Replay Challenge—A Thought Exercise

[Tristan Lall]

Although I'm not opposed to a structured implementation operated by the competition, I think it can be done differently, and with most of the (small) costs borne by the teams. As I proposed in that 2005 thread, an off-season event should try giving teams one instant replay coupon for the qualifying rounds and one for the elimination rounds, and then providing a brief window to present video for post-match review. This is practical and equitable.

A student alliance member would go to the challenge box and remit the coupons of all alliance members that showed up for the match (the whole alliance must agree it's replay-worthy), along with whatever video they want (cued to the crucial moment), within 1 min of the end of the match. The head referee will see the challenge is being made, and arrive to look at it (if the referee takes longer, then great, you have more time to cue the video). First, the referee decides if the video (as shown on the playback device) has enough information to show the situation at issue, then decides whether it changes the outcome, and finally, decides whether the evidence is sufficient that the video has not been tampered with or misrepresented.

If it fails the first test, the referee puts an end to it within moments. (For example, if there's not enough detail, or it doesn't show the right part of the field.) If it fails the second test, the team then knows it's not just the referee missing the call, but rather a deliberate decision to call it a particular way. (Rightly or wrongly, at that point it just becomes another futile challenge.) The third test may be a little more complicated—possibly requiring rewinding the video to hear the match being announced, or to verify that the right robots are on the field—but it's a straightforward test of credibility. (And because the burden of proof is on the team, and the timeline is tight, there's really no concern about the wrong video being reviewed, at least in the qualification round. In the elimination round, with the same teams playing, identification is more difficult, but at least the schedule isn't usually as constrained there.)

Why this? Because it strictly limits the number of reviews to minimize the aggregate delay, but still gives teams the opportunity to be heard. (Teams like it better when the officials hear them out, instead of ruling summarily.) It provides the referees with a procedural reason to change a bad call, without appearing indecisive, and gives the head referee a tool to better manage the rest of the referee crew (by understanding which calls are being blown). It also serves the teams notice that they share in the responsibility—if interested in challenging the officials' version of events, plan ahead to ensure that you're making video.

It's not perfect, but it doesn't have to be. I think you'd even see teams collaborating to supply post-match replay video to each other.

[coalhot]

Originally Posted by Tristan Lall

Although I'm not opposed to a structured implementation operated by the competition, I think it can be done differently, and with most of the (small) costs borne by the teams. As I proposed in that 2005 thread, an off-season event should try giving teams one instant replay coupon for the qualifying rounds and one for the elimination rounds, and then providing a brief window to present video for post-match review. This is practical and equitable.

A student alliance member would go to the challenge box and remit the coupons of all alliance members that showed up for the match (the whole alliance must agree it's replay-worthy), along with whatever video they want (cued to the crucial moment), within 1 min of the end of the match. The head referee will see the challenge is being made, and arrive to look at it (if the referee takes longer, then great, you have more time to cue the video). First, the referee decides if the video (as shown on the playback device) has enough information to show the situation at issue, then decides whether it changes the outcome, and finally, decides whether the evidence is sufficient that the video has not been tampered with or misrepresented.

If it fails the first test, the referee puts an end to it within moments. (For example, if there's not enough detail, or it doesn't show the right part of the field.) If it fails the second test, the team then knows it's not just the referee missing the call, but rather a deliberate decision to call it a particular way. (Rightly or wrongly, at that point it just becomes another futile challenge.) The third test may be a little more complicated—possibly requiring rewinding the video to hear the match being announced, or to verify that the right robots are on the field—but it's a straightforward test of credibility. (And because the burden of proof is on the team, and the timeline is tight, there's really no concern about the wrong video being reviewed, at least in the qualification round. In the elimination round, with the same teams playing, identification is more difficult, but at least the schedule isn't usually as constrained there.)

Why this? Because it strictly limits the number of reviews to minimize the aggregate delay, but still gives teams the opportunity to be heard. (Teams like it better when the officials hear them out, instead of ruling summarily.) It provides the referees with a procedural reason to change a bad call, without appearing indecisive, and gives the head referee a tool to better manage the rest of the referee crew (by understanding which calls are being blown). It also serves the teams notice that they share in the responsibility—if interested in challenging the officials' version of events, plan ahead to ensure that you're making video.

It's not perfect, but it doesn't have to be. I think you'd even see teams collaborating to supply post-match replay video to each other.

This is a really good suggestion/setup. I think you should add one more thing though (this has to do with the way the ref calls it though). Do it NFL style. There must be enough video evidence to clearly and without any doubt contest the call on the field. If there isn't, the ref shouldn't bother reviewing the footage. I realize this relies on a lot of interpretation, but still.

A great example of this was at Hatboto-Horsham. A friend of mine took a photo of a robot that made a 10 point hang, but appeared to have a ziptie sticking out of the bottom contacting the floor. At first glance, it seemed to be assisting the climb. However, when we took a few harder looks at the photo, and took a look at the robot (to satisfy our own curiosity), it looked like the ziptie was hanging about an inch above the ground and not actually touching the carpet. It's still hard to tell looking at the photo. We also casually asked a mentor in the pits about that ziptie, who stated "Oh that? I think someone forgot to cut it". This is something that could bog down the system, and is one of the reasons why the NFL has such a strict policy for their instant replay.

I know the NFL has a huge budget for the instant replay system. But, for FIRST, we don't need that. The burden is on the team/s challenging to have clear and decisive footage of their robot and the match. If they don't...well tough luck. This may also increase the number of teams recording/photographing matches at higher resolutions and better angles. A win all around. The only constraint: the whole challenge cycle needs to fit within the ~6 minute field reset time, and not take up all of the head refs time. If that can be satisfied, it would be a viable solution.

[EricH]

The problems I see with team recording are these:

1) The cameras will by nature be focused on one robot. This is not necessarily a bad thing, mind you--but tracking an object at zoom from 40+ feet away isn't easy if it's moving fast. This is where I'd prefer having a trained operator.

2) Time to get the film down to the field after the match. At an offseason, 1 minute is reasonable, if a bit of a stretch when cue time is factored in. At any event IRI or official, add 1-2 minutes for film to arrive. Or have a "Team Filming Area" (which would also address my third concern, below). Review time to determine if you caught something, 1 minute goes out the door.

3) Obstructions and bouncy cameras. Somebody pops up to cheer...right as a challengeable call happens. Maybe it's the camera person. Either way, lousy video quality. A team filming area would limit that.


Brain wave to deal with a few of these things...
What if every team was asked to have someone volunteer to record their robot every time it played? I'm sure there would be no shortage of volunteers for that one. However, each official replay volunteer would get a short training in "what to look for" (staying on the robot, appropriate zoom levels, etc), maybe 20-30 minutes long--this also gives a chance to check equipment for quality settings. Then, six to twelve spots around the arena are designated as "film spots" where the replay volunteers can set up cameras. Start when the MC begins the countdown, stop when the head ref gives the all-clear, head for the field in case of replay challenge.

For timing, it should vary based on venue layout, but say a team has 1 minute to initiate a video challenge, and then has 1-2 minutes to get the video to the head ref, cued up and ready (variable based on whether we're dealing with a high school gym or the Edward Jones Dome). Head ref watches the segment, reaches decision, and it's announced after the next match.


Edit: There is one other thing that I would suggest. If an event were to use instant replay, the head referee should have the ability to call for a video review, whether or not a team requests it. If the head referee calls without the team requesting a video review (say, the team asks about interpretation and the head ref decides that he wants to take a quick look again, or the ref huddle can't agree on how bad a foul is), no penalty to teams' ability to call for a video review later.