Video Review Needs to Happen Now

[CVR]

I think the root cause of this issue is that people think there are "bad" referees that miss calls. That may be true, but fixing a bad system by introducing a new system isn't a great fix. Instead people should focus on a direct fix.

If everyone who's posted in this thread volunteered to referee, head refs would have their pick of referees. They could staff events with "good" refs instead of "bad" refs. The problem is that events can't really be selective in who they get to ref it. Some events may only have 5 applicants for the 5 ref positions. This means if one ref is known to be prone to missing calls, the event has no choice, they have to play them. If events had 10 refs volunteer for 5 positions, they would be able to select their "favorite" 5 (or 7).

So if you really want better calls, the best way to make is happen is to volunteer as a referee. Anyone who is complaining should get on VIMS right now, sign up for a week 4, 5, or 6 event and start the ref training immediately. Although it can't do anything for the event you're complaining about, you have the chance to make someone else's season much better. And then volunteer next year! You may take the spot of that really great ref, and now that they don't have to do a week 2, 3, 4, and 5 event, maybe they are willing to cover your week 1 event!

If events are able to have their pick of referees and can choose the most capable refs, and there are still problems, then I think that might be a good time to talk about video review. But getting more volunteer refs is a simpler, cheaper, more direct, and overall better approach to solving the issue of bad calls. A video review system is going to need more volunteers at the same training level as refs anyway, so why not just have more refs?

[gblake]

Originally Posted by Tristan Lall

I think it's worth discussing exactly what a successful replay challenge should be able to accomplish, in order to more clearly distinguish between good and policies.
...
So ultimately, having a replay system doesn't need to mean that everyone will always get rematches and slow the event to a crawl. Instead, it should be tailored to the situations for which it is usually advantageous (and then applied consistently), and should be made unavailable when it would most often be detrimental.

Tristan, I know you set out to discuss "exactly" what a successful replay challenge ... But, what you actually wrote is some high-level thoughts about how a system might work, in general.

To be successful, I think a replay system would have to describe (at the least) exactly what action the ref should take, for every single gameplay and scoring rule, whenever a challenge reverses a call involving that rule. The time when the original call occurred, and the match score when the call occurred might or might not need to be factored in for some situations.

I think there is a fairly large gap between where you ended, and what must be included in a mature, robust, system that produces consistent/predictable results.

I'm not pretending that I know the answer for this season, or for any other. It will be almost certainly be darned hard to produce a complete set of replay instructions each year. Attempting to create those instructions is an exercise left up to the readers (those readers who think reviewing video is a good idea).

IMO, the off-season experiments folks are planning for this year won't be successful unless/until they produce instructions corresponding to reach rule of this season's game. The alternative of asking the refs to fly by the seat of their pants during a tournament would not be a good idea.

Blake

[Ryan Dognaux]

Originally Posted by gblake

IMO, the off-season experiments folks are planning for this year won't be successful unless/until they produce instructions corresponding to reach rule of this season's game. The alternative of asking the refs to fly by the seat of their pants during a tournament would not be a good idea.

For our testing this year we will just say if the head referee accepts the challenge request, a match replay will be issued. Keeps it really simple regardless of the play in question. May not be what would actually be implemented eventually, especially in certain cases like missing a game piece being scored, but it's what we'll try.

[gblake]

Originally Posted by Ryan Dognaux

For our testing this year we will just say if the head referee accepts the challenge request, a match replay will be issued. Keeps it really simple regardless of the play in question. May not be what would actually be implemented eventually, especially in certain cases like missing a game piece being scored, but it's what we'll try.

OK, let us know how it works out.

You probably already took into account that if I'm a team focused on winning, even if my alliance gets beaten by 100 to nothing in a match, I might decide to protest a minor call, just hoping to trigger a rematch.

Similarly, if I haven't used my protest(s) yet, I might automatically protest any/every loss that would take my alliance out of the eliminations, hoping to trigger a replay.

If I'm especially mischievous, I might purposefully create some sketchy situations, just so that I can increase my chances of having an opportunity to successfully protest a call.

If these scenarios are something you want to discourage, to avoid slow-downs, I'm not sure how you can do it.

Blake

[Ryan Dognaux]

Originally Posted by gblake

OK, let us know how it works out.

You probably already took into account that if I'm a team focused on winning, even if my alliance gets beaten by 100 to nothing in a match, I might decide to protest a minor call, just hoping to trigger a rematch.

Similarly, if I haven't used my protest(s) yet, I might automatically protest any/every loss that would take my alliance out of the eliminations, hoping to trigger a replay.

If I'm especially mischievous, I might purposefully create some sketchy situations, just so that I can increase my chances of having an opportunity to successfully protest a call.

If these scenarios are something you want to discourage, to avoid slow-downs, I'm not sure how you can do it.

Blake

Just make it so the head referee has the right to take away the alliance's challenge for erroneous & repeated protests. The question box exists today and while a few teams may abuse it from time to time, I don't see all teams doing it.

[bdaroz]

Originally Posted by Ryan Dognaux

Just make it so the head referee has the right to take away the alliance's challenge for erroneous & repeated protests. The question box exists today and while a few teams may abuse it from time to time, I don't see all teams doing it.

I'm not sure having every successful challenge result in a rematch is right either. If the match was a 30-100 blowout, and the challenge would net a maximum of 10 points to the 30pt team, there's no reason to replay... sort of.

Put in more "rule" type language:

After a successful challenge, the Head Referee shall take into account the match in its entirely. If, after considering the effect of a correct call on the match, the Head Referee shall determine if the correction, and likely actions of the Teams resulting thereof, could have reasonably changed the outcome of the match, the match shall be replayed. If the match is not to be replayed, the Head Referee shall adjust the score appropriately.

The wording isn't perfect, but I think you get the idea. I don't think it's enough that a score change alone would alter the winner/loser of the match, but I think the play of the match needs to be considered to trigger a reply. Eg. that 30-100 blowout, if a Robot had to attempt a 3rd crossing because the 2nd was missed by a ref and the Robot turtled as a result, there's a reasonable cause to replay the match. If it didn't turtle, but just took some time away from scoring a high-shot, no I wouldn't replay it.

[smitikshah]

Having been personally affected as drive coach from team 2869, I was hesitant to post as I didn't want my emotion getting in the way. However, I do like the proposed solution by many to try the video system out at any offseason comps. If anyone is planning on using this system at any events in the greater NYC region please let me know!

I would love to help out and let this emotion guide me in a positive direction in helping other teams!

[EricH]

Originally Posted by Kevin Sevcik

By way of example here, you have 2006, where the alliance with the highest autonomous score goes on offense first, which was considered a significant advantage. Or 2004, where knocking a ball off in auton got you your balls 40 seconds sooner.

Don't remind me about 2006. Had a replay in a finals match because some field resetter didn't put the starting balls into the bins like they were supposed to, and a 10-ball auto dump got stuck on the sensor. Sensor read 15 balls (for 15 points--should have been 10). Opposing alliance drained 4 high goals for 12 points. Entire match played backwards...


I'd like to thank CVR for putting part of the problem into words... anybody in SoCal, go act on that post, please...

[gblake]

Folks - I think I'm right if I say that one purpose of using video is replacing fallible judgements with measurable facts. Be careful not to introduce too much heat-of-the-moment judgement back into the process you purpose/test.

[Ryan Dognaux]

Originally Posted by gblake

Folks - I think I'm right if I say that one purpose of using video is replacing fallible judgements with measurable facts. Be careful not to introduce too much heat-of-the-moment judgement back into the process you purpose/test.

The purpose of video review should be to allow referees to do their jobs better - that's it. It's not there to let teams endlessly draw out a tournament. It's not there to bring up incidents that didn't really happen. I don't want to introduce opinions, I want to introduce facts. Video doesn't lie and that's a great thing.

I think everyone can agree that our head ref's have a tough job. I'd like to try and help them out by giving them an option to review the previous match. I get that it's different and hasn't been done much before (cool to read about off-seasons that have done it.) Let's try it out in the off-season. Worst that happens is we find that it doesn't support the current tournament structure and we don't pursue it any further. Best that happens is we change the FRC event structure a bit in the future with defined guidelines set by the GDC & FIRST to improve event quality overall across the program. It's all upside right now. Let's give it a go.

I'll take Blake up on that beverage bet BTW I have high hopes and confidence in the system we use here in St. Louis. I'd love to try and replicate it. There are plenty of people who can tell you something won't work, it's up to us to prove them that it could work - that it will work. It has to be bullet proof and simple... and I think what we have checks both of those boxes. Really excited for October

[grstex]

Originally Posted by Ryan Dognaux

The purpose of video review should be to allow referees to do their jobs better - that's it. It's not there to let teams endlessly draw out a tournament. It's not there to bring up incidents that didn't really happen. I don't want to introduce opinions, I want to introduce facts. Video doesn't lie and that's a great thing.<Emphasis added>

This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story. Even the NFL can't find a definitive angle for every reviewed play. I don't say this to discredit the idea video review, but to remind those pushing for it that everything has its limits.

I think you and many others on this thread are well aware that video review won't resolve EVERY questionable call. Part of testing at offseason events will be determining what makes for REASONABLE review. You may otherwise find yourself down a very deep rabbit hole of "if only we had more camera angles," "if only we had higher def," "if only we had higher frame rates," "if only we allowed this/that/the other," etc.

I'm skeptical of video review, but I support those willing to invest the time/money to experiment with it. Just don't get led astray. The quest for "informative" video is a noble one. The quest for "infallible" video is quixotic.

[EricH]

Originally Posted by grstex

This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story.

Heh. Reminds me of the time some friends and I were doing a video scavenger hunt--no editing allowed--and one of the items was making a half-court shot. We went for a full-court shot, and made it (on the video).

What happened, though, was that the video player was set to just play the list in order, and some quick start/stop work resulted in two videos looking close to one. Oh, and the observant folks in the audience might have noticed the leg near the rim, behind the backboard... (A second team member was up behind the backboard with a second basketball.)

[gblake]

Just to keep the robotics replays emphasis on things. The references to edited video *are* interesting, but ... remember that even unedited recordings of a robotics match can deceive an untrained eye.

Identifying, and compensating when possible, for the unavoidable distortions 2D video introduces into recording/recreating an evolving 3D (4D) scene is one real obstacle in the way of video replays.

Users would need some training to help them avoid misinterpreting what they see in the playback. That training would help keep replays in the helping-not-hindering category.

Blake

[Tyler Olds]

I posted in a separate thread, but if you missed it: This is what we had to say about video review on Recap

[Ryan Dognaux]

Originally Posted by grstex

This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story. Even the NFL can't find a definitive angle for every reviewed play. I don't say this to discredit the idea video review, but to remind those pushing for it that everything has its limits.

Yeesh - nitpicking a bit here. The system we have would make it impossible to 'cut, crop, shoot a deceptive angle' - you get what you get with it. There's no time to edit the video because you're immediately pulling it up after it was recorded. If there's not enough evidence to overturn it, the call stands. Simple.

A lot of people thinking it's not even worth trying because we can't get 100 angles and 4K resolution here. The idea is to provide the referees one more tool to help them out. It won't be perfect but it will be better than nothing, I can promise you that.