Video Review Needs to Happen Now

[Tristan Lall]

I think it's worth discussing exactly what a successful replay challenge should be able to accomplish, in order to more clearly distinguish between good and bad policies.

It's probably inherently infeasible to unwind a match just because the participants' reactions to a bad call were different than they would have been had the correct call been made. That's worse than judging intent: it's almost complete subjectivity, and trying to speculate about what constitutes a correct outcome is an exercise in conjecture. Instead of expecting to unwind the match, teams should play on, with the understanding that the call on the field might be reversed on appeal. It's up to the teams to choose how they complete the match based on incomplete information, but the fact that in hindsight they should have chosen differently is of no consequence.

By contrast, a replay challenge might be warranted in situations where the nature of a discrete event is unclear, and where that event is supposed to have a defined result when determining the outcome of the match. For example, did the robot complete a game task for which points are supposed to be awarded? The resolution is to either award points or not, and that takes effect at the end of a match, no matter what else happened.

But what about games where score depends on intra-match conditions? Those fall somewhere in between, and probably need to be considered as part of the game design process. One possible resolution is to credit the points mistakenly not awarded, but not unwind the gameplay that resulted from those points not being scored at their proper time. (Is that equitable? I guess it depends on the game.) Another resolution might be to trigger a rematch for certain missed calls: perhaps they're so fundamental to the game that to miss one is to threaten the legitimacy of the event. (In fact, that's the sort of situation where a strong replay rule is beneficial: most of the audience already knows the equitable outcome, so why not give the referees the opportunity to get with the program?)

So ultimately, having a replay system doesn't need to mean that everyone will always get rematches and slow the event to a crawl. Instead, it should be tailored to the situations for which it is usually advantageous (and then applied consistently), and should be made unavailable when it would most often be detrimental.

[Kevin Sevcik]

Originally Posted by Tristan Lall

By contrast, a replay challenge might be warranted in situations where the nature of a discrete event is unclear, and where that event is supposed to have a defined result when determining the outcome of the match. For example, did the robot complete a game task for which points are supposed to be awarded? The resolution is to either award points or not, and that takes effect at the end of a match, no matter what else happened.

By way of example here, you have 2006, where the alliance with the highest autonomous score goes on offense first, which was considered a significant advantage. Or 2004, where knocking a ball off in auton got you your balls 40 seconds sooner.

[EricH]

Originally Posted by Kevin Sevcik

By way of example here, you have 2006, where the alliance with the highest autonomous score goes on offense first, which was considered a significant advantage. Or 2004, where knocking a ball off in auton got you your balls 40 seconds sooner.

Don't remind me about 2006. Had a replay in a finals match because some field resetter didn't put the starting balls into the bins like they were supposed to, and a 10-ball auto dump got stuck on the sensor. Sensor read 15 balls (for 15 points--should have been 10). Opposing alliance drained 4 high goals for 12 points. Entire match played backwards...


I'd like to thank CVR for putting part of the problem into words... anybody in SoCal, go act on that post, please...

[gblake]

Folks - I think I'm right if I say that one purpose of using video is replacing fallible judgements with measurable facts. Be careful not to introduce too much heat-of-the-moment judgement back into the process you purpose/test.

[Ryan Dognaux]

Originally Posted by gblake

Folks - I think I'm right if I say that one purpose of using video is replacing fallible judgements with measurable facts. Be careful not to introduce too much heat-of-the-moment judgement back into the process you purpose/test.

The purpose of video review should be to allow referees to do their jobs better - that's it. It's not there to let teams endlessly draw out a tournament. It's not there to bring up incidents that didn't really happen. I don't want to introduce opinions, I want to introduce facts. Video doesn't lie and that's a great thing.

I think everyone can agree that our head ref's have a tough job. I'd like to try and help them out by giving them an option to review the previous match. I get that it's different and hasn't been done much before (cool to read about off-seasons that have done it.) Let's try it out in the off-season. Worst that happens is we find that it doesn't support the current tournament structure and we don't pursue it any further. Best that happens is we change the FRC event structure a bit in the future with defined guidelines set by the GDC & FIRST to improve event quality overall across the program. It's all upside right now. Let's give it a go.

I'll take Blake up on that beverage bet BTW I have high hopes and confidence in the system we use here in St. Louis. I'd love to try and replicate it. There are plenty of people who can tell you something won't work, it's up to us to prove them that it could work - that it will work. It has to be bullet proof and simple... and I think what we have checks both of those boxes. Really excited for October

[grstex]

Originally Posted by Ryan Dognaux

The purpose of video review should be to allow referees to do their jobs better - that's it. It's not there to let teams endlessly draw out a tournament. It's not there to bring up incidents that didn't really happen. I don't want to introduce opinions, I want to introduce facts. Video doesn't lie and that's a great thing.<Emphasis added>

This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story. Even the NFL can't find a definitive angle for every reviewed play. I don't say this to discredit the idea video review, but to remind those pushing for it that everything has its limits.

I think you and many others on this thread are well aware that video review won't resolve EVERY questionable call. Part of testing at offseason events will be determining what makes for REASONABLE review. You may otherwise find yourself down a very deep rabbit hole of "if only we had more camera angles," "if only we had higher def," "if only we had higher frame rates," "if only we allowed this/that/the other," etc.

I'm skeptical of video review, but I support those willing to invest the time/money to experiment with it. Just don't get led astray. The quest for "informative" video is a noble one. The quest for "infallible" video is quixotic.

[EricH]

Originally Posted by grstex

This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story.

Heh. Reminds me of the time some friends and I were doing a video scavenger hunt--no editing allowed--and one of the items was making a half-court shot. We went for a full-court shot, and made it (on the video).

What happened, though, was that the video player was set to just play the list in order, and some quick start/stop work resulted in two videos looking close to one. Oh, and the observant folks in the audience might have noticed the leg near the rim, behind the backboard... (A second team member was up behind the backboard with a second basketball.)

[gblake]

Just to keep the robotics replays emphasis on things. The references to edited video *are* interesting, but ... remember that even unedited recordings of a robotics match can deceive an untrained eye.

Identifying, and compensating when possible, for the unavoidable distortions 2D video introduces into recording/recreating an evolving 3D (4D) scene is one real obstacle in the way of video replays.

Users would need some training to help them avoid misinterpreting what they see in the playback. That training would help keep replays in the helping-not-hindering category.

Blake

[Tyler Olds]

I posted in a separate thread, but if you missed it: This is what we had to say about video review on Recap

[Ryan Dognaux]

Originally Posted by grstex

This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story. Even the NFL can't find a definitive angle for every reviewed play. I don't say this to discredit the idea video review, but to remind those pushing for it that everything has its limits.

Yeesh - nitpicking a bit here. The system we have would make it impossible to 'cut, crop, shoot a deceptive angle' - you get what you get with it. There's no time to edit the video because you're immediately pulling it up after it was recorded. If there's not enough evidence to overturn it, the call stands. Simple.

A lot of people thinking it's not even worth trying because we can't get 100 angles and 4K resolution here. The idea is to provide the referees one more tool to help them out. It won't be perfect but it will be better than nothing, I can promise you that.

[Lil' Lavery]

Originally Posted by Ryan Dognaux

It won't be perfect but it will be better than nothing, I can promise you that.

This is where contention is stemming from. The adverse effects may well be worse than not having a system (or investing those same resources* in alternative means of improving calls). The most debated call here is crossings. A fixed angle, full-field shot is pretty terrible at judging whether or not a team completed a crossing successfully. Those angles do a poor job at determining if a wheel (or other component) is still in contact with the outer works, if a team fully stopped touching a sallyport door, or seeing into the many blind spots around the defenses this year.

Moreover, you can't simply making scoring adjustments. When a team spends 30+ seconds of a match making sure they get a crossing that counts, that's all sorts of score impacts beyond the simple points. When a ball isn't registered properly on the tower, that has tons of impact for both alliances regarding capture strategy. Almost any circumstance where video replay is warranted also warrants a replay of a match. Even if you limit challenges to the eliminations, and only 50% of alliances use theirs, that's still 4 extra matches in the schedule. With a 7 minute cycle, at best that's 28 additional minutes added to the schedule (and likely more once you factor in 6 minute field timeouts for teams in consecutive matches).

Team experience is an important factor, but an event running on time is a part of team experience. Anyone who's been in FIRST for more than a few years has witnessed a team being called up for an award, only to find out they already left. One of the most constant complaints regarding CMP and Einstein is how late they run. On the rare occasions that events run ahead of time, you'll find plenty of posts on CD congratulating the volunteer crew for an well executed event.

*For example, one poster here is adamant about adding another key volunteer to this process. What if instead of that volunteer being a replay analyst, they were actually just another ref on the field, helping get the calls right the first time?

[alephzer0]

I just wanna point out that this argument has been taking place since at least 2005.

Don't believe me? Check out this thread:
http://www.chiefdelphi.com/forums/sh...737#post415737
If you ask me, this is sort of a

[Ryan Dognaux]

Originally Posted by alephzer0

I just wanna point out that this argument has been taking place since at least 2005.

Don't believe me? Check out this thread:
http://www.chiefdelphi.com/forums/sh...737#post415737
If you ask me, this is sort of a

Technology has come a long way in 11 years. Back in 2005 I would have completely dismissed this notion. Today it's very achievable for a fraction of the cost.

There's a reason you don't see many videos from that time and if you do it looks like it was recorded using a potato.

If we're serious and FIRST is serious about delivering a product that excites young people, we need to get serious about high quality video being a staple at all of our events.

[alephzer0]

Originally Posted by Ryan Dognaux

Technology has come a long way in 11 years. Back in 2005 I would have completely dismissed this notion. Today it's very achievable for a fraction of the cost.

There's a reason you don't see many videos from that time and if you do it looks like it was recorded using a potato.

True, but I was just pointing out that this isn't a new argument.

[Ryan Dognaux]

Originally Posted by alephzer0

True, but I was just pointing out that this isn't a new argument.

Agree - but it's not beating a dead horse. We should revisit topics like this from time to time as things become cheaper, easier and more obtainable.