Video Review Needs to Happen Now

[gblake]

Doh! Doh! and Double-Doh!

This is the link I forgot to include earlier today.

The missing link

Blake

[Gregor]

Video review made it's debut in Olympics Beach Volleyball this year with specific rules on what may be reviewed, such as line violations and in/out calls, but not judgement calls such as net violations and lifts, and additionally there's only a set amount of time after a play (5 seconds) in which they're allowed to request a challenge.

I think this is a good example of how to implement video review in the context of FRC. Not everything should be allowed to be challenged, only certain black or white calls, such as crossing a zone in 2014. A time limit on requesting a challenge should be implemented.

[EricH]

Originally Posted by Gregor

I think this is a good example of how to implement video review in the context of FRC. Not everything should be allowed to be challenged, only certain black or white calls, such as crossing a zone in 2014. A time limit on requesting a challenge should be implemented.

And that's a reasonable implementation, too. The game changes year-to-year, but there's almost always some hard-and-fast "do this like this" area that can/should be reviewed.

If I were to implement replay, judgement calls would be out of the question for team-implemented review. (I'd give the head refs the option to use it for judgment calls, if they thought it necessary, and I'd guess that most head refs wouldn't. The bigger the judgement call, the more refs are in the huddle.) To review a call, it'd have to be something that could be seen and quantified from the video, and isn't covered by any automated systems or placement at the end of the match. (End-of-match placement, you can get 2-3 opinions in a matter of seconds.) Sounds like auto line violations, secret passage calls, and defense crossings for 2016; zone entry in 2014; contact around the protected zones in 2012 and 2013.

[gblake]

Folks,

Now that the summer of 2016 is almost over, does anyone want to report any new (previously unreported) results of experimenting with video replay ("hard" data or anecdotes) during off-season tournaments/scrimmages?

What about any non-tournament / non-scrimmage experiments to determine numbers of cameras needed for good results (for Stronghold), or best camera locations (for Stronghold), or minimum useful frame rates (for fast robots and game pieces), or estimated time needed to accurately review each/any protested ruling (for Stronghold (for each rule and/or scoring event)) or ...

Blake

[Hitchhiker 42]

Again, as a theoretical - no hard data here - but there are easier ways than cameras. Requiring each robot to carry a small sensor packet that can tell where the robot is at any point in time would have fixed the crossings issue this year. Maybe in the future, it could even be built into the RIO.

[EricH]

Originally Posted by Hitchhiker 42

Again, as a theoretical - no hard data here - but there are easier ways than cameras. Requiring each robot to carry a small sensor packet that can tell where the robot is at any point in time would have fixed the crossings issue this year. Maybe in the future, it could even be built into the RIO.

Actually, it wouldn't have totally fixed that issue. You'd have to completely redefine Crossing for that to work automatically--Crossing is determined by bumpers, and making a foolproof system that could tell that the bumpers were clear from a sensor packet (from whom? team/FIRST/other?) that would need to be inside the bumpers would require a lot of complexity. Need to know size + shape + location + orientation of the robot with bumpers, the last two in three dimensions each. It'd be a lot easier for Scales--mount a device or two at the bottom of your bumpers, if they're all above a sensor mounted in the castle wall at the end of a match you're good. (Reasons to deny a Crossing included starting in contact with a defense--see also bad Sally Port crossing--as well as bumpers not finishing clearing and bumpers going over/above the shields.)

Now, they did do something like that in '08, where each robot carried a small IR beacon (or something like that--it was field-supplied with another piece of required equipment) mounted in a known area of the robot--pass under a receiver, get credit. But I don't recall how well that worked.

Can it be done, yes. But to be honest, the second-best tool we have right now is actually cameras, properly pointed, if the rules allow it. The best tool? The 6 pairs of eyes in the heads of the referees. Without those, there is not a good way to tell if the electronics are acting up and not counting stuff properly. And, of course, we sure hope the game is designed so it's a little easier on the refs...

[ATannahill]

Originally Posted by EricH

Now, they did do something like that in '08, where each robot carried a small IR beacon (or something like that--it was field-supplied with another piece of required equipment) mounted in a known area of the robot--pass under a receiver, get credit. But I don't recall how well that worked.

As I remember, poorly. I do not believe they were used at all events.

[Chris is me]

Originally Posted by gblake

Folks,

Now that the summer of 2016 is almost over, does anyone want to report any new (previously unreported) results of experimenting with video replay ("hard" data or anecdotes) during off-season tournaments/scrimmages?

What about any non-tournament / non-scrimmage experiments to determine numbers of cameras needed for good results (for Stronghold), or best camera locations (for Stronghold), or minimum useful frame rates (for fast robots and game pieces), or estimated time needed to accurately review each/any protested ruling (for Stronghold (for each rule and/or scoring event)) or ...

Blake

I would ask again in the fall. Most offseasons happen in the spring or fall; relatively few in the summer.

While I think nobody can expect perfect camera coverage of the field, even a single full field view results in some things being reviewable. If someone tries to get a review and there isn't clear video, the outcome doesn't change and the situation is no worse than having no video review at all.

I feel like a "it has to be perfect or it isn't worth doing at all" attitude here will really hold people back from experimenting with this.

[XaulZan11]

Originally Posted by Chris is me

. If someone tries to get a review and there isn't clear video, the outcome doesn't change and the situation is no worse than having no video review at all.
.

Other than the additional time wasted.

[Hitchhiker 42]

Originally Posted by Chris is me

I feel like a "it has to be perfect or it isn't worth doing at all" attitude here will really hold people back from experimenting with this.

Nothing's holding off-seasons from experimenting with this. The real problem lies with real competitions, where we should be wary in implementing something that can make them even longer without having a clear and proven positive effect (and a streamlined process for doing so).

[gblake]

Originally Posted by Chris is me

I would ask again in the fall. Most offseasons happen in the spring or fall; relatively few in the summer.

Sure - I'm aware, I thought a few were going to do some trials this summer, so I asked. I or someone should give this thread another poke in late Fall.

Originally Posted by Chris is me

I feel like a "it has to be perfect or it isn't worth doing at all" attitude here will really hold people back from experimenting with this.

I certainly hope not.

Conducting experiments is how you quantify "better, but not perfect". When a discussion reaches that stage, it can leave endless cycles of hand-waving behind.

Imperfect is fine, if the imperfections are known, and well-understood; and if the decision-makers agree that the imperfect system is the one they want (because it's better (in some hopefully well-defined sense) than the system it replaces).

Imperfect, and poorly understood, when/if it were implemented, ... Well, that would be a different kettle of fish.

Well designed experiments are exactly what this topic needs.

Blake

[Sperkowsky]

Originally Posted by XaulZan11

Other than the additional time wasted.

During the time the video review is happening. Pre match things can be going on. Ie connecting robots, and reseting the field. I can't see very much time wasted at all unless the system to replay matches is not working at all.

Besides I would rather spend 30 minutes longer at ever Frc event then see anyone leave an event how MY team did this year. (go look at the start of the thread my team was on that alliance). If there was video review we would have atleast been semi finalists. But, being our alliance set a higher score then that semi finals alliance there was a good chance if video review existed we could have been finalists. I have thought through every scenario I could have done to make the outcome different. Design choices, driving choices, repair choices, ect. But the bottom line is our season ended because of a bad ref call. I don't fault the refs. This game was ridiculously hard to referee. But, the fact of the situation is still there.

Video review would help the teams, and help the refs. It doesn't cost that much to implement and maybe it can be integrated into an frc dedicated livestream setup. Which would be a huge win.

[gblake]

Originally Posted by Sperkowsky

...
Video review would help the teams, and help the refs. It doesn't cost that much to implement and maybe it can be integrated into an frc dedicated livestream setup. Which would be a huge win.

In what well-designed experiments has this been qauntified?

Originally Posted by gblake

... leave endless cycles of hand-waving behind.
...
Well designed experiments are exactly what this topic needs.

Without proof that the net effect (the entire gestalt) will be better (in some well-defined sense) outcomes, both you and I are just announcing our opinions while figuratively waving our hands in the air.

Blake

[Hitchhiker 42]

I think that one of the biggest things we always need to keep in mind is the goal of FIRST and FRC. What are we really trying to achieve? Inspiration.

The next question we have to ask is "Is it really worth it to implement video review systems, or will the goal of inspiration* be just as attainable even with the few (and far between) errors of the ref?"

*Yes I understand that having teams' hard work recognized plays a part in this. That's why this is a question, not a statement.

[Tyler Olds]

Originally Posted by Hitchhiker 42

"Is it really worth it to implement video review systems, or will the goal of inspiration* be just as attainable even with the few (and far between) errors of the ref?"

The term inspiration is very subjective and to me having a higher percentage of calls correct is much more inspiring.