Movement Towards Video Review
 

After recent events at the World Championship, I believe the push for video reviews in FIRST. Many people don't see the need for them and I myself was originally skeptical about them. My view changed at the Championship this week when we were pinned for an extended period of time and the ref didn't call it.

https://youtu.be/y56cRWjvW-c?t=54s

Now on a separate note on what happened after the pin, being the driver I apologize to 48 about the flip. 100% didn't mean to flip and was just trying to get separation by pushing your robot away before I broke hard back to the secret passage. I hope no damage was done and that there are no hard feelings. The flip was unintentional but it happened so I will apologize for it.

Video review is something that I'd like to see happen in the near future to eliminate moments like these. Penalties as we saw this year have huge impacts on the outcomes of events. Everyone knows that the refs are humans and volunteers and make mistakes but it's unfortunate those mistakes can negate a team's hours and hard work put into the season. Video reviews would help make FIRST competitions run smoother and more fairly.

The point of this thread is not to complain about bad refs, but a thread to discuss ways to set video reviews into action.

What are the pros and cons everyone sees in making a movement like this?

What are some solutions?

Is this something that ChampsLive can help with?

-Sean

Re: Movement Towards Video Review
 

You are presenting this as though it is indisputable that you were pinned. However, that's not so indisputable.

G22 reads:
And yes, the blue box reads, in part:
But one could argue you had a way out. It was through the opponent's secret passage. And, it is a legal route for you to take at the moment the pin occurs.

Further, your opponent retreats and you chase the robot, hence ending the pin. True, it wasn't for a 6 foot distance, but the opponent backed away, and you pursued. At that point you were not against an object and hence not pinned.

You may not agree with the referee's decision, but that referee is looking right at the interaction and determined there was no pin. Perhaps what I said above is what was going through that referee's mind.

And, seeing that your dispute involves a referee's judgement, what would be your criteria for the resolution? If the ref isn't counting, then the team doesn't think they are pinning, and they don't move. Would it be equally unfair to that opponent team who believed they were within the rules to be suddenly told they have a penalty, or worse a card, when they had no opportunity to correct their behavior?

Re: Movement Towards Video Review
 

As a driver coach, that match is insanely frustrating to watch. 48 was definitely taking advantage of the situation. It was either a 7-second pin or a 19-second pin. 7 seconds if the ref thought you could go through the secret passage to get out, or 19 seconds if the ref considered the secret passage as part of the pin. Or perhaps the ref thought you could drive over the invisible shield on the batter.

It really just looks like that ref wasn't cognisant in that match. 120 matches in, and towards the end of a nearly 12-hour day, that ref was most definitely fatigued. This year the game didn't get easier to ref just because the season progressed further. I don't know that live reviewers would have changed anything in that match, because they may have been just as tired by that point.

Re: Movement Towards Video Review
 

I am not sure video replay would helped in this situation. I agree that you were pinned since you cannot go into the secret passage without high potential for a penalty. But nothing could really be done about that after the match. You would have still received a red card for flipping using what seems to be the criteria for that this year. You would no longer have been considered "pinned" the point you flipped him.

Re: Movement Towards Video Review
 

I highly suggest 16 sign up for Ryan Dognaux's offseason event in St. Louis. They will be implementing video review, from what I've heard. It should be a nice testing platform for the future.

Feel free to contact him to register!
http://www.chiefdelphi.com/forums/member.php?u=1812

Re: Movement Towards Video Review
 

*in elimination matches only, one video review per alliance :)

Come on back to St. Louis in October - Where we're cool and like to try stuff to make FIRST better.™

16 knows all about the GRC already though, they were at our inaugural event and they're awesome!

Re: Movement Towards Video Review
 

There were two instances of where 16 looked to be "trapped" for over 5 seconds without chasing 48. There was also over a 15 second period of time in which bomb squad could not escape through the secret passage either (noting that the game manual generally considers this an egregious pin that would've resulted in a red card for the blue alliance). Also, the referees were not even counting the pin when it was solid contact between both robots and 16 on the alliance wall.

Even though the secret passage was an option, it was very likely that 48 would have continued the pin onto Bomb Squad, also drawing a tech foul onto the red alliance.

The argument that the Bomb Squad's driver motions is that a video review would reduce the chance of any uncertainty and make these calls indisputable.

Re: Movement Towards Video Review
 

I wouldn't focus on our match in particular. Like I said this is more for finding solutions to problems instead of looking at things that are already over. I posted our video as more of an example than something to critique. I understand that the red card would mean that the pin is useless to us but the points that could come from the pin would be huge to the other teams on the alliance that the red card didn't affect. But like I said this is not about this one match, but missed or bad calls in general that some sort of review system can fix and I'd like to see everyone's ideas about how this can be done like the off season trials that were mentioned.

Re: Movement Towards Video Review
 

In what sport do they allow video review for a missed call? Let alone a missed penalty call.

Re: Movement Towards Video Review
 

Basketball (NBA) has video replays... Especially for flagrant calls, who touched the ball last, whether the shot went off before the shot clock/timer expired etc...

Re: Movement Towards Video Review
 

Baseball now allows reviewing of just about everything.

That being said, I don't agree with reviews for pins. The procedure for calling a pin specifically includes a 5-second warning of such. Without a referee signaling this, a driver has no way to know whether or not a referee thinks they're pinning someone.

Re: Movement Towards Video Review
 

But that still doesn't apply for 99% of fouls (common fouls, and the 99% isn't an exact statistic). Almost always, you can't retroactively go back and call fouls on a player in basketball, unless the foul is extremely egregious. If you're retroactively calling common fouls in basketball, then the basketball game is going to take all day.

Re: Movement Towards Video Review
 

I'm not saying to do that, I'm just saying there are video replays. FIRST would have to specify what they would use replays for.

Re: Movement Towards Video Review
 

So what can and cannot be reviewable?

In college football, it's things related to possession, side lines, goal line, and end line (including goal posts). That's it. You can't review holding, pass interference, the cheap shot on the line, or Ndamukong Suh stepping on your face.

Looking to FRC...

Well, on the definitely "can" be reviewed list would be scores that were missed. Things like a boulder being stuck in the chute, a defense cross being missed, bad frisbee count (2013), etc.

On the definitely "can't" be reviewed list are most fouls. Particularly any foul where you get a warning before the penalty, like pinning. To retroactively invoke fouls is unfair to the team that receives the penalty, as others have posted here.

To the OP's second post: You don't want us to focus on your particular situation yet you use it as your sole example for establishing video review. Video review wouldn't have helped your situation and would not have "eliminated moments like these".

I therefore fail to see a compelling argument to implement video review. QED.

Re: Movement Towards Video Review
 

Personally, I think implementing video review would cause more problems than it would solve. If a team is pinning, but the refs aren't calling it, then the drive team is under the impression that what they are doing is perfectly legal and has no reason to stop. Implementing video review and fouling that team after the match is unfair in my opinion. I understand that refs calls are not always 100% fair, but I don't think implementing video review is the way to solve that problem.

Re: Movement Towards Video Review
 

Any ideas on what could be used then if anything?

Re: Movement Towards Video Review
 

Another problem with Video Review is that it affects game play. In football, basketball, baseball, the review happens with the play has stopped.

For FRC, video review is after the game is over. What if a shot is reviewed, and it is determined that it was an illegal shot (robot did not fully cross into the courtyard before launching the boulder), and should not have counted? That affects Tower Strength, and the Alliance may have played differently if it knew that it needed one more boulder to defeat the tower.

Maybe video review for something like a Red Card where it affects the outcome of the match (alliance gets 0 points), but does not affect the game itself.

> Any ideas on what could be used then if anything?
I am of the opinion that mostly bad things will come of Video Replay, and very little good. You have Amateur Ref's refing the game, and you are trying to hold them to a professional ref standard. What can you do? Live with the fact that some calls will go against you, and some will go your way. Hopefully in the aggregate, it all evens out in the end.

BTW: I watched a lot of matches from the Scoring Table, and I can confirm there were a lot of bad calls (and non-calls). However, that was due to my vantage point (and what I was looking at), and the Ref was calling it best he could from his vantage point (and what he was looking at).

Re: Movement Towards Video Review
 

Unfortunately I don't have any at the moment. Usually I like to provide some sort of alternative when voicing a disagreement about a solution to an obvious problem. I'll give it some thought and post anything I come up with here. I agree that some sort of change should be made, but I'm not sure how to go about that change quite yet. I'll get back to you.

Re: Movement Towards Video Review
 

Maybe it doesnt fit exactly within the definition of a "call," but NASCAR has been using video replay for years (though replays should not take that long) and is used to determine placement of positions when cautions come out. This year alone they implemented a pit review system using cameras (which was not done beforehand) which drastically increased the amount of penalties that previously went unseen.

Re: Movement Towards Video Review
 

I would rather see FIRST train their head refs better so that matches are called more consistently. Video review doesn't do much when the refs don't do their jobs well in the first place.

However, it is not a bad idea for refs to review video before making the decision to hand out a red card (i.e. all the tipping fouls this year).

Re: Movement Towards Video Review
 

Video review is in almost all sports now, and I can almost guarantee it will be in FRC eventually (eventually is a long time). We are going to have some form of it at TRI this year. Mostly at the digression of the head ref and a review coupon in eliminations like you have a time out coupon in eliminations. Just like in the NFL their would need to be definitive evidence that a call should go the other way and a lot of judgement calls won't be able to be reviewed, we are still working out the exact details of the review rules.

The idea is to get as many calls right as we can just like it is used in other sports.

Re: Movement Towards Video Review
 

+1 Especially ones invited to the Championship. ;)

Or at least supply them with lots of caffeine.

Re: Movement Towards Video Review
 

I don't see Video Review for High School Level sports.

It is not just a Head Ref problem. It is all the other ref's too.

How much do you want your fees to increase so Ref's are paid, and not volunteers? Then you can complain about the quality of the Ref's.

That said, I do think there needs to be better consistency in some calls. First should clearly define the standard they want the Ref's to enforce. Not necessarily tell it to the world, but at least tell the Head Ref's, who then tell the other Ref's.

Re: Movement Towards Video Review
 

If you look at most sports common fouls are not usually reviewable so in the case of FRC something like a pin shouldn't be reviewable. What should be reviewable are things like assists in Aerial Assist, barrier crosses in Stronghold, and any penalties that would result in yellow or red cards. These types of things have score impacts and are very easy to review. The problem with reviewing something like pinning is that it is very subjective and would take too long. If FIRST were to implement a replay type system, it shouldn't be used in qualifications as it would take too long but instead they can give each alliance 3 challenges for the elimination rounds. At championship teams should be given 3 challenges in their divisor playoffs and then another 3 challenges for Einstein. I think this system would solve a lot of the missed calls in important elimination matches.

I also think that referees should not handle scoring and should only focus on calling fouls. FRC is the only sport were the ref has to look for fouls and keep track of the score at the same time, that is it too much for one person. They should have score keepers that handle scoring and refs who handle fouls.

Re: Movement Towards Video Review
 

Baseball umpires call runs at home plate.
Football refs call touchdowns, field goals, and safeties.

All sports, the Ref calls when a score has happened or not.

In FRC, the Ref's call some scores (crossings), but they don't keep track of the score (FMS does that).

IMHO, there are some things that can be handled by Field Stewards, like they did with Recycle Rush. I think there should have been Field Stewards doing boulder counting in the Castle, and letting Ref's know when they are at 6.

They had Ref's watching crossings, so having a Field Steward do it won't improve anything, other than freeing up a Ref to watch for fouls.

IMHO, FRC should have added one more ref, and done 1-on-1 ref'ing with one ref watching one robot for the entire match. Maybe do it during Practice Matches to see how it goes. The Ref won't miss a crossing (although still has a problem with sight lines for the Sally Port). Ref won't miss the start of a Pin (although the actual start can be subjective, but it should happen more promptly). The Ref for the robot that causes a foul enters it into the panel.

Re: Movement Towards Video Review
 

The lack thereof doesn't mean there shouldn't be.

Re: Movement Towards Video Review
 

Allen was trying to use Video Review of professional sports to support the position of video review at the High School level. Comparing apples to oranges.

Whether the apple should be sweet or tart, in and of itself or compared to other apples, is a valid issue.

Re: Movement Towards Video Review
 

FRC is the highest level of robotics for students. Comparing FRC to the Pros is actually apples to apples. The highest level of competition, no matter the sport, deserves the highest level of calls.

Re: Movement Towards Video Review
 

So far all of the examples I've seen require review with judgement.

I know of a match where the portcullis was crossed twice and neither time was registered. This wouldn't require judging--it's clearly evident; a fact.

The drive coach went to the 'question box' with the video fresh on a camera but they said "we can't do video review."

Please know that I feel really not-GP linking this, but to support this subject, I can't think of another way. I think video review can make things better.

But I also fully understand the time problem.

(removed match link)

Re: Movement Towards Video Review
 

So a referee is video reviewing a specific call, & he sees a different, unrelated penalty. Does he call it?

Re: Movement Towards Video Review
 

No. That's outside the scope of the review.

Re: Movement Towards Video Review
 

Really? The rules for this have all ready been written? Keeping in mind that this year rules don't apply to next years game. :]

So what if the referee decides the foul deserves a card?

I am accepting of either way. Just pointing out the complexities of video review.

Re: Movement Towards Video Review
 

I think 3 things should happen in regards to improving the accuracy of rule enforcement...

1. The GDC needs to develop rules that aren't based on subjective factors... things like intent and crossing boundaries that are hard to visualize. I think the GDC did well with the latter this year (boundaries pretty much all were based on contact with carpet, which is easier), but we all know some rules just require too much judgment of intent... looking at you, G24. I think G28 and G43 are pretty good examples of rules that take the judgment of intent out of the rule... I'd like to see a rule that describes what contact can make a flip (G24) result in a red card. Clearly if a robot tips themselves over, G24 is irrelevant, but some robots every year are tippy enough (at least when they have a tall elevator or arm up) that even ordinarily fair contact can unintentionally result in a tip. I've proposed using bumper-to-bumper contact as the deciding factor. If a robot can be tipped from only bumper-to-bumper contact, G24 shouldn't be relevant. I could see it reading like:

Strategies aimed at the destruction or inhibition of ROBOTS via attachment, damage, tipping or entanglements are not allowed. Destruction or inhibition is not considered strategic if it results exclusively from contact between ROBOT BUMPERS.

Violation: FOUL and YELLOW CARD. If harm or incapacitation occurs as a result of the strategy, RED CARD.

2. The Head, Head Referee (Aidan Brown) key FIRST HQ staff (i.e., Frank Merrick? GDC staff?) should be made aware of the tough calls and "controversies" at each event (and appropriate flowdown should occur afterward). If these aren't being consistently flowed up, there should be a process in place by which teams can submit their perspective of what happened and/or shouldn't have happened. Nothing should be done retroactively, but this is for future improvements... Each competition week at least the Head Referees volunteering that weekend (if not all Head Refs) should be made aware of how to call the controversial rules of the year (and how to handle the bizarre situations that sometimes result) as valuable case studies. I think this should help continuity significantly.

Related to this, the Aidan Brown, Frank Merrick and the GDC (or whoever is ultimately responsible for Head Ref training) should take special care to flow down a mindset to the Head Refs... if they want to accuracy and fairness to always be the top priority, then make that a top priority. If they want to minimize opening up and reviewing match decisions (reviewing calls, changing decisions, adjusting scores, etc.) for the sake of time or efficiency, then make that the top priority. Head Refs should know what should be prioritized above all else so that individuals are dealing less with the particular personality of the ref, and more with the policies standard across all events and matches.

3. Video Review should be implemented in clearly defined situations. I'd say ordinary fouls (called or not) shouldn't be reviewable. Yellow and Red Cards definitely should be reviewable. Scoring actions probably should be reviewable. Whatever the defined circumstances are, make them explicit. Also define the allowable time frame for review. Require that the teams be in the question box by X minutes (after arena reset signal? after scores are posted?) and define that if team supplied video is to be allowed (rather than event video only), it be submitted within a certain time window (perhaps in addition to the time to be in question box). Define if it is allowable for Qual matches and for Elim matches, or for one only. Give a single video review coupon per team for Quals and/or per alliance for Elims.

I think if implemented correctly, Video Review should actually take a lot of stress off referees... a lot of the time only a single zebra-ref is watching a play that is a critical, match-deciding call. Perhaps its a red/yellow card... with only one set of eyes and a situation that may have occurred quickly, that particular individual is under an enormous amount of pressure. When the refs huddle afterward to discuss, the only facts that they have available is usually a single account of a situation that happened in a matter of seconds. Its not an enviable situation, and if video review is implemented well, it can be dramatically minimized.

All-in-all, I do think the Head Refs and Refs do a tremendous job. They volunteer a lot of time and know they're in a position of which teams have tremendous expectations... They're a fantastic bunch whose dedication to fairness inspires the community to expect even more of them.

Re: Movement Towards Video Review
 

Its not complex. The review was initiated for a specific call. Thus the review can only pertain to that call/miscall.

Re: Movement Towards Video Review
 

Interesting to consider how this (or video review) would impact the "number of matches at an event" discussion on other threads.

Re: Movement Towards Video Review
 

It would make more sense to move towards more automated scoring/refereeing than to move towards video replay.

Re: Movement Towards Video Review
 

I would much rather have video replay as a backup to the automated systems. Things like Newton's qualification match 86 where the automated system miscounted the autonomous boulders could easily be resolved with a video review system.

I don't think video review would be an effective tool for correcting subjective fouls because the full match impact is difficult to assess which makes appropriate penalties tricky, but it would go a long way towards fixing things like missed crossings and other missed scoring conditions. The fact that missed scoring conditions are even a thing is ridiculous. Let the referees do the things humans are good at (assessing subjective penalties and fouls) and get video review for the objective calls.

Re: Movement Towards Video Review
 

Yes... very much. Even the generally excellent automated goal scoring this year was fallible (NE CMP SF2-2 original, CMP Newton Q71, CMP Newton Q86)... as far as we noticed. Each instance ended up having a different result though! NE CMP SF2-2 was replayed because of it (due to close score), CMP Newton Q71 had the score simply corrected, and CMP Newton Q86 had the score "corrected," but not to the right number.

In Newton Q86, 217 and 1519 together scored 2 auto HGs and then scored 9 teleop HGs. Video replay confirms this. Unfortunately, the automated scoring system only counted 1 auto HG and 9 teleop HGs. We lost by 10 points.

We attempted to get the score corrected, but even though the refs easily remembered the 2 auto HGs, there was no way anyone remembered exactly how many teleop HGs were scored. So the refs did end up "correcting" the auto score by changing a teleop HG to an auto HG (in Q71 one of our auto HGs was counted as a teleop HG due to it being shot right at the buzzer and then taking a while to filter down to the counter, and was corrected similarly)... this made our auto total correct - which is good for rankings - but still left the score 5 points lower than the truth, which meant that we still lost.

Anyway, we certainly appreciated the effort of Abby Perry, the Newton Head Ref, to correct the scores in Q71 and Q86. The Ref's memory was adequate to recall the correct scoring in Q71, but not in Q86 (no fault of the refs!!). A tool in their belt, in the form of instant replay, could help when there's uncertainty or gaps in the refs' memory...

Re: Movement Towards Video Review
 

I meant more like basketball or hockey I which majority of the refs job is to call fouls not scoring. In football the ref may signal that a touchdown or field goal is scored but in 2013 they made all scoring plays reviewed by a replay official. This means that the actual scoring is done by somebody other than the refs on the field.

I think it would make sense to have a ref per robot and then a score keeper per robot. This would require more volunteers but I think the FRC community would be able to support that.

Re: Movement Towards Video Review
 

Sure, however those are all sports where the scoring condition takes place under significant interference from the opposing team or simultaneous actions from multiple parties. Most scoring conditions in FRC games are most similar to basketball, where all that matters is the accomplishment of a single condition by a single player. I might be wrong but I can't think of a basketball game where a basket wasn't awarded because the ref wasn't watching or because the hoop sensor didn't trip. The closest it comes is a player missing the buzzer during a buzzer beater.

Re: Movement Towards Video Review
 

You are absolutely right. In basketball they will even review a shot to make sure it is a 3-pointer and if isn't then they will retroactively change the points. I think if FRC were to implement a review system they should follow the NBA and NHL.

Re: Movement Towards Video Review
 

I like some reviews. Others, not so much.

One kind of review I like is when someone performs a review/search to look for an existing on-target thread before starting a new one. ;)

In the case of this discussion, just a few weeks ago this topic was covered rather thoroughly here. Should we move the discussion back to there so that records of our thoughts are better organized, and so that we reduce duplicated effort?

Or perhaps we should re-open this 2005 thread ?

I haven't noticed any new insights in this thread yet (but, hey, I am only human, maybe I missed one?).

I'm not saying that the topic is dead. I am saying that anyone wanting to make real headway advancing their arguments should add to what has already been discussed. Let's not doom ourselves to repeating the past.

There are plenty of sound technical, game design, and implementation suggestions and objections in the past thread(s). Instead of (re)(re)(re)(re)...discovering them; how about reading, digesting, summarizing, and building upon them?

Blake

PS: It's obvious from the (Kennedy assassination) Zapruder film that a bullet drives President Kennedy's head backward when he is shot ... UNTIL, you carefully analyze the footage frame-by-frame, and notice that in roughly the time between two frames a bullet's initial impact drives his head forward, and then during the next few frames his head recoils (relatively slowly) backwards. The devil is in the details.

Re: Movement Towards Video Review
 

Honestly, I'd just like it if they told us what penalties had been called and who they were on.

Re: Movement Towards Video Review
 

Seconded.

The head ref on Hopper called fouls on 1678 and 971 during our two ball auto routines throughout the weekend. When approached by our teams in the question box, the head ref would not tell our students what foul they had called.

It was a bummer.

-Mike

Re: Movement Towards Video Review
 

Agreed.

The Ref's have radios. I have thought there should be a scribe. When a foul is called, the Ref calls out the color, robot number, and foul. The scribe then writes it down. Maybe even with a time mark (countdown clock).

I know, one more volunteer (probably with Ref level training). But, it keeps the ref from having to take their eyes off the field to write down the information. Maybe add one more ref to the team, and scribe is one of the positions they rotate through.

Re: Movement Towards Video Review
 

This was quite annoying, what makes me laugh is 971 dropped down their intake to prove they weren't over the line. Wonder what happened when auto started? :confused: