Video Review Needs to Happen Now

[bdaroz]

Originally Posted by DonRotolo

Side note: If you have never volunteered as a referee, you got nothing to say.

I would like to volunteer, as I had planned to this year, however as my son is on a rookie team I was needed far more there than the local regional.

I would just like to say, even though emotions and feelings may be running high here, everyone, I believe, wants the same goal. A positive, inspiring, experience for the students/teams, as well as the volunteers. I'm sure refs aren't thrilled to have to handle (or not) "missed" calls throughout a competition. Be it better training, more refs, video replay, better game design, or some combination thereof, we all seem to be trying to achieve a common goal of better officiating, and a better experience for everyone involved.

At the end of the day, let's remember, these are students we're trying to inspire, and volunteers who give their time and energy freely. We owe it to them to respect that, and to respect each other.

[alicen]

Originally Posted by bdaroz

Be it better training, more refs, video replay, better game design, or some combination thereof, we all seem to be trying to achieve a common goal of better officiating, and a better experience for everyone involved.

I just want to say that ref training this year was FAR improved from previous years. It was also far less forgiving!

[EricH]

Originally Posted by alicen

I just want to say that ref training this year was FAR improved from previous years. It was also far less forgiving!

Ouch, yes. I concur. And the on-the-job part (y'all call it practice day) is pretty tough too.

[Ryan Dognaux]

Originally Posted by DonRotolo

tl;dr.

No. This is a bad idea. Ignoring the cost and logistical challenges (which are not trivial), the time consumed is far too high.

....

Side note: If you have never volunteered as a referee, you got nothing to say.

This is simply false. Go back and actually read my post. If you want I will gladly post a BOM breakdown of St. Louis FIRST's setup to refute the cost claim.

As far as the claim that 'it will eat up too much time' - in my opinion, above all else, the team experience needs to be the emphasis of the event. I think this is something that is forgotten by some FRC volunteers who have been doing this for decades. It's not about 'keeping the event on schedule' to the exact minute - honestly who cares if we run 15 minutes late? These students have spent countless hours, teams are spending thousands of dollars. I think the least we can do is consider this to improve event quality.

Providing the head referee another tool to make the team experience better and improve the quality of calls at the cost of 15 additional minutes (if EVERY alliance challenged a call in this theoretical scenario) is a no brainer to me.

The argument that 'stuff happens / teams need to suck it up and move on' is weak. It's another excuse to not improve the consistency of calls at events. The solution I'm suggesting gives the head referee another tool to make the correct call - which I think they are always trying to do. The head referee job is really hard. Why wouldn't we want to help them out and give them an option?

Originally Posted by XaulZan11

I haven't read this entire thread so I apologize if it was already brought up, but which offseason event will be piloting a replay system? The best way to convince FIRST to make this change would be to do it and show them it works and is practical.

I volunteer the Gateway Robotics Challenge on October 1st.

[JohnFogarty]

I'll design, setup, and man a replay solution of my own design at SCRIW here in South Carolina.

Just in case you didn't know. SCRIW is played in a high school gym. Much like district events across the country are.

[Kevin Leonard]

Originally Posted by JohnFogarty

I'll design, setup, and man a replay solution of my own design at SCRIW here in South Carolina.

Originally Posted by Ryan Dognaux

I volunteer the Gateway Robotics Challenge on October 1st.

^^^Champions right here.

[patar8746]

Thank you all so much for keeping the discussion going, I've never been the OP for anything nearly as heavily discussed or important. Anyone keeping the subject open makes the discussion bigger and thereby attracting more views, but I really have to hand it to people volunteering to put in the effort through refereeing. I also plan on volunteering as a ref as soon as I'm in a position to (haven't been able to attend an event without teams I'm involved with for years), but those of you designing and building setups and running off season events to test them have the potential to prove to FIRST that this can work, and I know if we are wrong we will admit it, but at least then we'll know, and be much more able to accept the current system. Debate has gotten heated, but people on both sides are really making me proud to be a part of the FIRST community.

TLDR you guys are pretty great.

[ATannahill]

Originally Posted by Sperkowsky

<snip>

Before anyone signs me up on vims know that I already inspect for FTC and plan on inspecting for FRC. I just became a Deans List finalist and I am going to volunteer at worlds. This is not a matter of me complaining just simply supporting the fact that stuff needs to change.

Originally Posted by DonRotolo

<snip>

Side note: If you have never volunteered as a referee, you got nothing to say.

I want to address this. Inspecting is nothing like reffing. I have done both of these jobs (and many more) in the past two years. Reffing puts you in arguably one of the toughest decision making positions many times more than any other position, often without having the ability for someone else (let alone your volenteer lead (LRI, Head Ref, etc.)) to see it happen. An inspector can call another inspector or the CSA over to look at a problem during inspection, refs rarely have the ability to have someone else see what a robot did in the first six seconds of tele-op. Don't get me started on how vastly different the training/tests are for the positions.

I'm not going to totally disregard anyone's claim that calls have been missed or that ref quality could be improved, but it will come with much less weight than the experience of people that have the experience of being in the role for many years.

[Headphones]

I agree that something needs to be done to address situations where the referees miss a score by an alliance. In 2014, we were sent home from two separate events when referees missed an assist. Both times we had video evidence refuting the calls. It's ridiculous to invest hundreds of man-hours, pay thousands of dollars, and then be sent packing by a distracted referee.

[EricH]

Originally Posted by Headphones

I agree that something needs to be done to address situations where the referees miss a score by an alliance. In 2014, we were sent home from two separate events when referees missed an assist. Both times we had video evidence refuting the calls. It's ridiculous to invest hundreds of man-hours, pay thousands of dollars, and then be sent packing by a distracted referee.

Just be careful with the inflammatory language there. I agree that it's frustrating, but to call the referees distracted should be very far from the truth, and taken as a little bit of unnecessary "taunting" (for lack of a better word).


That being said: I can't say for sure, but I might be thinking about trying something involving replay at the local offseason out here. Too many variables need to fall into place, though, to make a call one way or the other at this point--including who ends up being the head ref at that event.

[gblake]

The OP wrote

Originally Posted by patar8746

Please ...

My reply is, "No."

[patar8746]

Originally Posted by Donut

I'll bite. I was a student Coach and Driver for 3 years, mentor Coach for 2, and have reffed the last 2 years (I'll be a Coach at AZ West later this year also). As alicen and EricH noted I'm sure there are a growing number of alumni who have been on both sides...

...I think video review could help with getting calls correct, but it needs to be used in limited cases to keep events running smooth and to limit its use to fixing errors that are significant and can be reasonably evaluated. Building off of other suggestions in this thread I would propose:

• Each Alliance is allowed one challenge/review in the playoffs. Unlimited challenges will result in reviewing every somewhat close match since no one wants their event/season to end. If the system works it could be expanded to each team having one challenge for all qualification matches also but not more to limit review quantities.
• Head Ref leads the review process and delivers the final call, with other refs who were involved in that call assisting if necessary. The rest of the refs keep the next match getting setup correctly so that minimal time is added to normal field reset. Adding a review process should not require an increase in referee headcount given the difficulty in finding referee volunteers already.
• Only match scoring errors (and penalties that lead to an automatic score) can be reviewed. This year that would mean defense crossings, challenges, scales, autonomous points, G13, G28, and boulders (though I am not sure if allowing review of an automatically scored element is reasonable to review, counting balls/disks scored in a match from a video is time consuming and more prone to error). Fouls are not reviewable as it is not easy to determine what fouls were assessed from a video and many involve a judgement call by a referee who has a better view than a camera or driver in their station will.
• The score or lack of being reviewed must be significant enough to affect the outcome of the match (or an RP being awarded for games like this year). Reviewing whether a crossing was awarded in autonomous or teleop in a 40 point blowout is a waste of time and the implications on ranking tiebreakers are not significant enough to justify the resources for that.
• Video evidence must be indisputable to change a call. The point is to receive credit for an obviously missed score, not debate further a close call that a referee already used their best judgement on (such as barely breaking contact with the Outer Works and Sally Port door)...

Overall I think that a review system would only be helpful in a small handful of circumstances, such as a missed stack last year or an obvious defense crossing this year...

...Also I thought I'd throw in that I love Stronghold, the referee jobs are hard this year but this is one of the best games (especially for spectators) I've seen in FRC.

^^^EXPERIENCED REF AND DRIVER

This ruleset is great, and I think this could be our base rules, minus defining where cameras are to be placed and what video feeds can be used. Some variations to try would be changing how many reviews a team gets (especially allowing them to keep reviewing if they aren't ever wrong), allowing foul reviews, implementing other penalities for baseless video requests, and whether or not to allow review in quals vs elims. Great start.

For camera placement, I'd suggest using existing stream setups to start. Variations could be placing cameras at or as close to bird's eye as possible, and not adding new cameras but allowing fans/media reps to submit video, and referee body cams?

Originally Posted by gblake

My reply is, "No."

No chute door?

[Ryan Dognaux]

Originally Posted by gblake

The OP wrote
My reply is, "No."

Super insightful, thanks Blake. Added a lot to this discussion. /s

[gblake]

Originally Posted by Ryan Dognaux

Super insightful, thanks Blake. Added a lot to this discussion. /s

Folks you can PM me if you care to. A longer post by me would repeat what other experienced people already wrote.

Whatever faith people put in my opinions should simply be added to the "No" column, if anyone is keeping score.

Hopefully this added context explains the purpose of my earlier, succint, no-soapbox, horse-is-already-dead post.

Blake

[patar8746]

Originally Posted by gblake

Folks you can PM me if you care to. A longer post by me would repeat what other experienced people already wrote.

Whatever faith people put in my opinions should simply be added to the "No" column, if anyone is keeping score.

Hopefully this added context explains the purpose of my earlier, succint, no-soapbox, horse-is-already-dead post.

Blake

Horse is both alive and dead until we open the box. We should get this tested and if we're wrong we're wrong, if not FIRST can be made better. Why not take that chance?