Video Review Needs to Happen Now

[patar8746]

Originally Posted by EricH

I might agree to the use of video replay, on one condition only.

But that one condition will, in all probability, ruin the entire concept for most of the proponents of using replay.

Ready?


You MUST use the video to prove that you LOST a match that you won, as well as the other way around. That is, if video you take shows that the other alliance should have won the match, and they did not, you need to provide that video to the referees and tell them that you should not have won the match.

Anybody got any objections? How about when it's F3 and you just won the regional by a dubious call that should have gone against you?

Thank you for posting this. I hadn't thought about this before and I fully agree. I think all steps possible should be taken to ensure fair play, and especially in the culture FIRST is trying to promote and is so important, if a team knows they lost and can prove it they should because it's the right thing to do, like using your timeout to give opponents more time to make repairs, or being honest in describing your abilities to teams looking to make picks.

That being said, I don't know how a rule mandating this would be enforced, or if it would need to be. To this point, I have seen so many examples of Gracious Professionalism in the way students handle competing. I've witnessed teams lend parts, timeouts, expertise, and even drivers to short-staffed teams, often contributing to losses. There's no way to determine whether or not a team has the necessary video, nor is there a way to confiscate a device or files so the refs could view them, but because of what FIRST is, I can't see a need for such a rule.

[Lij2015]

The only way for FIRST to accept video evidence is if they provide the video evidence on FIRST owned and operated equipment. I agree that teams can sometimes be cheated out of that, my team was one of them in Semi Final 3 at Virginia 2014 with a 50 pointer that really should've been a 20 pointer.

But with that being said, I do think they should accept video if they are the ones providing it so that it can be unbiased and incorporate all angles and such. From someone who's been there, I feel you.

[Chris is me]

While games will never be free of tough calls, quite honestly a lot of the problems that call for video review shouldn't be problems in the first place. We need a game that is designed to minimize referee calls and not patched up with dozens of rules trying to legislate the ideal / intended way to play the game.

Specifically, games with scoring determined by humans watching for actions should have humans devoted solely to watching those actions. We didn't learn this lesson in 2014?

[JohnFogarty]

You'd like me to go through every part of your argument? I have loads of time today.

Originally Posted by Lil' Lavery

Video review opens a can of worms that should be left closed. Beyond simple time delays, it poses....what level of evidence is needed, what is the timing/procedure for calling video reviews*, how many video reviews are allowed

Time delays are a valid concern. I'll use the Orlando Regional as a benchmark for a lot of things I'll talk about here since that was my most recent experience. We were 1.5 hours behind schedule pretty much all the time. Taking up the time of normal refs with this new video system I'm proposing would not work. Yet, I see the solution as simple and I'll provide my solution.

In qualifications if there is a disputed match where a team believes there was something not scored correctly we could have an additional referee who's entire job it was is to review video to sort these problems out. You don't even have to take the time of the normal match refs to do this sort of after-match verification. The video ref could take a look at the camera view footage to determine if the appropriate call was made.

The review time for videos would be kept short if a designated video review ref could not find indisputable evidence that the call was botched then there would be no changes.

In eliminations where the match scores are in my opinion even more critical to maintaining the quality of the event you can follow a similar procedure. Give at max 5 minutes to determine the call.

Originally Posted by Lil' Lavery

concerns regarding what source of video reviews should be allowed, how many videos should be allowed

As I said previously. The only video that would be reviewed would be that from an official FIRST mandated system.

Originally Posted by Lil' Lavery

A simple overhead camera? I'm assuming you mean similar to the fisheye cameras used by FiM/MAR/Indiana. Because if you mean a true overhead camera, that's simply not a realistic option at the vast majority of FRC venues (and certainly not an option without additional rigging costs on the venue side). And if you mean some other static camera (or even a true overhead), there are still plenty of disputes that will not be able to solve. Was a robot's wheel touching the outerworks at the end of autonomous, or just their bumper overhanging? Did their mechanism extend more than 15" beyond the frame perimeter? Did their robot fully stop contacting the drawbridge door momentarily? What was the game clock at that second during the overhead view? Heck, I'm watching the Tippecanoe stream right now, and there are still quite literal blind spots behind easy drawbridge/portcullis (as well as smaller ones behind the towers). No single camera is going to solve all FRC disputes.


e; To demonstrate my point, I just captured this from the NYC webstream. While the lack of quality is due to stream compression, the general point still stands. This is what typically constitutes an overhead view in FRC. Did the robot in the orange circle cross the defense? Commit a penalty?

A overhead camera like this one. https://youtu.be/PNs40CrPWUk?t=10s
A solution would have to be found. FIRST provides the field, the ref system, etc. This would have to become part of it.

It would very obvious that the camera would have to provide a high enough resolution video of the entire field to be validly able to determine calls.

Some of those calls that you just mentioned can't even be called consistently by the refs with their own eyes during the course of a match let alone an event. (i.e 15" perimeter rule, crossings, etc.) If you watched that video you'd even see that crossings were not being counted correctly even over the simple defenses.

Blind spots are valid and just like in football where sometimes a call can't be made definitively even with camera angles a call would just have to be left to stand. HOWEVER, if it can be proven with a simple system like the overhead camera like I am proposing than that alone is a drastic improvement.

[patar8746]

I'd actually argue that any video provided should be considered, especially if FIRST's camera(s) can't get a good angle on a situation but outside video can. When clear video can't be provided or the situation for any reason remains impossible to determine one way or another beyond any reasonable doubt, the Referee ruling should stand. Video of an event can't really add bias without showing an incomplete picture thereby making it ambiguous. Therefore I see no reason not to at least allow Referees to consider outside clips.

[EricH]

John, I'm assuming that you're volunteering as the video referee for any and all events you're attending. If you're not attending one event per week, you're not attending enough events. You just added one volunteer--in an extraordinarily tough position--that will need training. I don't see anybody volunteering for that one. We've got enough problems finding referees as it is.

Unless you're volunteering to head out to some of the areas that can't find refs and give them a hand, of course.

[dodar]

Originally Posted by EricH

John, I'm assuming that you're volunteering as the video referee for any and all events you're attending. If you're not attending one event per week, you're not attending enough events. You just added one volunteer--in an extraordinarily tough position--that will need training. I don't see anybody volunteering for that one. We've got enough problems finding referees as it is.

Unless you're volunteering to head out to some of the areas that can't find refs and give them a hand, of course.

Why does it need to be a ref? All it needs to be is someone who knows the game. They wont be looking for anything unless a team came forward with a specific request. And even then the team could point out the exact moment they believed something happened. The person doesnt need to be a true referee.

[JohnFogarty]

Originally Posted by EricH

John, I'm assuming that you're volunteering as the video referee for any and all events you're attending. If you're not attending one event per week, you're not attending enough events. You just added one volunteer--in an extraordinarily tough position--that will need training. I don't see anybody volunteering for that one. We've got enough problems finding referees as it is.

Unless you're volunteering to head out to some of the areas that can't find refs and give them a hand, of course.

Sure. If that's what it takes I'd volunteer. I'll make the whole system if I have to.

[James1902]

Originally Posted by JohnFogarty

Here's where I start to get rustled.

A simple overhead view of the field can be used to solve all disputes in FRC.
There is very little room to argue with this. FIRST could implement the system themselves as its not that complicated of a system to put in place.

As to who's video? Obviously the only video that would be reviewed would be from FIRST's own camera system. There would be no 3rd party camera footage viewing allowed, similarly to how things are now.

This isn't sports. In FRC the game pieces are simple, the rules are simple, and the problem is no where as complicated as you think it is.

My perspective: I've been a Head Ref for FTC events for several years now.

A few things:

True overhead camera views (like the one RoboShow set up at the Orlando Regional this year) are only simple if a number of venue conditions are met. It needs to have a convenient catwalk or overhead trusses that you can mount the camera on (usually a decent expectation at a regional, but not in a district event held in a local high school gym, or in a number of other venues that regionals use.) And you need a decent cable to connect it to your switcher that's usually a couple hundred feet long at least. And, of course, it needs to be a fairly high quality camera with either a wide field of view or a decent zoom depending on how close to the field your mount point is.

Even if all of these conditions are met, I wouldn't call the set up "simple".

Also, it's often not the regional that's running the cameras. Usually the video put onto the screen at the event is run by a contractor.

I would love to see every FIRST event with a broadcast that could support such a replay system, but I don't see that happening, especially with the push to go towards lower cost district events in smaller venues.

[EricH]

Originally Posted by dodar

Why does it need to be a ref? All it needs to be is someone who knows the game. They wont be looking for anything unless a team came forward with a specific request. And even then the team could point out the exact moment they believed something happened. The person doesnt need to be a true referee.

Because the vast majority of the calls that would be made are judgement calls. FIRST seems to have made the determination that only referees can provide that judgement in some years (2014 and 2016 come to mind). I can't say I'd trust someone who just "knows the game" with determining whether team X crossed a defense--there is a very specific definition of Crossing, and that very specific definition has led to numerous teams not quite meeting it and then complaining about it not being called. Just trust me on that.

If you don't have the training of a referee, why would you try to make a referee's call?

[Kevin Leonard]

So here's a situation that occurred last year:

2015 Tech Valley Regional, 20-5254-3624 had an incredible match in a make-it-or-break-it 2nd quarterfinal match, placing 4 stacks total for the first time. When the scores came up, it had our alliance with something like 119 points, and as we analyzed the score, we realized they had only credited us for 3 stacks!

We sent some students to the question box, and the referees came together and discussed that they did remember us having 4 stacks up.

From my understanding of the situation (I was not in the question box nor in the referee's discussion), they then looked at video provided by two different teams that showed 4 stacks built 5-6 high and ended up reversing the call of that match, which ended up allowing us to move on to the semifinals.

Tech Valley is a generally relaxed event, with some great referees and teams who are always gracious, and I don't think anyone involved thought what the referees did was unfair. Am I wrong?

[MrTechCenter]

I don't think video replay is ever going to work out in FRC, but one aspect I think needs to change is that when teams go to the question box, their concerns should actually be heard. This week I've seen some blatantly wrong calls and also some final scores that were incorrect and teams going to the question box usually got about a one sentence explanation before being waived off.

[smistthegreat]

Originally Posted by Kevin Leonard

So here's a situation that occurred last year:

2015 Tech Valley Regional, 20-5254-3624 had an incredible match in a make-it-or-break-it 2nd quarterfinal match, placing 4 stacks total for the first time. When the scores came up, it had our alliance with something like 119 points, and as we analyzed the score, we realized they had only credited us for 3 stacks!

We sent some students to the question box, and the referees came together and discussed that they did remember us having 4 stacks up.

From my understanding of the situation (I was not in the question box nor in the referee's discussion), they then looked at video provided by two different teams that showed 4 stacks built 5-6 high and ended up reversing the call of that match, which ended up allowing us to move on to the semifinals.

Tech Valley is a generally relaxed event, with some great referees and teams who are always gracious, and I don't think anyone involved thought what the referees did was unfair. Am I wrong?

No comment on anything else, but I will say that the game manual (both last year and this year) contained the following statement:

No event personnel, including the Head REFEREE, will review video, photos, artistic renderings, etc. of any MATCH, from any source, under any circumstances.

[Kevin Leonard]

Originally Posted by smistthegreat

No comment on anything else, but I will say that the game manual (both last year and this year) contained the following statement:

You're right, but we're also dealing with high school students who put their heart and soul into a robot and into every match, and kids seeing their seasons end because of something not being counted inspires nobody.

(also in this specific case its very easy to tell regardless of what video it was that the stack was up- that's not really arguable)

[Landonh12]

With the addition of a fish-eye view camera, two seperate cameras on each side of the field could also be used. This doesn't seem like a very hard concept. The Orlando regional archives all video footage on YouTube literally minutes after the match is played. This requires a good internet connection at the venue and a lot of other factors.

Simply setting up three cameras and taking video of each match is not a hard thing to do. Currently there are three cameras set up at CVR that give a pretty good view of most of the field. For instance, a student can go to the question box and tell the refs that there was a mistake on their part in the last match at 1:39. The refs can then go on a computer on the FMS table and watch that specific portion of the match and make a ruling.

If there is no indisputable evidence of the question by the student, then it can be said that, just like in the NFL, there is no hard evidence for the call to be overturned.

I have been affected by wrong calls several times in my 3 years as driver of 364. (especially at worlds, where our ranking was affected). Sure, it's an iffy issue, but I'd like to see what FIRST has to say about something like this.

Also, you'd think that with all of the money that comes in for a regional, something like this wouldn't be much of an issue.