View Full Version : Video Review Needs to Happen Now
patar8746
13-03-2016, 15:42
Please share this, tweet this, etc. FIRST students put too much effort into their robots to be cheated out of their achievements like this anymore.
As much as FIRST has done for me over the years, one of my biggest problems with the organization is the rule that forces Referees to ignore video evidence exposing bad calls made on the field. Just today at the New York City Regional, teams 395, 2869, and 1546 were cheated out of their win in the second quarterfinals match by a failure to count a defense cross that would have earned 5 points for cross as well as a 20 point bonus for a successful breach of defenses. This forced a third match which this alliance, flustered and confused by the flawed results, lost, causing a deserving group of students to miss a chance to play in the semifinal matches. Footage was provided from 2 different angles but not considered as per rules, and the match remained in the wrong team's hands. Even a replay of the match couldn't be considered. We all know from experience how hard all these students worked for this moment, and to lose in an unfairly judged match is an insult to their efforts and could drive them away from their interests in science and technology, the exact opposite of what FIRST stands for. The teams' hopes were destroyed because the rules don't allow Referees to use all available information to admit their mistakes. This is not what FIRST is about and I am ashamed of how this situation wasn't handled. Its probably too late to correct this individual incident, but we can stop this from happening again. FIRST, please change this policy against video reviews, and referees, please at least consider replays of the whole match. Some already do this, but it needs be standardized to stop this from happening again. I'm open to discussion here, because this is something that needs to be talked about, but please spread the word.
I've heard counterarguments that this would prolong regional events and video could be unreliable. These are no excuses to discredit the effort of these students, as I witnessed firsthand Referees spending more time refusing to watch video than it would have taken to watch the clips 3 times over. In addition, professional sports have already solved the video reliability issue by necessitating that the calls be overturned beyond any reasonable doubt by the evidence, and it's worked well. This can happen, and I hope as a community we can come together to make sure matches are fair and teams earn the credit they deserve.
Note regarding edits: the original edit of this post contained content unfairly judging the way the Referees handled this situation that I sincerely regret ever writing, and I sincerely apologize for any damage I may have caused. For every one mistake made, volunteer referees have made thousands upon thousands of fair calls and correct decisions, without which FIRST wouldn't be able to exist. Thank you to all involved in this tough rule writing and enforcing process for taking the time to help students despite harsh negative feedback and disrespect. The few and far between problems have to this point been handled as well as they could have been, but I hope to be able to help further improve the process for teams in the future to have the best experiences they can.
Regardless of your feeling towards the rules, the referees acted fully within the rules with regards to video rules and replays, so don't express your anger towards them.
Referee mistakes suck (hello 2014), and they happen, and I'm not going to tell you to get over it, but don't hate the player, hate the game.
patar8746
13-03-2016, 15:53
My apologies, I was rather angry when I wrote that and attacked the wrong thing. Edited to reflect the real problem in the rule.
Refs, thanks for volunteering, regardless of calls events don't run without you guys at all.
I agree with Gregor on this one, but also, I'd shoot an email to frcteams@firstinspires.org regarding your concern. I think it's a good idea for refs to accept video evidence (many a time have bad referee calls been detrimental to my own team's performance).
Lil' Lavery
13-03-2016, 16:11
I strongly disagree. Video review opens a can of worms that should be left closed. Beyond simple time delays, it poses concerns regarding what source of video reviews should be allowed, how many videos should be allowed, what level of evidence is needed, what is the timing/procedure for calling video reviews*, how many video reviews are allowed, etc.
*Anyone who's been an alliance captain before already knows the confusion and heartburn of figuring out timeout and back-up coupon timings.
Video review in sports has been anything but a panacea, and they have countless more camera angles and video analysis tools than will be at the disposal of referees in FRC.
Dear Patrick
(and Dear Others that think a team got robbed of a victory)
Thanks for posting and a bigger thanks for you volunteering to help out at the next two events. Your help will be deeply appreciated.
Video replay / electronic scoring isn't what you think it is. NASCAR races at ~30 tracks and has spent millions on putting sensors into the tracks to mark the position of the cars. MLB / NFL with their limited number of stadiums has spent millions of dollars along with more millions by FOX/NBC/CBS/ABC/ESPN on video technology to be able to manage video replays.
I'm super pleased that you've decided to dedicate you next two years to doing this work for FIRST and I'm super excited to listen to your announcement on the partner dollars you've signed up to contribute, your new TV funded dollars and how that charging people to come watch the event to also help fund this will work out. Or that you hit Powerball to pay for all of this and are donating the proceeds.
I'm hoping that a referee reaches out to you and has you come stand next to them on the field. One would think, from sitting in the stands, that being a referee is a piece of cake. In reality down on the field it's overly bright, it's noisy, it's a huge amount to watch.
Other than being the Volunteer Coordinator, there is no worse job in FIRST than being a referee. There isn't any upside since you are not getting paid, the pizza was pretty bad (cheese only really?) , and frankly you have a closet full of robot shirts. And I'm calling BS on that "Vertical Stripes are slimming"
Other than you are out there making a difference, making an effort to improve the world. Giving up your only non-replaceable thing in your life, time, to help other people.
To find out that they are ungrateful, condescending, and mocking your efforts to help them.
But, since you've signed up to be one of them at the next two events, I look forward to your post in 4 weeks to see how your world view has changed. Welcome to the world of being a volunteer!
Warmest personal regards,
Foster
(Why yes, that was filled full of snark and sarcasm. Make sure you send me your VIMS number on how many events you are helping at when you send your complaint mail).
[[ Edited to add: So I posted right after the post came up. I type slow and try to proofread, so I'm now number 6 behind an edited post. ]]
JohnFogarty
13-03-2016, 16:17
I strongly disagree. Video review opens a can of worms that should be left closed. Beyond simple time delays, it poses concerns regarding what source of video reviews should be allowed, how many videos should be allowed, what level of evidence is needed, what is the timing/procedure for calling video reviews*, how many video reviews are allowed, etc.
*Anyone who's been an alliance captain before already knows the confusion and heartburn of figuring out timeout and back-up coupon timings.
Video review in sports has been anything but a panacea, and they have countless more camera angles and video analysis tools than will be at the disposal of referees in FRC.
Here's where I start to get rustled.
A simple overhead view of the field can be used to solve all disputes in FRC.
There is very little room to argue with this. FIRST could implement the system themselves as its not that complicated of a system to put in place.
As to who's video? Obviously the only video that would be reviewed would be from FIRST's own camera system. There would be no 3rd party camera footage viewing allowed, similarly to how things are now.
This isn't sports. In FRC the game pieces are simple, the rules are simple, and the problem is no where as complicated as you think it is.
My perspective: I've been a Head Ref for FTC events for several years now.
Lil' Lavery
13-03-2016, 16:26
Here's where I start to get rustled.
A simple overhead view of the field can be used to solve all disputes in FRC.
There is very little room to argue with this. FIRST could implement the system themselves as its not that Complicated of a system to put in place.
As to who's video? Obviously the only video that would be reviewed would be from FIRST's own camera system. There would be no 3rd party camera footage viewing allowed, similarly to how things are now.
This isn't sports. In FRC the game pieces are simple, the rules are simple, and the problem is no where as complicated as you think it is.
I call BS.
A simple overhead camera? I'm assuming you mean similar to the fisheye cameras used by FiM/MAR/Indiana. Because if you mean a true overhead camera, that's simply not a realistic option at the vast majority of FRC venues (and certainly not an option without additional rigging costs on the venue side). And if you mean some other static camera (or even a true overhead), there are still plenty of disputes that will not be able to solve. Was a robot's wheel touching the outerworks at the end of autonomous, or just their bumper overhanging? Did their mechanism extend more than 15" beyond the frame perimeter? Did their robot fully stop contacting the drawbridge door momentarily? What was the game clock at that second during the overhead view? Heck, I'm watching the Tippecanoe stream right now, and there are still quite literal blind spots behind easy drawbridge/portcullis (as well as smaller ones behind the towers). No single camera is going to solve all FRC disputes.
Nor did you address any of the other questions I raised.
e; To demonstrate my point, I just captured this from the NYC webstream. While the lack of quality is due to stream compression, the general point still stands. This is what typically constitutes an overhead view in FRC. Did the robot in the orange circle cross the defense? Commit a penalty?
http://i.imgur.com/ZvDusul.jpg
Also, one thing with FRC that is different than traditional sports is that the game changes every year. Let that sink in for a moment. In traditional sports, let's say, soccer for example, the game never changes. Albeit maybe a few minor rules (I know the offside rule changes frequently) but nowhere near the frequency that new FRC games get introduced. This essentially means that at this time in the season (week 2), any FRC referee can only have 2-3 months of experience and by champs they could have 4-5 months (maybe) of referee experience, whereas traditional sports referees have 20 to 30 years playing the game with only minor variations.
May I add that FRC games are WAY more complicated than traditional sports games. I could argue this but I do not have the energy nor the full keyboard to use.
patar8746
13-03-2016, 16:42
Dear Patrick
(and Dear Others that think a team got robbed of a victory)
Thanks for posting and a bigger thanks for you volunteering to help out at the next two events. Your help will be deeply appreciated.
Video replay / electronic scoring isn't what you think it is. NASCAR races at ~30 tracks and has spent millions on putting sensors into the tracks to mark the position of the cars. MLB / NFL with their limited number of stadiums has spent millions of dollars along with more millions by FOX/NBC/CBS/ABC/ESPN on video technology to be able to manage video replays.
I'm super pleased that you've decided to dedicate you next two years to doing this work for FIRST and I'm super excited to listen to your announcement on the partner dollars you've signed up to contribute, your new TV funded dollars and how that charging people to come watch the event to also help fund this will work out. Or that you hit Powerball to pay for all of this and are donating the proceeds.
I'm hoping that a referee reaches out to you and has you come stand next to them on the field. One would think, from sitting in the stands, that being a referee is a piece of cake. In reality down on the field it's overly bright, it's noisy, it's a huge amount to watch.
Other than being the Volunteer Coordinator, there is no worse job in FIRST than being a referee. There isn't any upside since you are not getting paid, the pizza was pretty bad (cheese only really?) , and frankly you have a closet full of robot shirts. And I'm calling BS on that "Vertical Stripes are slimming"
Other than you are out there making a difference, making an effort to improve the world. Giving up your only non-replaceable thing in your life, time, to help other people.
To find out that they are ungrateful, condescending, and mocking your efforts to help them.
But, since you've signed up to be one of them at the next two events, I look forward to your post in 4 weeks to see how your world view has changed. Welcome to the world of being a volunteer!
Warmest personal regards,
Foster
(Why yes, that was filled full of snark and sarcasm. Make sure you send me your VIMS number on how many events you are helping at when you send your complaint mail).
[[ Edited to add: So I posted right after the post came up. I type slow and try to proofread, so I'm now number 6 behind an edited post. ]]
I don't know if you read the edits before noticing themon your edit, but by no means am I trying to discredit referees themselves. I agree that you have it tough, students and mentors alike can be ungrateful at times. However, I have never seen a more respectful approach than the one my students on 395 took to the issue. They did not get nearly as angry as I did, they did not lose their heads, and they did not root against the alliance that advanced to semis. Instead they decided to start making efforts to stop this from happening in future years despite being largely composed of seniors. All FIRST volunteers deserve respect, especially Referees and Judges, but a change to the video policy would be a huge step in the right direction. No teams could feel like they have any right to feel unfairly judged because calls would be right every time, or at least far more often. The students deserve fair play, and the refs deserve respect for their efforts and a way to make sure they can deliver what students ask of them. A video review policy will help both groups get what they deserve. If this still offends you, I don't know what to tell you.
Jessica Boucher
13-03-2016, 16:42
Just a note - this is an perennial tough subject and the conversation is good, but make sure to write it out, breathe, think, and edit appropriately.
If you're interested, theres a few threads in the archives about this subject too.
The fact that I have seen more than a handful of crossings not given credit several in auto is garbage. In the heat of a match I could see things being missed, but how does one miss a crossing in auto?
Tottanka
13-03-2016, 17:22
A related thread to video reviews this year:
http://www.chiefdelphi.com/forums/showthread.php?p=1555484#post1555484
I might agree to the use of video replay, on one condition only.
But that one condition will, in all probability, ruin the entire concept for most of the proponents of using replay.
Ready?
You MUST use the video to prove that you LOST a match that you won, as well as the other way around. That is, if video you take shows that the other alliance should have won the match, and they did not, you need to provide that video to the referees and tell them that you should not have won the match.
Anybody got any objections? How about when it's F3 and you just won the regional by a dubious call that should have gone against you?
Lil' Lavery
13-03-2016, 18:00
I might agree to the use of video replay, on one condition only.
But that one condition will, in all probability, ruin the entire concept for most of the proponents of using replay.
Ready?
You MUST use the video to prove that you LOST a match that you won, as well as the other way around. That is, if video you take shows that the other alliance should have won the match, and they did not, you need to provide that video to the referees and tell them that you should not have won the match.
Anybody got any objections? How about when it's F3 and you just won the regional by a dubious call that should have gone against you?
It's not video review, but big props to 4342 did something very similar in 2014 at Chestnut Hill. Their alliance was improperly given credit for an autonomous shot they missed in QF1-3, and members of 4342's drive team came to the question box with members of the opposing alliance. As a result, the head ref allowed for a replay of QF1-3, in which 4342's alliance ultimately ended up being eliminated. 4342 being willing to risk their trip to the SFs (thus effectively ending their season) on behalf of getting the call on the field correct was a terrific example of graciousness. I still applaud them for it.
patar8746
13-03-2016, 18:08
I might agree to the use of video replay, on one condition only.
But that one condition will, in all probability, ruin the entire concept for most of the proponents of using replay.
Ready?
You MUST use the video to prove that you LOST a match that you won, as well as the other way around. That is, if video you take shows that the other alliance should have won the match, and they did not, you need to provide that video to the referees and tell them that you should not have won the match.
Anybody got any objections? How about when it's F3 and you just won the regional by a dubious call that should have gone against you?
Thank you for posting this. I hadn't thought about this before and I fully agree. I think all steps possible should be taken to ensure fair play, and especially in the culture FIRST is trying to promote and is so important, if a team knows they lost and can prove it they should because it's the right thing to do, like using your timeout to give opponents more time to make repairs, or being honest in describing your abilities to teams looking to make picks.
That being said, I don't know how a rule mandating this would be enforced, or if it would need to be. To this point, I have seen so many examples of Gracious Professionalism in the way students handle competing. I've witnessed teams lend parts, timeouts, expertise, and even drivers to short-staffed teams, often contributing to losses. There's no way to determine whether or not a team has the necessary video, nor is there a way to confiscate a device or files so the refs could view them, but because of what FIRST is, I can't see a need for such a rule.
The only way for FIRST to accept video evidence is if they provide the video evidence on FIRST owned and operated equipment. I agree that teams can sometimes be cheated out of that, my team was one of them in Semi Final 3 at Virginia 2014 with a 50 pointer that really should've been a 20 pointer.
But with that being said, I do think they should accept video if they are the ones providing it so that it can be unbiased and incorporate all angles and such. From someone who's been there, I feel you.
Chris is me
13-03-2016, 18:09
While games will never be free of tough calls, quite honestly a lot of the problems that call for video review shouldn't be problems in the first place. We need a game that is designed to minimize referee calls and not patched up with dozens of rules trying to legislate the ideal / intended way to play the game.
Specifically, games with scoring determined by humans watching for actions should have humans devoted solely to watching those actions. We didn't learn this lesson in 2014?
JohnFogarty
13-03-2016, 18:11
You'd like me to go through every part of your argument? I have loads of time today.
Video review opens a can of worms that should be left closed. Beyond simple time delays, it poses....what level of evidence is needed, what is the timing/procedure for calling video reviews*, how many video reviews are allowed
Time delays are a valid concern. I'll use the Orlando Regional as a benchmark for a lot of things I'll talk about here since that was my most recent experience. We were 1.5 hours behind schedule pretty much all the time. Taking up the time of normal refs with this new video system I'm proposing would not work. Yet, I see the solution as simple and I'll provide my solution.
In qualifications if there is a disputed match where a team believes there was something not scored correctly we could have an additional referee who's entire job it was is to review video to sort these problems out. You don't even have to take the time of the normal match refs to do this sort of after-match verification. The video ref could take a look at the camera view footage to determine if the appropriate call was made.
The review time for videos would be kept short if a designated video review ref could not find indisputable evidence that the call was botched then there would be no changes.
In eliminations where the match scores are in my opinion even more critical to maintaining the quality of the event you can follow a similar procedure. Give at max 5 minutes to determine the call.
concerns regarding what source of video reviews should be allowed, how many videos should be allowed
As I said previously. The only video that would be reviewed would be that from an official FIRST mandated system.
A simple overhead camera? I'm assuming you mean similar to the fisheye cameras used by FiM/MAR/Indiana. Because if you mean a true overhead camera, that's simply not a realistic option at the vast majority of FRC venues (and certainly not an option without additional rigging costs on the venue side). And if you mean some other static camera (or even a true overhead), there are still plenty of disputes that will not be able to solve. Was a robot's wheel touching the outerworks at the end of autonomous, or just their bumper overhanging? Did their mechanism extend more than 15" beyond the frame perimeter? Did their robot fully stop contacting the drawbridge door momentarily? What was the game clock at that second during the overhead view? Heck, I'm watching the Tippecanoe stream right now, and there are still quite literal blind spots behind easy drawbridge/portcullis (as well as smaller ones behind the towers). No single camera is going to solve all FRC disputes.
e; To demonstrate my point, I just captured this from the NYC webstream. While the lack of quality is due to stream compression, the general point still stands. This is what typically constitutes an overhead view in FRC. Did the robot in the orange circle cross the defense? Commit a penalty?
A overhead camera like this one. https://youtu.be/PNs40CrPWUk?t=10s
A solution would have to be found. FIRST provides the field, the ref system, etc. This would have to become part of it.
It would very obvious that the camera would have to provide a high enough resolution video of the entire field to be validly able to determine calls.
Some of those calls that you just mentioned can't even be called consistently by the refs with their own eyes during the course of a match let alone an event. (i.e 15" perimeter rule, crossings, etc.) If you watched that video you'd even see that crossings were not being counted correctly even over the simple defenses.
Blind spots are valid and just like in football where sometimes a call can't be made definitively even with camera angles a call would just have to be left to stand. HOWEVER, if it can be proven with a simple system like the overhead camera like I am proposing than that alone is a drastic improvement.
patar8746
13-03-2016, 18:15
I'd actually argue that any video provided should be considered, especially if FIRST's camera(s) can't get a good angle on a situation but outside video can. When clear video can't be provided or the situation for any reason remains impossible to determine one way or another beyond any reasonable doubt, the Referee ruling should stand. Video of an event can't really add bias without showing an incomplete picture thereby making it ambiguous. Therefore I see no reason not to at least allow Referees to consider outside clips.
John, I'm assuming that you're volunteering as the video referee for any and all events you're attending. If you're not attending one event per week, you're not attending enough events. You just added one volunteer--in an extraordinarily tough position--that will need training. I don't see anybody volunteering for that one. We've got enough problems finding referees as it is.
Unless you're volunteering to head out to some of the areas that can't find refs and give them a hand, of course.
John, I'm assuming that you're volunteering as the video referee for any and all events you're attending. If you're not attending one event per week, you're not attending enough events. You just added one volunteer--in an extraordinarily tough position--that will need training. I don't see anybody volunteering for that one. We've got enough problems finding referees as it is.
Unless you're volunteering to head out to some of the areas that can't find refs and give them a hand, of course.
Why does it need to be a ref? All it needs to be is someone who knows the game. They wont be looking for anything unless a team came forward with a specific request. And even then the team could point out the exact moment they believed something happened. The person doesnt need to be a true referee.
JohnFogarty
13-03-2016, 18:30
John, I'm assuming that you're volunteering as the video referee for any and all events you're attending. If you're not attending one event per week, you're not attending enough events. You just added one volunteer--in an extraordinarily tough position--that will need training. I don't see anybody volunteering for that one. We've got enough problems finding referees as it is.
Unless you're volunteering to head out to some of the areas that can't find refs and give them a hand, of course.
Sure. If that's what it takes I'd volunteer. I'll make the whole system if I have to.
James1902
13-03-2016, 18:32
Here's where I start to get rustled.
A simple overhead view of the field can be used to solve all disputes in FRC.
There is very little room to argue with this. FIRST could implement the system themselves as its not that complicated of a system to put in place.
As to who's video? Obviously the only video that would be reviewed would be from FIRST's own camera system. There would be no 3rd party camera footage viewing allowed, similarly to how things are now.
This isn't sports. In FRC the game pieces are simple, the rules are simple, and the problem is no where as complicated as you think it is.
My perspective: I've been a Head Ref for FTC events for several years now.
A few things:
True overhead camera views (like the one RoboShow set up at the Orlando Regional this year (https://youtu.be/NKTn9B38IOw?t=10s)) are only simple if a number of venue conditions are met. It needs to have a convenient catwalk or overhead trusses that you can mount the camera on (usually a decent expectation at a regional, but not in a district event held in a local high school gym, or in a number of other venues that regionals use.) And you need a decent cable to connect it to your switcher that's usually a couple hundred feet long at least. And, of course, it needs to be a fairly high quality camera with either a wide field of view or a decent zoom depending on how close to the field your mount point is.
Even if all of these conditions are met, I wouldn't call the set up "simple".
Also, it's often not the regional that's running the cameras. Usually the video put onto the screen at the event is run by a contractor.
I would love to see every FIRST event with a broadcast that could support such a replay system, but I don't see that happening, especially with the push to go towards lower cost district events in smaller venues.
Why does it need to be a ref? All it needs to be is someone who knows the game. They wont be looking for anything unless a team came forward with a specific request. And even then the team could point out the exact moment they believed something happened. The person doesnt need to be a true referee.
Because the vast majority of the calls that would be made are judgement calls. FIRST seems to have made the determination that only referees can provide that judgement in some years (2014 and 2016 come to mind). I can't say I'd trust someone who just "knows the game" with determining whether team X crossed a defense--there is a very specific definition of Crossing, and that very specific definition has led to numerous teams not quite meeting it and then complaining about it not being called. Just trust me on that.
If you don't have the training of a referee, why would you try to make a referee's call?
Kevin Leonard
13-03-2016, 18:43
So here's a situation that occurred last year:
2015 Tech Valley Regional, 20-5254-3624 had an incredible match in a make-it-or-break-it 2nd quarterfinal match, placing 4 stacks total for the first time. When the scores came up, it had our alliance with something like 119 points, and as we analyzed the score, we realized they had only credited us for 3 stacks!
We sent some students to the question box, and the referees came together and discussed that they did remember us having 4 stacks up.
From my understanding of the situation (I was not in the question box nor in the referee's discussion), they then looked at video provided by two different teams that showed 4 stacks built 5-6 high and ended up reversing the call of that match, which ended up allowing us to move on to the semifinals.
Tech Valley is a generally relaxed event, with some great referees and teams who are always gracious, and I don't think anyone involved thought what the referees did was unfair. Am I wrong?
MrTechCenter
13-03-2016, 18:47
I don't think video replay is ever going to work out in FRC, but one aspect I think needs to change is that when teams go to the question box, their concerns should actually be heard. This week I've seen some blatantly wrong calls and also some final scores that were incorrect and teams going to the question box usually got about a one sentence explanation before being waived off.
smistthegreat
13-03-2016, 18:49
So here's a situation that occurred last year:
2015 Tech Valley Regional, 20-5254-3624 had an incredible match in a make-it-or-break-it 2nd quarterfinal match, placing 4 stacks total for the first time. When the scores came up, it had our alliance with something like 119 points, and as we analyzed the score, we realized they had only credited us for 3 stacks!
We sent some students to the question box, and the referees came together and discussed that they did remember us having 4 stacks up.
From my understanding of the situation (I was not in the question box nor in the referee's discussion), they then looked at video provided by two different teams that showed 4 stacks built 5-6 high and ended up reversing the call of that match, which ended up allowing us to move on to the semifinals.
Tech Valley is a generally relaxed event, with some great referees and teams who are always gracious, and I don't think anyone involved thought what the referees did was unfair. Am I wrong?
No comment on anything else, but I will say that the game manual (both last year and this year) contained the following statement:
No event personnel, including the Head REFEREE, will review video, photos, artistic renderings, etc. of any MATCH, from any source, under any circumstances.
Kevin Leonard
13-03-2016, 18:52
No comment on anything else, but I will say that the game manual (both last year and this year) contained the following statement:
You're right, but we're also dealing with high school students who put their heart and soul into a robot and into every match, and kids seeing their seasons end because of something not being counted inspires nobody.
(also in this specific case its very easy to tell regardless of what video it was that the stack was up- that's not really arguable)
Landonh12
13-03-2016, 18:52
With the addition of a fish-eye view camera, two seperate cameras on each side of the field could also be used. This doesn't seem like a very hard concept. The Orlando regional archives all video footage on YouTube literally minutes after the match is played. This requires a good internet connection at the venue and a lot of other factors.
Simply setting up three cameras and taking video of each match is not a hard thing to do. Currently there are three cameras set up at CVR that give a pretty good view of most of the field. For instance, a student can go to the question box and tell the refs that there was a mistake on their part in the last match at 1:39. The refs can then go on a computer on the FMS table and watch that specific portion of the match and make a ruling.
If there is no indisputable evidence of the question by the student, then it can be said that, just like in the NFL, there is no hard evidence for the call to be overturned.
I have been affected by wrong calls several times in my 3 years as driver of 364. (especially at worlds, where our ranking was affected). Sure, it's an iffy issue, but I'd like to see what FIRST has to say about something like this.
Also, you'd think that with all of the money that comes in for a regional, something like this wouldn't be much of an issue.
smistthegreat
13-03-2016, 18:57
You're right, but we're also dealing with high school students who put their heart and soul into a robot and into every match, and kids seeing their seasons end because of something not being counted inspires nobody.
(also in this specific case its very easy to tell regardless of what video it was that the stack was up- that's not really arguable)
You're 100% correct, and I'm glad that the situation was resolved to the satisfaction of those involved. I just wanted to point out that this verbiage was present in the manual last year as well. I wasn't implying that the revision shouldn't have counted.
I still want to know how so many crossings are being missed. I saw 5 today alone while watching streams on and off of wpi and the Blacksburg event. Let alone the ones I saw from the Waterbury event last weekend.
I asked my drivers about and they told me that during the drivers meeting the refs said if they didn't see it then it didn't happen, and not to come to the question box about it.
JohnBoucher
13-03-2016, 18:58
This discussion is good to have but I don't like bashing the referees or the disregard for the rules the you agreed to play under.
Please remember that 99% of the volunteers working any event are doing this for the love of the game. The game is played as fairly and evenhanded as possible. To suggest otherwise is a discredit to all those who give their time and energy to this.
We have all had thing go against us. GP allows us to move beyond it and appreciate what we have accomplished.
This discussion is good to have but I don't like bashing the referees or the disregard for the rules the you agreed to play under.
Please remember that 99% of the volunteers working any event are doing this for the love of the game. The game is played as fairly and evenhanded as possible. To suggest otherwise is a discredit to all those who give their time and energy to this.
We have all
This always gets me. Cant not agree to them when its the only game in town.
I still want to know how so many crossings are being missed. I saw 5 today alone while watching streams on and off of wpi and the Blacksburg event. Let alone the ones I saw from the Waterbury event last weekend.
Something to think about:
If a referee isn't sure that a crossing happened, they will not count it. There's a blue box to that effect. What that means in some cases is that a team doesn't quite clear the ramps (thereby remaining in the defenses) and reverses back over. That's not a crossing. Might not be terribly obvious on the webcast that their bumper (or other appendage) is hanging over, but there's that possibility.
Rangel(kf7fdb)
13-03-2016, 19:01
Because the vast majority of the calls that would be made are judgement calls. FIRST seems to have made the determination that only referees can provide that judgement in some years (2014 and 2016 come to mind). I can't say I'd trust someone who just "knows the game" with determining whether team X crossed a defense--there is a very specific definition of Crossing, and that very specific definition has led to numerous teams not quite meeting it and then complaining about it not being called. Just trust me on that.
If you don't have the training of a referee, why would you try to make a referee's call?
The person running it simply has to go to the spot where the student says something was missed or penalty was called wrong and bring a ref over to look at it real quick. Could honestly be in between matches too and only if a student has a valid complaint and a general time in the match in which it happened. And hey if the videos are all saved too then now we have videos of every match at every regional and we solved that problem too! :] Idk maybe that's the real reason I would like to see this happen.
JohnBoucher
13-03-2016, 19:02
This always gets me. Cant not agree to them when its the only game in town.
It's not the only game in town. I am the biggest advocate that the team is the customer but if you have done this for more one season, you should understand how this works.
It's not the only game in town. I am the biggest advocate that the team is the customer but if you have done this for more one season, you should understand how this works.
I have, 10 years in fact, and over those 10 years the rules have most definitely changed quite a bit.
Something to think about:
If a referee isn't sure that a crossing happened, they will not count it. There's a blue box to that effect. What that means in some cases is that a team doesn't quite clear the ramps (thereby remaining in the defenses) and reverses back over. That's not a crossing. Might not be terribly obvious on the webcast that their bumper (or other appendage) is hanging over, but there's that possibility.
Well when a team goes over one of the defenses and is 4' away from the driver station wall and doesn't get credit for crossing in auto that is a problem. Even if you didn't see them cross but as you look about the field and see 2 bots well past the outer works and don't push a button on the scoring screens that is an issue. I mean how did they end up there if they didn't cross??
Well when a team goes over one of the defenses and is 4' away from the driver station wall and doesn't get credit for crossing in auto that is a problem. Even if you didn't see them cross but as you look about the field and see 2 bots well past the outer works and don't push a button on the scoring screens that is an issue. I mean how did they end up there if they didn't cross??
Then that is something to bring up to the head referee. Just tell him that you were clearly past the outer works in auto and got no credit. If the Head Referee isn't listening, the next person to talk to would be the Volunteer Coordinator or Regional Director--they'll back up the head referee in on-field calls, but if enough teams are saying something they'll probably have a chat. And it it's just that one ref isn't pressing their buttons, then the head ref can take measures...
JohnBoucher
13-03-2016, 19:18
I have, 10 years in fact, and over those 10 years the rules have most definitely changed quite a bit.
The whole reason we do this has not changed.
The whole reason we do this has not changed.
We do this for the students. And we should do right by the students. Both at home and at competitions.
We do this for the students. And we should do right by the students. Both at home and at competitions.
In the real world, does somebody sometimes win X that they shouldn't have because Y?
Now the question becomes, is it better for the students to expose them to this now, or not?
Understand that I'm not saying that missed calls are a good thing. Learning how to handle missed calls, however, can be invaluable.
Then that is something to bring up to the head referee. Just tell him that you were clearly past the outer works in auto and got no credit. If the Head Referee isn't listening, the next person to talk to would be the Volunteer Coordinator or Regional Director--they'll back up the head referee in on-field calls, but if enough teams are saying something they'll probably have a chat. And it it's just that one ref isn't pressing their buttons, then the head ref can take measures...
Again when teams are told during driver meeting not to even come to the box for a missed cross it is kind of hard to expect these teenagers to push like that.
In the real world, does somebody sometimes win X that they shouldn't have because Y?
Now the question becomes, is it better for the students to expose them to this now, or not?
Understand that I'm not saying that missed calls are a good thing. Learning how to handle missed calls, however, can be invaluable.
In real life, you have access to lots of ways to challenge a ruling too.
Again when teams are told during driver meeting not to even come to the box for a missed cross it is kind of hard to expect these teenagers to push like that.
That's something to bring up with the RD/VC. I'd like to think that if a ref crew was missing crossings, they'd like to know that they need to keep a sharper eye out.
By the way, you weren't looking at any spy-bots by any chance, were you?
As a Rookie Mentor on a Rookie team I hesitate to wade into this, but I do have a question....
It seems clear between the 2015 and 2016 games the referees jobs have gotten clearly more difficult, mostly because of the number of things they must watch and keep track of now.
As to the question: Has the GDC increased the number of referees from 2015 to 2016 to handle this change?
As a Rookie Mentor on a Rookie team I hesitate to wade into this, but I do have a question....
It seems clear between the 2015 and 2016 games the referees jobs have gotten clearly more difficult, mostly because of the number of things they must watch and keep track of now.
As to the question: Has the GDC increased the number of referees from 2015 to 2016 to handle this change?
I'll preface my response with: You should watch 2014. Referees everywhere breathed a sigh of relief when 2015 came out. I'd like to think 2012 and 2013 were somewhat easier than 2014 (didn't ref those two so I can't compare as well). 2016 is harder still due to sightlines.
But the answer to your question is both yes and no. Many events try to have one extra ref "on staff" so they can carry on with a full on-field crew if someone has to drop out, or more usually to give refs a break every so often. Some head refs would put that "extra" ref on-field for key matches like playoffs. That "extra" position is now an official one, and with a full crew can be manned and still have a ref taking a break.
Ryan Dognaux
13-03-2016, 20:05
I don't know that I 100% agree with video replay, but people are blowing the level of difficulty WAY out of proportion here. It's 2016, there are some very affordable & simple A/V setups that are possible now.
Here's what it would take to implement a basic level of high quality video review -
- 1 volunteer to man the webcast PC. I get that volunteers are scarce but that's a bad excuse to not do something that will improve events big time.
- 1 GoPro on a tall pole. The one we used at St. Louis this past weekend (https://www.youtube.com/watch?v=BHEgVMBvI38&list=PLx4q2eQTttq1IHfaUcxhhIo8AGpaS4Nyb) used a $20 speaker stand, a 7-8 ft. tall PVC pipe, a GoPro and a cell phone charger with a USB cable to give the GoPro power all weekend.
- An HDMI input recorder like the Elegato (http://www.amazon.com/gp/product/B00MIQ40JQ?keywords=elegato&qid=1457913513&ref_=sr_1_1&sr=8-1) to allow for recording of the GoPro's view.
- Software to record the stream locally on the PC. We use XSplit because it's so easy to use, but there are other options too. Match files are saved automatically to the PC's hard drive and can be opened immediately after the match ends.
Here's what I envision the process looking like -
1. Each alliance gets one challenge flag during the elimination tournament. The challenge must be issued within 2 minutes of the match ending. Once the match has been challenged, the head referee must watch the match / incident in question.
2. Head referee coordinates with the webcast PC volunteer and pulls up the locally recorded file of the last match. This would literally take a minute to do.
3. Head referee watches the video and based on the evidence shown makes a call to replay the match or let the match stand. Similar to the NFL, the video would need to show overwhelming evidence that the match should be replayed i.e. no close calls.
My opinion - if we want FRC to be represented as a truly competitive sport then we need to present it as most sports are presented. One great example of this is how E-sports have exploded over the past few years. The coverage of online gaming tournaments is incredible and is a model FRC should look to follow. For roughly $1000 in equipment, every event could implement a basic level of coverage that would up the home viewing experience ten fold. There's no reason this same setup couldn't be used for a basic level of video replay. Will it be like the NFL? Of course not. But it has to be better than what we have today - which is nothing. FIRST could easily include the kit I described to travel with the fields from event to event and include a tip sheet on how to set it up. Anyone that can hook up their Xbox to their TV could handle setting it up.
Saying we can't do this because 'it's hard' and 'would take too much effort' is a total cop out. This is FIRST, we're supposed to be doing incredible stuff right? What happened to trying to make it loud - or is that not a thing anymore?
That's something to bring up with the RD/VC. I'd like to think that if a ref crew was missing crossings, they'd like to know that they need to keep a sharper eye out.
By the way, you weren't looking at any spy-bots by any chance, were you?
Not spy bots. I have noticed it more with the low bar. Although I have seen it happen on other defenses as well. In one match I watched today (shame on me for not writing down the match or team numbers) one team opened the sallyport from tower side then second team drove over to door and held it while first team drove completely away second bot then drove part way into sallyport as not to let door close then first bot came in and both crossed one after another. Only one cross was given.
I am not saying things aren't going to be missed, and I am quite certain that if I were a ref I would also miss things. But I would make sure at a minimum that teams that cross in auto would absolutely get the points earned.
I see first hand how much time and effort gets put into not just the building of the robot, but the effort programming puts forth to improve and get various auto programs to work well. Again I expect things to be missed during teleop with so much going on, but in auto there isn't a single reason a cross should be missed.
patar8746
13-03-2016, 20:18
I don't know that I 100% agree with video replay, but people are blowing the level of difficulty WAY out of proportion here.
Here's what it would take to implement a basic level of high quality video review -
- 1 volunteer to man the webcast PC.
- 1 GoPro on a tall pole. The one we used at St. Louis this past weekend (https://www.youtube.com/watch?v=BHEgVMBvI38&list=PLx4q2eQTttq1IHfaUcxhhIo8AGpaS4Nyb) used a $20 speaker stand, a 7-8 ft. tall PVC pipe, a GoPro and a cell phone charger with a USB cable to give the GoPro power all weekend.
- An HDMI input recorder like the Elegato (http://www.amazon.com/gp/product/B00MIQ40JQ?keywords=elegato&qid=1457913513&ref_=sr_1_1&sr=8-1) to allow for recording of the GoPro's view.
- Software to record the stream locally on the PC. We use XSplit because it's so easy to use, but there are other options too. Match files are saved automatically to the PC's hard drive and can be opened immediately after the match ends.
Here's what I envision the process looking like -
1. Each alliance gets one challenge flag during the elimination tournament. The challenge must be issued within 2 minutes of the match ending. Once the match has been challenged, the head referee must watch the match / incident in question.
2. Head referee coordinates with the webcast PC volunteer and pulls up the locally recorded file of the last match. This would literally take a minute to do.
3. Head referee watches the video and based on the evidence shown makes a call to replay the match or let the match stand. Similar to the NFL, the video would need to show overwhelming evidence that the match should be replayed i.e. no close calls.
My opinion - if we want FRC to be represented as a truly competitive sport then we need to present it as most sports are presented. One great example of this is how E-sports have exploded over the past few years. The coverage of online gaming tournaments is incredible and is a model FRC should look to follow. For roughly $1000 in equipment, every event could implement a basic level of coverage that would up the home viewing experience ten fold. There's no reason this same setup couldn't be used for a basic level of video replay. Will it be like the NFL? Of course not. But it has to be better than what we have today - which is nothing.
Exactly. Combined with the fact that almost every event is streaming and therefore already has at least one camera constantly recording matches, many more for tons of events. If these recordings could be saved we'd have far fewer issues.
Reading through some earlier replies in this thread, I'm shocked about how many missed defense crosses are being reported. My opinion stands that one incorrect match is too many, but since there have been so many more, we've got to implement a system for review, preferably before the end of this season. However, I definitely understand why FIRST would be against changing rules midseason, but FIRST should make a strong attempt at implementing a video system for the 2017 season.
(Snip)
But the answer to your question is both yes and no. Many events try to have one extra ref "on staff" so they can carry on with a full on-field crew if someone has to drop out, or more usually to give refs a break every so often. Some head refs would put that "extra" ref on-field for key matches like playoffs. That "extra" position is now an official one, and with a full crew can be manned and still have a ref taking a break.
Ok, so I take it the number of "regularly-staffed ref positions" is the same between 2015 and 2016 (and my take away from your entire post is that it may have been the same even longer). The fact there is an "extra" ref to allow for breaks who can be pulled in for eliminations is a possibly recent addition that doesn't affect qualifications.
The point I'm inclined to make is that the issue of missed calls (as opposed to flat-out errors) clearly has a "difficulty of game" component to it. Perhaps not a 1:1 correlation, but a strong causation nonetheless. Taking this further, these kinds of situations are, I'd argue to a large degree, a result of the GDC game design, either in referee difficulty, or in failing to allocate more resources (refs/video) to compensate.
Our competition is this coming week, so I have no first-hand experience to base this on, yet... But I understand it can be difficult to find an extra referee, or a volunteer to sit and work video all day for several days. I also have the utmost respect for the referees, and all the volunteers, and the time they invest and volunteer to do what can be a thankless job.
I think, however, with this complicated a game, another set of eyes would have been a wise investment. Perhaps it would be easier to fill a "Video Replay Official" position rather than an additional referee.
Ok, so I take it the number of "regularly-staffed ref positions" is the same between 2015 and 2016 (and my take away from your entire post is that it may have been the same even longer). The fact there is an "extra" ref to allow for breaks who can be pulled in for eliminations is a possibly recent addition that doesn't affect qualifications. Read my post again. In 2015, and 2014 Week 1, there were 5 refs on the field. One extra was on staff, for a full crew of 6 refs, and during playoffs there could be 6 on the field depending on the head ref. (2014 kept ramping up the ref count, though--went to 6 on-field later in the season, I was on a crew with 8 on-field at one point; CMP had 10 or so I hear.)
This year, there are 6 refs on field with one ref sitting out (ideally). There is an increase.
The point I'm inclined to make is that the issue of missed calls (as opposed to flat-out errors) clearly has a "difficulty of game" component to it. Perhaps not a 1:1 correlation, but a strong causation nonetheless. Taking this further, these kinds of situations are, I'd argue to a large degree, a result of the GDC game design, either in referee difficulty, or in failing to allocate more resources (refs/video) to compensate.
That's a possibility. Or it's a case of the refs having eyes away from the area of the missed call for some reason. As far as game design... There's one aspect of the game that SOMEBODY made a trivial mistake on that really makes life difficult for the refs; what that is is left as an exercise for the game viewer (with a hint to look at the defenses on the left side of your screen when you're watching the webcasts).
Just a small note to add to all this, but in my experience, people are either mad (to some degree) at the GDC for a boring or broken game, or they're mad at the refs for not calling every perceived penalty.
If a game is exciting, a lot of times that means there is A LOT going on. Given that most of the time you're looking at a team of 4-6 refs + head ref, that's a lot to look at when refs have to do some amount of scoring, while also calling penalties. To make matters worse, those penalties aren't always in the same place. Penalties can occur behind the driver's station, right in front of you, over in the spy box, across the field, heck even after the match!
More to the point of this thread - yes video review is used extensively for all other sports for making the right calls. Think about it this way, the average football game would last according to google:
"An average professional football game lasts 3 hours and 12 minutes, but if you tally up the time when the ball is actually in play, the action amounts to a mere 11 minutes."
11 minutes of play turns into 3 hours and 12 minutes. I'm fairly certain that if we reviewed video every time someone had a complaint or saw something that they felt a ref missed, regionals would take upwards of a week. Or you'd get about 4 matches total out of the whole thing.
I'm not saying the system is perfect and to leave it alone, I'm just saying that instead of kicking and screaming about not getting your way, propose a reasonable solution. :)
side note to the line in the original post for the thread about ruining the chances of some very deserving students -- were the students on the opposing alliance not deserving? I think everyone is deserving, but please be GP!
Just a small note to add to all this, but in my experience, people are either mad (to some degree) at the GDC for a boring or broken game, or they're mad at the refs for not calling every perceived penalty.
If a game is exciting, a lot of times that means there is A LOT going on. Given that most of the time you're looking at a team of 4-6 refs + head ref, that's a lot to look at when refs have to do some amount of scoring, while also calling penalties. To make matters worse, those penalties aren't always in the same place. Penalties can occur behind the driver's station, right in front of you, over in the spy box, across the field, heck even after the match!
More to the point of this thread - yes video review is used extensively for all other sports for making the right calls. Think about it this way, the average football game would last according to google:
"An average professional football game lasts 3 hours and 12 minutes, but if you tally up the time when the ball is actually in play, the action amounts to a mere 11 minutes."
11 minutes of play turns into 3 hours and 12 minutes. I'm fairly certain that if we reviewed video every time someone had a complaint or saw something that they felt a ref missed, regionals would take upwards of a week. Or you'd get about 4 matches total out of the whole thing.
I'm not saying the system is perfect and to leave it alone, I'm just saying that instead of kicking and screaming about not getting your way, propose a reasonable solution. :)
side note to the line in the original post for the thread about ruining the chances of some very deserving students -- were the students on the opposing alliance not deserving? I think everyone is deserving, but please be GP!
Taking 2-3 minutes of a ref's time during 7 minute field reset wont add any time.
patar8746
13-03-2016, 20:54
side note to the line in the original post for the thread about ruining the chances of some very deserving students -- were the students on the opposing alliance not deserving? I think everyone is deserving, but please be GP!
I definitely agree that the other alliance was well-qualified, being the first seed, winning the third match without error, and bringing it to within single digits with the error accounted for. However, having lost in quarterfinals as a student plenty of times, it was easy to be GP and congratulate the winning alliance because we knew we had lost fair and square. I still believe the way that second match was judged was unfair, but ultimately as of that event the scoring dispute was handled exactly according to the rules: we sent student representatives to discuss the situation, Referees listened to all we had to say but couldn't overturn the call. The situation was played by the rules so I can't do anything to change what happened to my students.
What I can do is spread awareness of the problem and open up discussion about what the best way to solve it is. Specific implementation of a replay system will be tough to figure out, but as you said, every team is deserving. To me that means every team deserves as fair an evaluation as can be given to them, and an opportunity to advance based on those fair evaluations. A video system is one of the best ways I can see to ensure fair evaluation for this game.
Game design can make things easier, like how some years (ex: Ultimate Ascent, Recycle Rush, Logomotion) almost all scoring could be calculated based on the state of the field at the end of the match, notable exceptions being autonomous bonuses and penalties. This, however, is not such a year, so I see no other way to ensure games are always judged properly.
Taking 2-3 minutes of a ref's time during 7 minute field reset wont add any time.
You mean a 4:30 field reset. It's a 7-minute cycle (officially).
But, of course, the refs don't have anything else to do like monitor traffic onto and off of the field for safety, check robot starting positions, check ball starting positions, take a quick scan for frame perimeter violations and call teams out to fix position/perimeter issues, direct team members to the question box (or answer questions there, if you happen to be the Head Referee), grab a drink, help take care of problems with the field (or point staff to them), look up the rules from the last match's tough call, circle the zebra herd to finish discussing a call...
Honestly, I'd use the off-field referee, if available and at leisure. If it's confirmed to be a missed call, he/she advises the head ref of what was missed at the next available point in time and--here's the key thing--whether it would have changed the outcome of the match. Missing one crossing in a 60-point blowout? Sorry, folks, not makin' a difference. Missing a 20-second courtyard contact violation in a 2-point match? Yep, that one's going to be reviewed by the head ref for what the action is going to be.
patar8746
13-03-2016, 20:56
Taking 2-3 minutes of a ref's time during 7 minute field reset wont add any time.
Preach.
Lil' Lavery
13-03-2016, 21:00
Time delays are a valid concern. I'll use the Orlando Regional as a benchmark for a lot of things I'll talk about here since that was my most recent experience. We were 1.5 hours behind schedule pretty much all the time. Taking up the time of normal refs with this new video system I'm proposing would not work. Yet, I see the solution as simple and I'll provide my solution.
In qualifications if there is a disputed match where a team believes there was something not scored correctly we could have an additional referee who's entire job it was is to review video to sort these problems out. You don't even have to take the time of the normal match refs to do this sort of after-match verification. The video ref could take a look at the camera view footage to determine if the appropriate call was made.
The review time for videos would be kept short if a designated video review ref could not find indisputable evidence that the call was botched then there would be no changes.
In eliminations where the match scores are in my opinion even more critical to maintaining the quality of the event you can follow a similar procedure. Give at max 5 minutes to determine the call.
So, your solution to the time delay is to add additional volunteers? With a potentially different interpretation of the rules, and who quite possibly did not see the event first-hand given that they were reviewing another match?
You're essentially adding another "key volunteer" position. Something that many events already struggle to fill.
A overhead camera like this one. https://youtu.be/PNs40CrPWUk?t=10s
A solution would have to be found. FIRST provides the field, the ref system, etc. This would have to become part of it.
It would very obvious that the camera would have to provide a high enough resolution video of the entire field to be validly able to determine calls.
So, the district system is over then? Because you're not going to find many high school gyms with the scaffolding/catwalk to support an overhead camera like that. Heck, I'd bet more than 50% of regionals would have to find new venues as well, as plenty of college gyms and convention centers wouldn't be able to support that.
Some of those calls that you just mentioned can't even be called consistently by the refs with their own eyes during the course of a match let alone an event. (i.e 15" perimeter rule, crossings, etc.) If you watched that video you'd even see that crossings were not being counted correctly even over the simple defenses.
I've watched plenty of videos. Mistakes will always be made, but mistakes are part of any sport. Not just FRC. Video replay is not a panacea to fix those officiating errors, which is my point. It's a lot of added cost and complexity, for minimal return. There's still plenty of opportunity for blown calls. Anyone who has followed the NHL this season, and its roll out of expanded video review, will attest to that. More people are complaining about the additional video review than any other officiating issue this season.
Blind spots are valid and just like in football where sometimes a call can't be made definitively even with camera angles a call would just have to be left to stand. HOWEVER, if it can be proven with a simple system like the overhead camera like I am proposing than that alone is a drastic improvement.
That "simple solution" isn't viable in the majority of FRC venues. That simple solution doesn't solve anything regarding interactions underneath bumpers. Specifically, that system cannot answer close calls in terms of CROSSINGS this year, because it cannot see underneath the frame/bumpers of a robot to know if its wheels fully cleared the defense.
It'sAScoot
13-03-2016, 21:02
"An average professional football game lasts 3 hours and 12 minutes, but if you tally up the time when the ball is actually in play, the action amounts to a mere 11 minutes."
11 minutes of play turns into 3 hours and 12 minutes. I'm fairly certain that if we reviewed video every time someone had a complaint or saw something that they felt a ref missed, regionals would take upwards of a week. Or you'd get about 4 matches total out of the whole thing.
I think you are not thinking about other factors which may influence the length of a football game. By no means is the 3:12 time due to review, but to timeouts, huddles, every 15 min of play a 30 min ish break... A regional would not lengthen if a field review took place between matches during a reset.
It'sAScoot
13-03-2016, 21:05
Say what you have to say and please turn down the passive aggressiveness. We all get that you disagree with video review but there is no reason to be snarky.
So, your solution to the time delay is to add additional volunteers? With a potentially different interpretation of the rules, and who quite possibly did not see the event first-hand given that they were reviewing another match?
You're essentially adding another "key volunteer" position. Something that many events already struggle to fill.
So, the district system is over then? Because you're not going to find many high school gyms with the scaffolding/catwalk to support an overhead camera like that. Heck, I'd bet more than 50% of regionals would have to find new venues as well, as plenty of college gyms and convention centers wouldn't be able to support that.
I've watched plenty of videos. Mistakes will always be made, but mistakes are part of any sport. Not just FRC. Video replay is not a panacea to fix those officiating errors, which is my point. It's a lot of added cost and complexity, for minimal return. There's still plenty of opportunity for blown calls. Anyone who has followed the NHL this season, and its roll out of expanded video review, will attest to that. More people are complaining about the additional video review than any other officiating issue this season.
That "simple solution" isn't viable in the majority of FRC venues. That simple solution doesn't solve anything regarding interactions underneath bumpers. Specifically, that system cannot answer close calls in terms of CROSSINGS this year, because it cannot see underneath the frame/bumpers of a robot to know if its wheels fully cleared the defense.
Im sorry, but this should be re-worded. Teams that lose good chunks of points and/or get eliminated through sometimes blatant missed calls is not "minimal return".
So, the district system is over then? Because you're not going to find many high school gyms with the scaffolding/catwalk to support an overhead camera like that. Heck, I'd bet more than 50% of regionals would have to find new venues as well, as plenty of college gyms and convention centers wouldn't be able to support that.
Ryan had a good way of combating this about 4 posts before yours:
I don't know that I 100% agree with video replay, but people are blowing the level of difficulty WAY out of proportion here. It's 2016, there are some very affordable & simple A/V setups that are possible now.
Here's what it would take to implement a basic level of high quality video review -
- 1 volunteer to man the webcast PC. I get that volunteers are scarce but that's a bad excuse to not do something that will improve events big time.
- 1 GoPro on a tall pole. The one we used at St. Louis this past weekend (https://www.youtube.com/watch?v=BHEgVMBvI38&list=PLx4q2eQTttq1IHfaUcxhhIo8AGpaS4Nyb) used a $20 speaker stand, a 7-8 ft. tall PVC pipe, a GoPro and a cell phone charger with a USB cable to give the GoPro power all weekend.
- An HDMI input recorder like the Elegato (http://www.amazon.com/gp/product/B00MIQ40JQ?keywords=elegato&qid=1457913513&ref_=sr_1_1&sr=8-1) to allow for recording of the GoPro's view.
- Software to record the stream locally on the PC. We use XSplit because it's so easy to use, but there are other options too. Match files are saved automatically to the PC's hard drive and can be opened immediately after the match ends.
Here's what I envision the process looking like -
1. Each alliance gets one challenge flag during the elimination tournament. The challenge must be issued within 2 minutes of the match ending. Once the match has been challenged, the head referee must watch the match / incident in question.
2. Head referee coordinates with the webcast PC volunteer and pulls up the locally recorded file of the last match. This would literally take a minute to do.
3. Head referee watches the video and based on the evidence shown makes a call to replay the match or let the match stand. Similar to the NFL, the video would need to show overwhelming evidence that the match should be replayed i.e. no close calls.
My opinion - if we want FRC to be represented as a truly competitive sport then we need to present it as most sports are presented. One great example of this is how E-sports have exploded over the past few years. The coverage of online gaming tournaments is incredible and is a model FRC should look to follow. For roughly $1000 in equipment, every event could implement a basic level of coverage that would up the home viewing experience ten fold. There's no reason this same setup couldn't be used for a basic level of video replay. Will it be like the NFL? Of course not. But it has to be better than what we have today - which is nothing. FIRST could easily include the kit I described to travel with the fields from event to event and include a tip sheet on how to set it up. Anyone that can hook up their Xbox to their TV could handle setting it up.
Saying we can't do this because 'it's hard' and 'would take too much effort' is a total cop out. This is FIRST, we're supposed to be doing incredible stuff right? What happened to trying to make it loud - or is that not a thing anymore?
Lil' Lavery
13-03-2016, 21:09
Im sorry, but this should be re-worded. Teams that lose good chunks of points and/or get eliminated through sometimes blatant missed calls is not "minimal return".
I don't think video review would fix nearly as many of those cases as some people think they would.
And even if they did, it's minimal return. What happens on the field is such a small part of FIRST's mission. Don't take me wrong, I'm a huge fan of watching and participating in FRC events. But the outcome is not what matters. It really isn't.
Lil' Lavery
13-03-2016, 21:10
Ryan had a good way of combating this about 4 posts before yours:
The post I was responding to specifically said a solution like Ryan's was not what he was talking about. I brought up the Fisheye GoPro in my previous response. He had a specific request for an overhead camera like Orlando's.
Kevin Leonard
13-03-2016, 21:11
So, your solution to the time delay is to add additional volunteers? With a potentially different interpretation of the rules, and who quite possibly did not see the event first-hand given that they were reviewing another match?
You're essentially adding another "key volunteer" position. Something that many events already struggle to fill.
So, the district system is over then? Because you're not going to find many high school gyms with the scaffolding/catwalk to support an overhead camera like that. Heck, I'd bet more than 50% of regionals would have to find new venues as well, as plenty of college gyms and convention centers wouldn't be able to support that.
I've watched plenty of videos. Mistakes will always be made, but mistakes are part of any sport. Not just FRC. Video replay is not a panacea to fix those officiating errors, which is my point. It's a lot of added cost and complexity, for minimal return. There's still plenty of opportunity for blown calls. Anyone who has followed the NHL this season, and its roll out of expanded video review, will attest to that. More people are complaining about the additional video review than any other officiating issue this season.
That "simple solution" isn't viable in the majority of FRC venues. That simple solution doesn't solve anything regarding interactions underneath bumpers. Specifically, that system cannot answer close calls in terms of CROSSINGS this year, because it cannot see underneath the frame/bumpers of a robot to know if its wheels fully cleared the defense.
I believe Ryan Dognaux explained a pretty low cost solution to this problem.
And all you seem to be doing right now is attempting to refute someone's position using examples of challenges to the idea. Challenges that can be worked through instead of pretending they're insurmountable. What's your alternative?
If your alternative is to pretend the problem doesn't exist, I have to disagree. Students getting turned off to a career in science and technology due to a referee's call isn't something we like to see in FIRST.
Personally, I would implement a hardware solution like the one described by Ryan and only allow for official challenges and video review in eliminations to start, with each alliance getting a "challenge" coupon of some sort.
Taking 2-3 minutes of a ref's time during 7 minute field reset wont add any time.
EricH beat me to exactly why this isn't as simple as you want to make it sound.
Refs don't just idly stand by while field reset is going on, much of the time they're running around like crazy doing 100 different things, or (heaven's forbid!) taking a half a moment to run to the bathroom so they don't delay matches :yikes:
I don't think video review would fix nearly as many of those cases as some people think they would.
And even if they did, it's minimal return. What happens on the field is such a small part of FIRST's mission. Don't take me wrong, I'm a huge fan of watching and participating in FRC events. But the outcome is not what matters. It really isn't.
Any return is more than minimal. The students on these teams deserve the best experience possible and to take that away because we think it would be too hard is just plain ludicrous. There have been ways shown and logical ideas put forth to do this at minimal costs to regionals/districts. Even if this fixes only 1 event a year, that is more than enough to add this to events.
And this is an old argument on CD but the field results is a main part of FIRST's mission, thus why it is call FIRST Robotics Competition not FIRST Robotics Event.
EricH beat me to exactly why this isn't as simple as you want to make it sound.
Refs don't just idly stand by while field reset is going on, much of the time they're running around like crazy doing 100 different things, or (heaven's forbid!) taking a half a moment to run to the bathroom so they don't delay matches :yikes:
The Head Ref is standing by for questions, arent they? How would this change that?
Sperkowsky
13-03-2016, 21:14
EDIT: OP was talking about our alliance did not even know
People who are insulting this guy need to stop.
We had a similar thing happen to us today.
We were 1st pick of Alliance 8 and extremely happy to get into the playoffs.
We went into our first match ready to go the alliance looked really good. We then won our first match Alliance 1 vs 8. We had some issues. To start we as an alliance crossing the ramparts 5 times before they finally got all of the lights off. Apparently there was a short with one of the leds. But, Match 2 is where stuff got bad. We had a decent match except for 2 big things. To start the refs missed a cheval de frise crossing which we verified happened by video causing a net loss of 25pts. They also did not foul an alliance after multiple extended pins and them trying to flip our alliance partner. After the match we went up to the question box to talk to the refs. They ackowleged they should have put heavier fouls on them and talked to the Field Supervisor I believe (Not certain on who exactly). His reply to the situation was "Cool, nothing I am going to do about it"......
We were outraged.
Third match we came out of the gates strong until defense hit. I forget the team # but essentially they tried to tip us twice along with our alliance partner twice. They had both of our robots at 45 degree angles 4 times throughout a match. They then hit our alliance partner while they were attempting to scale on the batter during the last 20 seconds causing their scaling mechanism to be misaligned and cost us the match. We lost our third QF match by 4 pts. One foul and we would have won along with the 10pts for the scale which we should have had.
Overall these "Misses" seem too obvious. We need Better Ref Training along with more refs.
Before anyone signs me up on vims know that I already inspect for FTC and plan on inspecting for FRC. I just became a Deans List finalist and I am going to volunteer at worlds. This is not a matter of me complaining just simply supporting the fact that stuff needs to change.
patar8746
13-03-2016, 21:14
So, the district system is over then? Because you're not going to find many high school gyms with the scaffolding/catwalk to support an overhead camera like that. Heck, I'd bet more than 50% of regionals would have to find new venues as well, as plenty of college gyms and convention centers wouldn't be able to support that.
Even if we can't get the ideal view of the field, it doesn't mean there's no sense in trying to get as good a view as possible. Some things can be imperfect but still very much worth implementing due to great improvements (ie having refs that occasionally make mistakes as opposed to not having refs at all)
I've watched plenty of videos. Mistakes will always be made, but mistakes are part of any sport. Not just FRC. Video replay is not a panacea to fix those officiating errors, which is my point. It's a lot of added cost and complexity, for minimal return. There's still plenty of opportunity for blown calls. Anyone who has followed the NHL this season, and its roll out of expanded video review, will attest to that. More people are complaining about the additional video review than any other officiating issue this season.
I'd argue that we aren't adding much complexity with cameras already set up for streaming, and I wouldn't call overturning matches minimal return. While the NHL has had problems, when the MLB introduced video review it undoubtedly improved the fairness of the game in a huge way despite not being called up nearly as much as other video review systems.
That "simple solution" isn't viable in the majority of FRC venues. That simple solution doesn't solve anything regarding interactions underneath bumpers. Specifically, that system cannot answer close calls in terms of CROSSINGS this year, because it cannot see underneath the frame/bumpers of a robot to know if its wheels fully cleared the defense.
Some things are easier to call than others, and with a game like this no matter how many cameras you have there is bound to be a little ambiguity. However as I've said before, video evidence should only overturn calls if it indisputably proves the call incorrect as judged by referees reviewing the video. This may lead to a lot of improvements, and it may not, but I can definitely say that it can only help to give it a shot.
Lil' Lavery
13-03-2016, 21:15
I'm going to leave it at this. Find me one example of a student who changed career paths because of a blown call in a FIRST match.
I'm willing to wager they don't exist. Because the real value of FIRST isn't what happens on the field. It's what happens in your shop, in your pits, at your outreach events, and at your meetings. The real value of FIRST is the time spent working with mentors and teammates. It's not a 135 second period of shooting a foam ball into a fake castle.
And this is an old argument on CD but the field results is a main part of FIRST's mission, thus why it is call FIRST Robotics Competition not FIRST Robotics Event.
You forgot about FIRST LEGO League and FIRST Tech Challenge. Neither of those mentions competition... or robotics...
FIRST is not just FRC. FIRST's mission is, in paraphrase, to inspire young people to pursue STEM careers through exciting mentor-based programs. (I'm too lazy to go look up the exact mission statement right now.)
Now the question becomes: Where does that inspiration come from? And what can deny it? I would argue that that comes from building a robot to compete. I won't deny that winning is fun, and losing is not, and more particularly losing because a ref misses a call isn't fun at all, but does that totally negate the inspiration of build season? (I can't answer that one. I'd even go so far as to say that it depends on the individual student(s) involved.)
Overall these "Misses" seem too obvious. We need Better Ref Training along with more refs.
In 2015, and 2014 Week 1, there were 5 refs on the field. One extra was on staff, for a full crew of 6 refs, and during playoffs there could be 6 on the field depending on the head ref. (2014 kept ramping up the ref count, though--went to 6 on-field later in the season, I was on a crew with 8 on-field at one point; CMP had 10 or so I hear.)
This needs to be brought up also. More refs are going to be needed, no matter how this goes.
The Head Ref is standing by for questions, arent they? How would this change that?
Head refs also need to take bathroom breaks, stay hydrated, sit down every once in a while. Sometimes the head refs have to review difficult calls with the other refs, discuss how something needs to be called, talk to various other volunteers (LRI, Scorekeeper, FTA) about field related issues, the list goes on.
I'm not trying to say that the refs are perfect, I'm just trying to say that the job is hard and instead of telling these volunteers that they have to be better, work with this amazing community to find a solution that can be implemented to make calls more consistent without adding unnecessary complexities
You forgot about FIRST LEGO League and FIRST Tech Challenge. Neither of those mentions competition... or robotics...
FIRST is not just FRC. FIRST's mission is, in paraphrase, to inspire young people to pursue STEM careers through exciting mentor-based programs. (I'm too lazy to go look up the exact mission statement right now.)
Now the question becomes: Where does that inspiration come from? And what can deny it? I would argue that that comes from building a robot to compete. I won't deny that winning is fun, and losing is not, and more particularly losing because a ref misses a call isn't fun at all, but does that totally negate the inspiration of build season? (I can't answer that one. I'd even go so far as to say that it depends on the individual student(s) involved.)
Sorry if I come off as blunt but we are talking about FRC, not FLL or FTC. FIRST had the opportunity to call it FIRST Robotics Challenge back when they named it but they actively decided to call it a Competition. FIRST wanted to emphasize it as a main goal for the high school level. We may not think it inspires the current students, but I can for sure tell you that the regionals/districts/championship matches all inspire middle schoolers, elementary schoolers, college students, and parents. And if it affects them then the inspiration is diminished.
Lil' Lavery
13-03-2016, 21:23
The competition is a vehicle to serve the ultimate goal. The competition itself is not the ultimate goal.
Head refs also need to take bathroom breaks, stay hydrated, sit down every once in a while. Sometimes the head refs have to review difficult calls with the other refs, discuss how something needs to be called, talk to various other volunteers (LRI, Scorekeeper, FTA) about field related issues, the list goes on.
I'm not trying to say that the refs are perfect, I'm just trying to say that the job is hard and instead of telling these volunteers that they have to be better, work with this amazing community to find a solution that can be implemented to make calls more consistent without adding unnecessary complexities
But its not adding anything, those students would come up just the same as they do now; the only difference is instead of just telling the student "sorry it is what it is" they can go over to a computer/tv screen and actually take 30 seconds to see if their claim is valid or can be definitively shown the call was made correctly. I would wager a great sum of money every head ref would have no problem doing this.
Kevin Leonard
13-03-2016, 21:26
People who are insulting this guy need to stop.
We had a similar thing happen to us today.
We were 1st pick of Alliance 8 and extremely happy to get into the playoffs.
We went into our first match ready to go the alliance looked really good. We then won our first match Alliance 1 vs 8. We had some issues. To start we as an alliance crossing the ramparts 5 times before they finally got all of the lights off. Apparently there was a short with one of the leds. But, Match 2 is where stuff got bad. We had a decent match except for 2 big things. To start the refs missed a cheval de frise crossing which we verified happened by video causing a net loss of 25pts. They also did not foul an alliance after multiple extended pins and them trying to flip our alliance partner. After the match we went up to the question box to talk to the refs. They ackowleged they should have put heavier fouls on them and talked to the Field Supervisor I believe (Not certain on who exactly). His reply to the situation was "Cool, nothing I am going to do about it"......
We were outraged.
Third match we came out of the gates strong until defense hit. I forget the team # but essentially they tried to tip us twice along with our alliance partner twice. They had both of our robots at 45 degree angles 4 times throughout a match. They then hit our alliance partner while they were attempting to scale on the batter during the last 20 seconds causing their scaling mechanism to be misaligned and cost us the match. We lost our third QF match by 4 pts. One foul and we would have won along with the 10pts for the scale which we should have had.
Overall these "Misses" seem too obvious. We need Better Ref Training along with more refs.
Before anyone signs me up on vims know that I already inspect for FTC and plan on inspecting for FRC. I just became a Deans List finalist and I am going to volunteer at worlds. This is not a matter of me complaining just simply supporting the fact that stuff needs to change.
The OP was literally talking about your alliance.
What happens on the field is such a small part of FIRST's mission. Don't take me wrong, I'm a huge fan of watching and participating in FRC events. But the outcome is not what matters. It really isn't.
The competition is a vehicle to serve the ultimate goal. The competition itself is not the ultimate goal.
Never said it was; but you are talking as if it means near nothing. Without the competition, FRC wouldnt be nearly as inspirational as it is. It isnt the ultimate goal, but it is a major part to the ultimate goal.
But its not adding anything, those students would come up just the same as they do now; the only difference is instead of just telling the student "sorry it is what it is" they can go over to a computer/tv screen and actually take 30 seconds to see if their claim is valid or can be definitively shown the call was made correctly. I would wager a great sum of money every head ref would have no problem doing this.
I would honestly wager the opposite. And the only reason I say this is because it is known that refs are not supposed to look at video evidence. If suddenly any student/team can come up and say "you called this wrong! Look!" then it will be happening after almost every match. Sometimes the students will be right, a call was missed, other times they'll be wrong because they didn't see it from a good angle, or they were mistaken about the rules.
I do like the solution that I saw somewhere in this thread stating that video evidence could be used on difficult calls in elims, but not in quals. That could be the first step forward to figuring out a system that can work, but it would still need unbiased video controlled by the field in some way.
I would honestly wager the opposite. And the only reason I say this is because it is known that refs are not supposed to look at video evidence. If suddenly any student/team can come up and say "you called this wrong! Look!" then it will be happening after almost every match. Sometimes the students will be right, a call was missed, other times they'll be wrong because they didn't see it from a good angle, or they were mistaken about the rules.
I do like the solution that I saw somewhere in this thread stating that video evidence could be used on difficult calls in elims, but not in quals. That could be the first step forward to figuring out a system that can work, but it would still need unbiased video controlled by the field in some way.
This is literally what this thread has been talking about since the beginning; we think it should be changed. And the student coming up after every match happens even now anyways.
JohnFogarty
13-03-2016, 21:33
I don't know that I 100% agree with video replay, but people are blowing the level of difficulty WAY out of proportion here. It's 2016, there are some very affordable & simple A/V setups that are possible now.
Here's what it would take to implement a basic level of high quality video review -
- 1 volunteer to man the webcast PC. I get that volunteers are scarce but that's a bad excuse to not do something that will improve events big time.
- 1 GoPro on a tall pole. The one we used at St. Louis this past weekend (https://www.youtube.com/watch?v=BHEgVMBvI38&list=PLx4q2eQTttq1IHfaUcxhhIo8AGpaS4Nyb) used a $20 speaker stand, a 7-8 ft. tall PVC pipe, a GoPro and a cell phone charger with a USB cable to give the GoPro power all weekend.
- An HDMI input recorder like the Elegato (http://www.amazon.com/gp/product/B00MIQ40JQ?keywords=elegato&qid=1457913513&ref_=sr_1_1&sr=8-1) to allow for recording of the GoPro's view.
- Software to record the stream locally on the PC. We use XSplit because it's so easy to use, but there are other options too. Match files are saved automatically to the PC's hard drive and can be opened immediately after the match ends.
Here's what I envision the process looking like -
1. Each alliance gets one challenge flag during the elimination tournament. The challenge must be issued within 2 minutes of the match ending. Once the match has been challenged, the head referee must watch the match / incident in question.
2. Head referee coordinates with the webcast PC volunteer and pulls up the locally recorded file of the last match. This would literally take a minute to do.
3. Head referee watches the video and based on the evidence shown makes a call to replay the match or let the match stand. Similar to the NFL, the video would need to show overwhelming evidence that the match should be replayed i.e. no close calls.
My opinion - if we want FRC to be represented as a truly competitive sport then we need to present it as most sports are presented. One great example of this is how E-sports have exploded over the past few years. The coverage of online gaming tournaments is incredible and is a model FRC should look to follow. For roughly $1000 in equipment, every event could implement a basic level of coverage that would up the home viewing experience ten fold. There's no reason this same setup couldn't be used for a basic level of video replay. Will it be like the NFL? Of course not. But it has to be better than what we have today - which is nothing. FIRST could easily include the kit I described to travel with the fields from event to event and include a tip sheet on how to set it up. Anyone that can hook up their Xbox to their TV could handle setting it up.
Saying we can't do this because 'it's hard' and 'would take too much effort' is a total cop out. This is FIRST, we're supposed to be doing incredible stuff right? What happened to trying to make it loud - or is that not a thing anymore?
I like what you are proposing a lot and I know exactly what you are referencing when it comes to e-sports as well.
I hope what everyone understands from what I was posting earlier in response to LL was just an idea. I do think that a key volunteer for this position should be created. I'm very adamant about that much.
As dodar has mentioned. I'm looking out for the kids, and not just mine.
Without the competition, FRC wouldnt be nearly as inspirational as it is. It isnt the ultimate goal, but it is a major part to the ultimate goal.
This^
Sperkowsky
13-03-2016, 21:33
The OP was literally talking about your alliance.
HAHA done....
I skimmed through a bit and saw he was from a different state. Thanks for telling me.
This is literally what this thread has been talking about since the beginning; we think it should be changed. And the student coming up after every match happens even now anyways.
In my experience students coming up after every single match does not "happen even now anyway". At least not until elims.
I'd be curious to see how many people who have volunteered as refs AND also previously been on a drive team would choose to solve this problem that everyone has. I say that because they have the perspective of having calls missed, and being in the position of seeing how it's possible that they can miss a call.
DonRotolo
13-03-2016, 21:43
tl;dr.
No. This is a bad idea. Ignoring the cost and logistical challenges (which are not trivial), the time consumed is far too high.
It is a game folks. Not everything is rainbows and unicorns, same as real life. It can as easily go against you as in your favor, so just let it be.
Side note: If you have never volunteered as a referee, you got nothing to say.
Lil' Lavery
13-03-2016, 21:45
Never said it was; but you are talking as if it means near nothing. Without the competition, FRC wouldnt be nearly as inspirational as it is. It isnt the ultimate goal, but it is a major part to the ultimate goal.
It isn't ANY part of the ultimate goal (culture change). It's a method to achieve that culture change. But the change in culture is not to increase emphasis on robotics competitions. The very first thing you see when you land on FIRST's homepage is "More Than Robots." The change in culture is aimed at "creating a world where science and technology are celebrated and where young people dream of becoming science and technology leaders." Nowhere in that vision statement does it mention a competition.
The competition is important, and getting calls right is important. However, I simply do not think video review really improves anything. It only helps correct a relatively small subset of calls, at the cost of additional equipment, additional volunteers, and/or additional event length. For every hypothetical about teams left uninspired at the hands of blown calls, I can point out actual teams that left before an award ceremony because the event was running longer than they had planned to stay. And even after video review, there will still be plenty of calls that cannot be corrected (think of how many "unreviewable plays" you see during a football game), or that evidence is not there to support a change in the call. In the end, I think you'll have just as many people left unsatisfied with officiating after video review as before, and you have to look no further than professional sports as evidence of that.
The competition itself is always going to drive competitors to be upset about officiating. Emotions run high, and people will naturally take the viewpoint that favors their interests. I have calls I still remember from my participation in FRC. Even with video review, that's not going to change. However, video review is certain to open all sort of new fiascos that aren't available now. Suddenly, there's incentive for teams to have their own video review staff in the stands (just like in pro sports), which creates another source of inequity between teams. The procedures for video review cannot simply be spending 30 seconds watching video, in particular if the basis of the review centers around scoring errors. By definition, for scoring errors, you almost always have to watch the duration of the match to ensure the error wasn't corrected later (anyone who's ever watched the "real time scoring" knows what I'm talking about there). Further still, does the ref simply have to review the single portion of the scoring being challenged, or rescore the ENTIRE match? What level of detail does the team have to provide in order to focus the ref's efforts?
There's a huge slate of issues here, and people are whitewashing them away. I'm quite confident that video review would introduce more inequity and issues than it would correct.
It'sAScoot
13-03-2016, 21:49
Side note: If you have never volunteered as a referee, you got nothing to say.
That's not true. Anyone can provide input. A similar thing can be said that the referees and GDC are not taking our input because they have never lost 2 matches in a row with a game changing alliance including 2 below 40 teams against first seed. They don't know how huge of a hit that was on all of our alliance members.
That's not true. Anyone can provide input. A similar thing can be said that the referees and GDC are not taking our input because they have never lost 2 matches in a row with a game changing alliance including 2 below 40 teams against first seed. They don't know how huge of a hit that was on all of our alliance members.
You realize that a growing number of volunteers used to be on FIRST teams when they were in high school?
I was on a team before, now I volunteer. I argue many of my points because I've been on both sides of the driver station wall.
Side note: If you have never volunteered as a referee, you got nothing to say.
And why not? Are students and other mentors/volunteers opinions worthless to you? Are they below a "Refs" saying?
Im not saying that they will be correct, but that doesnt mean that they should be ignored. Explain, and help them understand.
You realize that a growing number of volunteers used to be on FIRST teams when they were in high school?
I was on a team before, now I volunteer. I argue many of my points because I've been on both sides of the driver station wall.
Yep. Not the driver station wall, but when play that would have passed on one field as good is an alliance DQ with a disable on another, and you're in the audience... Ouch.
Sperkowsky
13-03-2016, 21:59
That's not true. Anyone can provide input. A similar thing can be said that the referees and GDC are not taking our input because they have never lost 2 matches in a row with a game changing alliance including 2 below 40 teams against first seed. They don't know how huge of a hit that was on all of our alliance members.
Agreed. I may be slightly biased as I was on the team they picked but, that quoted statement (not yours) is invalid.
I am 16 and all of our drivers/hps/ect are also under 18. There was only 1 adult coach on our alliance.
But we know the rules well arguably better then some refs. We are personally very invested as well. This is the first time our team has gotten into playoffs at an event since 2012. 2013,2014,and 2015 were extreme failures. This year we knew we had some sort of chance so we studied the rules well. Everyone on our team took a 13 page test and had to pass. I have gotten to the point where I can read fouls off my their prefix and, overall I knew the rules and so did our alliance partners.
I volunteer when I can but I can not volunteer as a ref yet so why say that sort of stuff. It is extremely petty.
I will probably leave this thread in fears of it getting too heated and I know now its a small conflict of interest.
I entered alliance selections not thinking we were going to be picked and was extremely happy to be an alliances first pick. I won DL and our mentor won WFFA I can not ask for much more. Coming from a team that has not won an award since 2009. Time to hang up our first blue banner.
Side note: If you have never volunteered as a referee, you got nothing to say.
I would like to volunteer, as I had planned to this year, however as my son is on a rookie team I was needed far more there than the local regional.
I would just like to say, even though emotions and feelings may be running high here, everyone, I believe, wants the same goal. A positive, inspiring, experience for the students/teams, as well as the volunteers. I'm sure refs aren't thrilled to have to handle (or not) "missed" calls throughout a competition. Be it better training, more refs, video replay, better game design, or some combination thereof, we all seem to be trying to achieve a common goal of better officiating, and a better experience for everyone involved.
At the end of the day, let's remember, these are students we're trying to inspire, and volunteers who give their time and energy freely. We owe it to them to respect that, and to respect each other.
Be it better training, more refs, video replay, better game design, or some combination thereof, we all seem to be trying to achieve a common goal of better officiating, and a better experience for everyone involved.
I just want to say that ref training this year was FAR improved from previous years. It was also far less forgiving!
I just want to say that ref training this year was FAR improved from previous years. It was also far less forgiving!
Ouch, yes. I concur. And the on-the-job part (y'all call it practice day) is pretty tough too.
XaulZan11
13-03-2016, 22:10
I haven't read this entire thread so I apologize if it was already brought up, but which offseason event will be piloting a replay system? The best way to convince FIRST to make this change would be to do it and show them it works and is practical.
Lil' Lavery
13-03-2016, 22:12
I haven't read this entire thread so I apologize if it was already brought up, but which offseason event will be piloting a replay system? The best way to convince FIRST to make this change would be to do it and show them it works and is practical.
This I agree with 100%
Sperkowsky
13-03-2016, 22:15
This I agree with 100%
I am going to break my own rule and post here again agreeing.
It is too late for a replay system this year. There could be a couple of better practices in place but I doubt it.
This is probably a thing for the big offseason events (IE Chezy Champs, Battlecry, and IRI) but we will create a system if anyone wants it.
We ran the livestream for the HHH invitational last year (A small local offseason in Long island) and would be willing to help implement a similar system if it was wanted.
Ryan Dognaux
13-03-2016, 22:17
tl;dr.
No. This is a bad idea. Ignoring the cost and logistical challenges (which are not trivial), the time consumed is far too high.
....
Side note: If you have never volunteered as a referee, you got nothing to say.
This is simply false. Go back and actually read my post (http://www.chiefdelphi.com/forums/showpost.php?p=1556392&postcount=49). If you want I will gladly post a BOM breakdown of St. Louis FIRST's setup to refute the cost claim.
As far as the claim that 'it will eat up too much time' - in my opinion, above all else, the team experience needs to be the emphasis of the event. I think this is something that is forgotten by some FRC volunteers who have been doing this for decades. It's not about 'keeping the event on schedule' to the exact minute - honestly who cares if we run 15 minutes late? These students have spent countless hours, teams are spending thousands of dollars. I think the least we can do is consider this to improve event quality.
Providing the head referee another tool to make the team experience better and improve the quality of calls at the cost of 15 additional minutes (if EVERY alliance challenged a call in this theoretical scenario) is a no brainer to me.
The argument that 'stuff happens / teams need to suck it up and move on' is weak. It's another excuse to not improve the consistency of calls at events. The solution I'm suggesting gives the head referee another tool to make the correct call - which I think they are always trying to do. The head referee job is really hard. Why wouldn't we want to help them out and give them an option?
I haven't read this entire thread so I apologize if it was already brought up, but which offseason event will be piloting a replay system? The best way to convince FIRST to make this change would be to do it and show them it works and is practical.
I volunteer the Gateway Robotics Challenge (http://gatewayroboticschallenge.com/) on October 1st.
JohnFogarty
13-03-2016, 22:23
I'll design, setup, and man a replay solution of my own design at SCRIW here in South Carolina.
Just in case you didn't know. SCRIW is played in a high school gym. Much like district events across the country are.
Kevin Leonard
13-03-2016, 22:26
I'll design, setup, and man a replay solution of my own design at SCRIW here in South Carolina.
I volunteer the Gateway Robotics Challenge (http://gatewayroboticschallenge.com/) on October 1st.
^^^Champions right here.
patar8746
13-03-2016, 22:35
Thank you all so much for keeping the discussion going, I've never been the OP for anything nearly as heavily discussed or important. Anyone keeping the subject open makes the discussion bigger and thereby attracting more views, but I really have to hand it to people volunteering to put in the effort through refereeing. I also plan on volunteering as a ref as soon as I'm in a position to (haven't been able to attend an event without teams I'm involved with for years), but those of you designing and building setups and running off season events to test them have the potential to prove to FIRST that this can work, and I know if we are wrong we will admit it, but at least then we'll know, and be much more able to accept the current system. Debate has gotten heated, but people on both sides are really making me proud to be a part of the FIRST community.
TLDR you guys are pretty great.
ATannahill
13-03-2016, 22:43
<snip>
Before anyone signs me up on vims know that I already inspect for FTC and plan on inspecting for FRC. I just became a Deans List finalist and I am going to volunteer at worlds. This is not a matter of me complaining just simply supporting the fact that stuff needs to change.
<snip>
Side note: If you have never volunteered as a referee, you got nothing to say.
I want to address this. Inspecting is nothing like reffing. I have done both of these jobs (and many more) in the past two years. Reffing puts you in arguably one of the toughest decision making positions many times more than any other position, often without having the ability for someone else (let alone your volenteer lead (LRI, Head Ref, etc.)) to see it happen. An inspector can call another inspector or the CSA over to look at a problem during inspection, refs rarely have the ability to have someone else see what a robot did in the first six seconds of tele-op. Don't get me started on how vastly different the training/tests are for the positions.
I'm not going to totally disregard anyone's claim that calls have been missed or that ref quality could be improved, but it will come with much less weight than the experience of people that have the experience of being in the role for many years.
jajabinx124
13-03-2016, 22:48
I haven't read this entire thread so I apologize if it was already brought up, but which offseason event will be piloting a replay system? The best way to convince FIRST to make this change would be to do it and show them it works and is practical.
+1
Best way to find out whether it is practical or not.
Angeliukm
13-03-2016, 22:53
So here's a situation that occurred last year:
2015 Tech Valley Regional, 20-5254-3624 had an incredible match in a make-it-or-break-it 2nd quarterfinal match, placing 4 stacks total for the first time. When the scores came up, it had our alliance with something like 119 points, and as we analyzed the score, we realized they had only credited us for 3 stacks!
We sent some students to the question box, and the referees came together and discussed that they did remember us having 4 stacks up.
From my understanding of the situation (I was not in the question box nor in the referee's discussion), they then looked at video provided by two different teams that showed 4 stacks built 5-6 high and ended up reversing the call of that match, which ended up allowing us to move on to the semifinals.
Tech Valley is a generally relaxed event, with some great referees and teams who are always gracious, and I don't think anyone involved thought what the referees did was unfair. Am I wrong?
I recognize that I'm a bit late to this thread, but I'd like to clarify the situation at TVR last year. As the team's drive team mentor for that year, I sent our student coach to the question box after we realized that there had been a scoring error and stood at the side of the field while the head ref talked to our students and then called a meeting with the other referees to discuss the match. While one of our alliance partners had a picture of the field at the end of the match with the correct number of stacks and we had a video, we neither asked the referees to look at them nor did the refs use them to make their decision. Luckily for us, the TVR refs did the right thing, both in terms of making a fair call and following the letter of the rules. Had that call gone the other way, I'm sure the aforementioned picture would have ended up on CD in a thread very similar to this one. I fully understand the sentiment behind the OP, as the desperation and lack of control our alliance felt trying to get the match rescored isn't something I'd wish on any other team. However, it seems to me that posts like these are best made a couple of days after the event once everyone has gotten some sleep and emotions aren't running as high on either side.
Ryan Dognaux
13-03-2016, 22:54
+1
Best way to find out whether it is practical or not.
Agree entirely. We will do this at the off-season event I run here in St. Louis this year. We're already doing the "hard part" - recording 1080P full field video in real-time. The easy part is opening the video file and just watching it again. I don't see a reason not to try this out.
MrTechCenter
13-03-2016, 23:06
What if reviews were only allowed for SCORING disputes and not fouls/no-calls? I was quite shocked by how many matches were just simply scored incorrectly this weekend. Most of the time, the outcome of the match wouldn't have changed in these instances, but we shouldn't be giving teams points that they didn't earn. It's just wrong. I saw a few instances of the referees crediting an alliance with three robots on the batter at the end lf the match when the third robot on that alliance was on the complete opposite side of the outer works (I'm not even kidding, this actually happened more than once). I can give tge referees some slack on foul calls because most of those are based on judgement but how can you even mess up the match score like that?
What if reviews were only allowed for SCORING disputes and not fouls/no-calls? I was quite shocked by how many matches were just simply scored incorrectly this weekend. Most of the time, the outcome of the match wouldn't have changed in these instances, but we shouldn't be giving teams points that they didn't earn. It's just wrong. I saw a few instances of the referees crediting an alliance with three robots on the batter at the end lf the match when the third robot on that alliance was on the complete opposite side of the outer works (I'm not even kidding, this actually happened more than once). I can give tge referees some slack on foul calls because most of those are based on judgement but how can you even mess up the match score like that?
But fouls are a part of scores. So they would have to be reviewable too.
rich2202
13-03-2016, 23:15
I apologize for not reading 8 pages of posts, but reviewing videos is a bad idea.
The Original Poster brought up a valid point: They potentially lost 25 points and the match.
What about other fouls that were not called against their alliance? What about crossings for the other alliance the Ref's may have missed? Did the robot really start fully in the Neutral Zone and fully cross into the Courtyard?
Is this going to be like the NFL where each team gets to identify a play for review (throw an instant replay flag)? How long will the have to identify the point in the game? What is the penalty if the original ruling is upheld?
Not all fouls are on the field itself. What about cameras in the Castle area to enforce the 6 boulder limit? No touching across the line during Autonomous?
That Team delayed the match 2 minutes 37 seconds, and should have been called for delay. Robot broke contact with the Sally Port for 0.2 seconds and should have been credited with a cross.
The only source of instant replay video is FRC Cameras only, and one field camera from above won't be sufficient for all the rules. You can't allow an opportune video by a bystander, because that would be an unfair advantage. One team could have dozens of cameras, and only show the video when it helps their case.
The Ref's make the call which rule is enforced when. It happens real-time, and that is a major consistency of the enforcement of the rules between matches.
patar8746
13-03-2016, 23:17
What if reviews were only allowed for SCORING disputes and not fouls/no-calls? I was quite shocked by how many matches were just simply scored incorrectly this weekend. Most of the time, the outcome of the match wouldn't have changed in these instances, but we shouldn't be giving teams points that they didn't earn. It's just wrong. I saw a few instances of the referees crediting an alliance with three robots on the batter at the end lf the match when the third robot on that alliance was on the complete opposite side of the outer works (I'm not even kidding, this actually happened more than once). I can give tge referees some slack on foul calls because most of those are based on judgement but how can you even mess up the match score like that?
But fouls are a part of scores. So they would have to be reviewable too.
I'd say the best way to figure this out is to establish clear communication between Offseason Event Organizers dedicated enough to implement review systems, and intentionally use different rulesets at each, preferably picking one set to be the "control" (not sure what to use, thoughts on this?) changing one aspect we're not sure about between each event, so we'll figure out what works better in each case that way. We're already experimenting with doing video review in general, why not try to learn as much as we can?
patar8746
13-03-2016, 23:24
I apologize for not reading 8 pages of posts, but
Stop right there. Please read the posts, as I'm pretty sure your concerns were addressed in previous posts. However, a few of the concerns you're citing are things none of us know how to answer because FRC has never had official video review. These are things that I think we should test at offseasons.
Can't take you offseason organizers for granted but knowing the community, I have faith that there will be people that put in more effort than I can imagine to get this done for the benefit of future students. I know I'm going to do all I can, but this is going to take a community effort.
MrTechCenter
13-03-2016, 23:28
I'd say the best way to figure this out is to establish clear communication between Offseason Event Organizers dedicated enough to implement review systems, and intentionally use different rulesets at each, preferably picking one set to be the "control" (not sure what to use, thoughts on this?) changing one aspect we're not sure about between each event, so we'll figure out what works better in each case that way. We're already experimenting with doing video review in general, why not try to learn as much as we can?
As an offseason event organizer (more of a planning comittee member now), I would totally have us be the lab rat for a match review system and I'm sure the rest of the committee would be as well. The only challenge would be finding a head referee willing to do it.
patar8746
14-03-2016, 00:03
As an offseason event organizer (more of a planning comittee member now), I would totally have us be the lab rat for a match review system and I'm sure the rest of the committee would be as well. The only challenge would be finding a head referee willing to do it.
^^^ Well we've got one (THANK YOU SIR), any more takers? The more events that are willing to test ideas, the more ideas that get to be tested/the more general examples of using video review the community has to show FIRST. Anybody know people? IRI? Chezy Champs? Suffield Shakedown? Week 0s? R2OC? Between Preseason and Offseason, Blue Alliance shows that for the 2015 season there were 61 events. If less than half of that number of events offered to try video we'd have more than enough data to work with.
rich2202
14-03-2016, 00:16
Stop right there. Please read the posts, as I'm pretty sure your concerns were addressed in previous posts.
Ok, I read through all the posts.
First of all, one camera can only solve one problem, and only partially. To be a "fair" Instant Replay (IR) system, it needs to handle most problems. That, one camera cannot solve (i.e. how many boulders are behind the castle wall).
You don't necessarily need one camera on a catwalk. There could be multiple cameras mounted on each castle wall (with at least 2 being really high up). While that would be more feasible for more events, it increases the complexity of the IR system. Don't forget, you also need to game time stamp each frame of the footage so you can know if the potential infraction happened at 21 seconds prior to the end of the game, or at 20 seconds.
That said, I think you might need a camera following each robot for some of the trickier calls where a general field camera would not provide enough detail. In Major League sports, that is easier when you really need to follow only one ball.
While limiting IR to Eliminations, and only potentially game changing calls, in close games, every call is potentially game changing (114 vs 110, when each foul is 5 points). What about Foul Cards? Do you allow IR for Foul Cards (both earned it, and it wasn't given; or it was given, but did they really earn it?). If a team has a Card, it can really affect how they play subsequent matches (more than just a game changer).
IMHO, only a Ref can be the IR initial reviewer. They have to know the game rules in detail (what constitutes a valid crossing).
I was chatting with a Ref at the KC regional this weekend. He said the Outer Works Ref was the most challenging position (the position that watches crossings). IMHO, the genesis of the problem is the Boulder Rules. Not only is the Ref watching crossings of the Robots, but they also have to watch crossings of the Boulders. A lot of Boulders illegally crossed. With the complexity of the Boulder rules, details is everything. That's when I joked on another thread that they will need 18 Ref's, one to watch each Boulder, and remember where it came from.
A few years ago, there were 2 Ref's at each panel (vs. last year when there was a Ref and a Field Reset person to help count totes). IMHO, this year, there should be 2 refs at each panel, at least for the Outer Works Position.
IMHO, increasing the number of Ref's is easier than IR. If you reduce the pressure on an individual Ref, maybe more people would be willing to be a Ref.
Remember: This is High School Competition. How many High School Football games have Instant Replay?
James1902
14-03-2016, 00:17
I don't know that I 100% agree with video replay, but people are blowing the level of difficulty WAY out of proportion here. It's 2016, there are some very affordable & simple A/V setups that are possible now.
I just wanted to clarify that the "GoPro on a stick" setup wasn't what I was considering cost or labor prohibitive. I was talking about a pure top down shot like John Fogarty was suggesting.
And, I don't want to speak for Sean Lavery, but I believe his point about that particular setup being prone to blind spots on this field still stands.
That being said, let's give it a shot at an offseason! I'm always for more and better webcasts from events and if the side effect is that we may have a way to improve team experiences, i'll take it.
I think the main disagreement we're having here is between people who think a system that will be able to catch most missed calls is too expensive so why bother, and people who think an affordable system would have flaws (blind spots) but is worth trying to improve the experience at least a little.
We should be able to come up with something in between the two that fixes some problems but still bothers everybody equally in it's own way.
Oh and this:
if we want FRC to be represented as a truly competitive sport then we need to present it as most sports are presented. One great example of this is how E-sports have exploded over the past few years. The coverage of online gaming tournaments is incredible and is a model FRC should look to follow. For roughly $1000 in equipment, every event could implement a basic level of coverage that would up the home viewing experience ten fold.
is why we started the RoboShow in the first place. I disagree that the GoPro on the stick is the best way to go about it in the long run, but it'd certainly be a start. Would love to talk about getting better coverage to more regionals.
I've been skimming through these posts (there are a lot), and after being on an alliance that lost out on a ranking point today because of one of these types of errors, I would like to hear from any referee of Stronghold so far.
Do the refs simply have too much on their plates to be watching for defense crossings, fouls, and making sure stuff generally doesn't go sour? It would be good to get input on the people actually making the calls as they will have the best insight towards this issue.
Ryan Dognaux
14-03-2016, 00:34
And, I don't want to speak for Sean Lavery, but I believe his point about that particular setup being prone to blind spots on this field still stands.
...
I disagree that the GoPro on the stick is the best way to go about it in the long run, but it'd certainly be a start.
Agree that one camera will miss some things, especially with the height of drawbridges and sally ports this year. I would think instant replay would be used for major missed items such as defense crossings, scored boulders and egregious fouls for tipping and such. Maybe the head ref should reserve the right to deny the challenge if it won't affect the outcome over the match overall.
The GoPro definitely isn't on the level of what RoboShow, GameSense, IRI Live, etc. are doing but it's a start. Crawl > walk > run.
Tristan Lall
14-03-2016, 00:53
Not much has changed in 11 years (http://www.chiefdelphi.com/forums/showthread.php?p=420888#post420888). People are still basically arguing that because it is impractical to have a perfect replay system, every replay system is detrimental or infeasible. But the point isn't to be perfect; the point is to be better.
Part of the process of implementing a replay system is establishing the rules for its use. If you're concerned that too many replays will slow down a regional, then limit the number of replays and the circumstances in which replays are available to teams. Maybe every team can have one replay in the qualifying matches, and one in the eliminations, and all the alliance partners have to spend their replays together. (http://www.chiefdelphi.com/forums/showthread.php?p=1247938#post1247938) I would postulate that much of the benefit of a replay system is to give everyone the certainty that it's there in the rare occasions where it would clearly be beneficial—but making replays artificially scarce conveys the strong implication that they should be used wisely, and would put a tight cap on the likely quantity of delay. Obviously a good technical implementation would also serve to control delay.
Maybe a given replay scheme wouldn't address the rare situation where the referees are so terrible or overworked that nearly every match is questionable. But it would start to address the far likelier scenario that a team feels bad about an entire event (and perhaps the competition in general) because one major, game-changing call was blown. And even if the replay footage was inconclusive, the act of a review will serve to placate the team. Something was done, so it can't be as easily argued that the team is being ignored. And the team and the referee have an explicit common frame of reference to guide the conversation—even if the challenge is unsuccessful, that will cut down on speculation about motive or competence. In this way, the referees appear deferential instead of capricious (without compromising their authority), and it therefore makes the competition appear more credible.
Side note: If you have never volunteered as a referee, you got nothing to say.
I can't get behind that. Referees know that they're subject to scrutiny, and should expect commentary substantiated by fact. And competitors should not be made to feel like they're not welcome to contribute their assessments of the problem—because obviously it's they, and not the referees, that are most affected by officiating errors.
I'm going to leave it at this. Find me one example of a student who changed career paths because of a blown call in a FIRST match.
This is clearly an unreasonable way of looking at the problem. What if the effect was subtle or not yet realized (e.g. the student doesn't rejoin the team the next year, and instead joins the chemistry club, which might, in a couple years, lead them to get a degree in science not engineering)? How would you propose to find such a student, given that this probably isn't a metric that anyone tracks?
patar8746
14-03-2016, 01:04
Tristan's second link contained this Gem http://www.chiefdelphi.com/forums/showpost.php?p=1248424&postcount=15 but also both threads made me realize there's a problem: People have actually been saying the same things we are on this thread for over a decade. Anyone from FIRST know what's causing the decision to be made against video review?
PayneTrain
14-03-2016, 01:18
I love the FIRST Robotics Competition. Some people say "Wil, you complain a lot about FRC, why do you do something you obviously hate?" This is a program that has entirely enraptured me and to the chagrin of other obvious high priorities, I put a lot of time into my team. I know a lot of people put a lot of time into their teams as well. Don't we all want FIRST, an organization that by its very nature must always be sprinting forward instead of dragging its feet, to constantly be improving and tweaking the status quo to make the program better?
Some disagreements lie in both parties wanting to make the organization better in fundamentally different ways (2 champs), some disagreements lie in two parties wanting some change but have differences in methods (Chairman's feedback). The worst ones, and the ones that get me really fired up, are ones where under an almost instinctive impetus to make a change, the controlling party wants to actively fight against the change (Cali districts).
I think piloting a replay system at an event like IRI and other offseasons should lead FRC down the path to pilot it throughout the season in certain regions or globally, and see if it works by 2champs.
Raise your hand if you think missed calls are not a buzzkill and something no one ever feels bad about.
Stand and be counted if you think field disconnects make FRC better.
Shout from the high horse on which you ride, proclaiming "yes, I love to be in a program that says 'No! We cannot come up with any way of improving a problem that has existed for years! There is not even a problem! Scorekeeping errors enable teams, not disable them!"
I know who some of you are, I just want to see how this line of thinking will be swiftly dragged into the mud. FIRST can be all things to all people. It can be a program that transforms people of all ages and backgrounds. It can always be better.
Briansmithtown
14-03-2016, 01:20
Heres my input on this...
Back in 2014, out last match (match 90) at NYC was.... horrible to say the least. 2 of out alliances failed in the first 3 seconds, and we were the only running robot (you would not want to hear the anger that was behind the alliance wall) During the match, we pushed an opponent bot into our ball, and the ball scored low goal... that caused mass confusion with the field crew... 14 seconds later, we finally got the ball, and we have video of it taking that long... now this is where i flip the table. No, there should not be video reviewing, and you should just accept the results. BUT if there were video reviewing, the rule should be that you get one per regional/district. Now why one? Because there is simply no time. Especially this year when the field reset crew has a lot to do. Yes, me and my team were angry with the results, and did it help make us lose? Sure it did. But over time you just learn to take one on the chin, raise your head up high, and be the better man. You can sorta turn it into a life lesson if you want, by saying its like the real world where a mistake happens, and it doest work out for you at all, but you learn to deal with it and just keep moving on. Because trust me, there have been plenty of times where I wish I could show video and get things right, but you simply just can't. Thats a FIRST lesson.
Headphones
14-03-2016, 01:23
I agree that something needs to be done to address situations where the referees miss a score by an alliance. In 2014, we were sent home from two separate events when referees missed an assist. Both times we had video evidence refuting the calls. It's ridiculous to invest hundreds of man-hours, pay thousands of dollars, and then be sent packing by a distracted referee.
VacioArconte
14-03-2016, 01:25
While it has its merits, I think that a full video replay system is only going to be yet another stressor placed on event organizers, referees and FIRST.
Initially I was thinking: Why not just change the current rule? Often, teams go to the question box with a complaint about the way a foul was called or match was scored, and they try to argue their case without video evidence. If teams that go to the question box could simply provide the replay of the exact moment of the missed call, would it be too unreasonable for the refs to use that as evidence to overturn a call?
But this only introduces the potential for more delays. The Alamo Regional was a full 2 hours behind schedule, and lord knows how much more time would be lost scrutinizing every detail of a match just played.
One other (smaller) factor: video replay of every match would mean that scores would be delayed 2-3 minutes after a match. I can't see that being well-received by the audience :)
I agree that something needs to be done to address situations where the referees miss a score by an alliance. In 2014, we were sent home from two separate events when referees missed an assist. Both times we had video evidence refuting the calls. It's ridiculous to invest hundreds of man-hours, pay thousands of dollars, and then be sent packing by a distracted referee.
Just be careful with the inflammatory language there. I agree that it's frustrating, but to call the referees distracted should be very far from the truth, and taken as a little bit of unnecessary "taunting" (for lack of a better word).
That being said: I can't say for sure, but I might be thinking about trying something involving replay at the local offseason out here. Too many variables need to fall into place, though, to make a call one way or the other at this point--including who ends up being the head ref at that event.
I'd be curious to see how many people who have volunteered as refs AND also previously been on a drive team would choose to solve this problem that everyone has. I say that because they have the perspective of having calls missed, and being in the position of seeing how it's possible that they can miss a call.
I'll bite. I was a student Coach and Driver for 3 years, mentor Coach for 2, and have reffed the last 2 years (I'll be a Coach at AZ West later this year also). As alicen and EricH noted I'm sure there are a growing number of alumni who have been on both sides.
Despite the missed calls that have happened FIRST actually has made some strides to improve referee calls. The refs do not have to watch Boulder scores at all this year (automated scoring is the best way to not miss scores even if past failures like 2006 make us wary of it) and the training has very detailed questions on difficult scenarios that could happen in match. There was also a referee update after week 1 to put more focus on Outer Works Crossings.
I think video review could help with getting calls correct, but it needs to be used in limited cases to keep events running smooth and to limit its use to fixing errors that are significant and can be reasonably evaluated. Building off of other suggestions in this thread I would propose:
Each Alliance is allowed one challenge/review in the playoffs. Unlimited challenges will result in reviewing every somewhat close match since no one wants their event/season to end. If the system works it could be expanded to each team having one challenge for all qualification matches also but not more to limit review quantities.
Head Ref leads the review process and delivers the final call, with other refs who were involved in that call assisting if necessary. The rest of the refs keep the next match getting setup correctly so that minimal time is added to normal field reset. Adding a review process should not require an increase in referee headcount given the difficulty in finding referee volunteers already.
Only match scoring errors (and penalties that lead to an automatic score) can be reviewed. This year that would mean defense crossings, challenges, scales, autonomous points, G13, G28, and boulders (though I am not sure if allowing review of an automatically scored element is reasonable to review, counting balls/disks scored in a match from a video is time consuming and more prone to error). Fouls are not reviewable as it is not easy to determine what fouls were assessed from a video and many involve a judgement call by a referee who has a better view than a camera or driver in their station will.
The score or lack of being reviewed must be significant enough to affect the outcome of the match (or an RP being awarded for games like this year). Reviewing whether a crossing was awarded in autonomous or teleop in a 40 point blowout is a waste of time and the implications on ranking tiebreakers are not significant enough to justify the resources for that.
Video evidence must be indisputable to change a call. The point is to receive credit for an obviously missed score, not debate further a close call that a referee already used their best judgement on (such as barely breaking contact with the Outer Works and Sally Port door).
I'm not sure that anything other than a full field view all match should be allowed, as it would be difficult to determine anything from a video flying all over the field following one robot. I don't have good suggestions on implementing a video system.
Overall I think that a review system would only be helpful in a small handful of circumstances, such as a missed stack last year or an obvious defense crossing this year. A video is not going to give a good view of things like bumpers being barely in or out of the Outer Works for when a crossing was not awarded as complete, and those are the more common scenarios that as a driver I would think "we should have scored for that" but as a referee I call a no score because of a different view on field.
Also I thought I'd throw in that I love Stronghold, the referee jobs are hard this year but this is one of the best games (especially for spectators) I've seen in FRC.
The OP wrote Please ...
My reply is, "No."
patar8746
14-03-2016, 01:50
...Don't we all want FIRST, an organization that by its very nature must always be sprinting forward instead of dragging its feet, to constantly be improving and tweaking the status quo to make the program better?
Some disagreements lie in both parties wanting to make the organization better in fundamentally different ways (2 champs), some disagreements lie in two parties wanting some change but have differences in methods (Chairman's feedback). The worst ones, and the ones that get me really fired up, are ones where under an almost instinctive impetus to make a change, the controlling party wants to actively fight against the change (Cali districts).
I think piloting a replay system at an event like IRI and other offseasons should lead FRC down the path to pilot it throughout the season in certain regions or globally, and see if it works by 2champs.
Raise your hand if you think missed calls are not a buzzkill and something no one ever feels bad about.
Stand and be counted if you think field disconnects make FRC better.
Shout from the high horse on which you ride, proclaiming "yes, I love to be in a program that says 'No! We cannot come up with any way of improving a problem that has existed for years! There is not even a problem! Scorekeeping errors enable teams, not disable them!"
I know who some of you are, I just want to see how this line of thinking will be swiftly dragged into the mud. FIRST can be all things to all people. It can be a program that transforms people of all ages and backgrounds. It can always be better.
Well said. We all know no one will stand for those reasons when put the right way. FIRST is about more than the robot, its about inspiring students and always becoming better, whether it be robot capability, gracious professionalism, or running of the competition itself.
Heres my input on this...
Back in 2014, out last match (match 90) at NYC was.... horrible to say the least. 2 of out alliances failed in the first 3 seconds, and we were the only running robot (you would not want to hear the anger that was behind the alliance wall) During the match, we pushed an opponent bot into our ball, and the ball scored low goal... that caused mass confusion with the field crew... 14 seconds later, we finally got the ball, and we have video of it taking that long... now this is where i flip the table. No, there should not be video reviewing, and you should just accept the results. BUT if there were video reviewing, the rule should be that you get one per regional/district. Now why one? Because there is simply no time. Especially this year when the field reset crew has a lot to do. Yes, me and my team were angry with the results, and did it help make us lose? Sure it did. But over time you just learn to take one on the chin, raise your head up high, and be the better man. You can sorta turn it into a life lesson if you want, by saying its like the real world where a mistake happens, and it doest work out for you at all, but you learn to deal with it and just keep moving on. Because trust me, there have been plenty of times where I wish I could show video and get things right, but you simply just can't. Thats a FIRST lesson.
That to me is a lesson in just giving up. Do we want our future engineers to accept defeat when they fail? No, that is not a FIRST lesson. You wished you could show a video, what if you could? We identified a problem, and are working to solve it. Being the better man is not giving up, though solid GP for not getting mad, but while part of being the better man is accepting what you can't change, the other part that is just as important is working at the things you CAN. We can't change results from past regionals but we can do our part to stop it from happening again by implementing a review system. This was never about complaining about the past. Over time, things do get better, often teams get another shot, but there are plenty of Seniors that dont get another chance. You say it can't happen but don't say why. How about you help us try? I'd love to be proven wrong, but at least give it a chance.
See earlier posts concerning the time issues, and proposals have already been made regarding limiting challenges and I fully agree, though I'd like to allow teams to continue to challenge if they get the call right. Losing your ability to challenge for being wrong would be a HUGE incentive for teams to not overuse that option, eliminating time problems resulting from too many reviews.
While it has its merits, I think that a full video replay system is only going to be yet another stressor placed on event organizers, referees and FIRST...
...The Alamo Regional was a full 2 hours behind schedule, and lord knows how much more time would be lost scrutinizing every detail of a match just played.
One other (smaller) factor: video replay of every match would mean that scores would be delayed 2-3 minutes after a match. I can't see that being well-received by the audience :)
Again, video replays would only be called upon in contested situations, and incentives can easily be provided to make sure this privilege is not abused (described above).
Also, whether you're for or against replays, I think we should all work together to test ideas at offseason events. How else can we say with any sort of confidence which option is better? Contact your local offseason coordinators to see how you can help get this together, I know I will.
patar8746
14-03-2016, 02:04
I'll bite. I was a student Coach and Driver for 3 years, mentor Coach for 2, and have reffed the last 2 years (I'll be a Coach at AZ West later this year also). As alicen and EricH noted I'm sure there are a growing number of alumni who have been on both sides...
...I think video review could help with getting calls correct, but it needs to be used in limited cases to keep events running smooth and to limit its use to fixing errors that are significant and can be reasonably evaluated. Building off of other suggestions in this thread I would propose:
Each Alliance is allowed one challenge/review in the playoffs. Unlimited challenges will result in reviewing every somewhat close match since no one wants their event/season to end. If the system works it could be expanded to each team having one challenge for all qualification matches also but not more to limit review quantities.
Head Ref leads the review process and delivers the final call, with other refs who were involved in that call assisting if necessary. The rest of the refs keep the next match getting setup correctly so that minimal time is added to normal field reset. Adding a review process should not require an increase in referee headcount given the difficulty in finding referee volunteers already.
Only match scoring errors (and penalties that lead to an automatic score) can be reviewed. This year that would mean defense crossings, challenges, scales, autonomous points, G13, G28, and boulders (though I am not sure if allowing review of an automatically scored element is reasonable to review, counting balls/disks scored in a match from a video is time consuming and more prone to error). Fouls are not reviewable as it is not easy to determine what fouls were assessed from a video and many involve a judgement call by a referee who has a better view than a camera or driver in their station will.
The score or lack of being reviewed must be significant enough to affect the outcome of the match (or an RP being awarded for games like this year). Reviewing whether a crossing was awarded in autonomous or teleop in a 40 point blowout is a waste of time and the implications on ranking tiebreakers are not significant enough to justify the resources for that.
Video evidence must be indisputable to change a call. The point is to receive credit for an obviously missed score, not debate further a close call that a referee already used their best judgement on (such as barely breaking contact with the Outer Works and Sally Port door)...
Overall I think that a review system would only be helpful in a small handful of circumstances, such as a missed stack last year or an obvious defense crossing this year...
...Also I thought I'd throw in that I love Stronghold, the referee jobs are hard this year but this is one of the best games (especially for spectators) I've seen in FRC.
^^^EXPERIENCED REF AND DRIVER
This ruleset is great, and I think this could be our base rules, minus defining where cameras are to be placed and what video feeds can be used. Some variations to try would be changing how many reviews a team gets (especially allowing them to keep reviewing if they aren't ever wrong), allowing foul reviews, implementing other penalities for baseless video requests, and whether or not to allow review in quals vs elims. Great start.
For camera placement, I'd suggest using existing stream setups to start. Variations could be placing cameras at or as close to bird's eye as possible, and not adding new cameras but allowing fans/media reps to submit video, and referee body cams?
My reply is, "No."
No chute door?
Ryan Dognaux
14-03-2016, 02:09
The OP wrote
My reply is, "No."
Super insightful, thanks Blake. Added a lot to this discussion. /s
Super insightful, thanks Blake. Added a lot to this discussion. /s
Folks you can PM me if you care to. A longer post by me would repeat what other experienced people already wrote.
Whatever faith people put in my opinions should simply be added to the "No" column, if anyone is keeping score.
Hopefully this added context explains the purpose of my earlier, succint, no-soapbox, horse-is-already-dead post.
Blake
patar8746
14-03-2016, 02:41
Folks you can PM me if you care to. A longer post by me would repeat what other experienced people already wrote.
Whatever faith people put in my opinions should simply be added to the "No" column, if anyone is keeping score.
Hopefully this added context explains the purpose of my earlier, succint, no-soapbox, horse-is-already-dead post.
Blake
Horse is both alive and dead until we open the box. We should get this tested and if we're wrong we're wrong, if not FIRST can be made better. Why not take that chance?
FlyingHedgeHog
14-03-2016, 02:42
I've been skimming through these posts (there are a lot), and after being on an alliance that lost out on a ranking point today because of one of these types of errors, I would like to hear from any referee of Stronghold so far.
Do the refs simply have too much on their plates to be watching for defense crossings, fouls, and making sure stuff generally doesn't go sour? It would be good to get input on the people actually making the calls as they will have the best insight towards this issue.
Hi there, I was a ref this last week at the PNW Wilsonville district event. Personally I think while being an outerworks ref is challenging, it's still doable. The biggest issue in my experience has been the Sallyport, which opens TOWARDS the referee, such that it is often difficult to determine if a robot began their crossing free of contact with said defense.
That said, there have been many instances at my event alone where teams were in the question box after talking about missed crossings, when many attempts were not valid in the first place. I remember several specific instances where a team either didn't begin the crossing entirely within the neutral zone, or never completed the crossing entirely into the courtyard before backing up and crossing again. While there are certainly missed calls, as with every game, I don't think there are nearly the number of uncalled crossing that teams are asserting there are. Many teams seem to be confused as to the exact definition of a crossing, and it's often hard to see exact robot positioning across the field. As such, teams should make it incredibly clear where they start and end their crossings, to help avoid any problems on both ends.
... Why not take that chance?For all the reasons other people I agree with have already explained accurately.
patar8746
14-03-2016, 09:02
For all the reasons other people I agree with have already explained accurately.
I've seen reasons it might not work, but nothing refuting trying the system over the off-season
svpracer
14-03-2016, 09:12
Being on drive team this year, the validity of this argument is quite clear... It's not a matter of if but when. We clearly crossed a defense 2 times but the refs didn't see it/ made it unclear if we crossed to we had to waste more time going back through it. This clearly could be solved if an "over head" camera was installed (use the top of either tower). But the case is when to use a video replay. I believe that if the refs themselves are unsure of a call or if a valid question is raised (post match, in the question area) that they can call a match under review and sort it out.
Ryan Dognaux
14-03-2016, 09:23
For all the reasons other people I agree with have already explained accurately.
From what I've read the main arguments against in-house video replay (no external video, that opens a can of worms) are cost, time and "this is the way it's always been done."
I look forward to proving all of these wrong when we run a pilot of this at the Gateway Robotics Challenge in October. Expect a whitepaper that addresses all of the previously mentioned concerns.
Remember everyone - tons of people said the district system would never work in FIRST. Did Michigan take no for an answer? Now districts are the main path forward for FRC. There are a million reasons to say no, I'm proposing a solution that makes it easy to say yes, let's try this.
I have had experience with refs making the wrong call, effecting the winner of the match in 2014. I was driving for 3324, the Metrobots, in the semi-finals against 16. In the first match, 16 won, but we put up a good fight. In the second match, I actually managed to defend enough against them to win that match, however, they went to the judge's table afterward, and somehow convinced them that we cheated/broke a rule, so they redid the match entirely, when video evidence would have proven we did nothing wrong. The third (second official) match, 16 won, and our season was over. I am still extremely proud of beating mentor built, swerve drive with pneumatic shooter, with a student built mecanum drive with a sketchy kicker, and it's really unfortunate to not even acknowledge officially that we won that match.
That being said, I still think video evidence should not be shown to refs. If a video proves a ref wrong, people begin to trust the ref less and less, just ensuring more drama and chaos. I know the feeling of refs making mistakes, and it is extremely frustrating and makes you hate the system at the time, but it's really the most logical thing to do. Videos don't always show what happens, such as a drawbridge blocking a view, so it can and will just bring up more controversy.
Sam_Mills
14-03-2016, 10:00
I am still extremely proud of beating mentor built, swerve drive with pneumatic shooter, with a student built mecanum drive with a sketchy kicker, and it's really unfortunate to not even acknowledge officially that we won that match.
Oh boy...
Oh boy...
What's the problem?
rzoeller
14-03-2016, 10:07
What's the problem?
My issue with it is that the post comes across as accusatory unnecessarily, and makes assumptions about another team's robot and team as a whole without any sort of validation or proof.
Jessica Boucher
14-03-2016, 10:08
Ok guys. We're taking a breather for now.
2:25pm EDT: Reopened. Please take extra care with your posts.
IronicDeadBird
14-03-2016, 14:33
Ok guys. We're taking a breather for now.
2:25pm EDT: Reopened. Please take extra care with your posts.
I've been waiting for this.
So instead of jumping to video review I think the FIRST needs to make the fields smarter overall. If they moved the burden of decisions away from Ref's I would think most people would be happy. Adding moar sensors to the field and having the field keep track and the refs verify would make for a more accurate game overall, however this is a slow, expensive process. For a video system IMO the most perfect awesome amazing thingy would be SOLOSHOT. Its a camera, a smart tripod, and a tracking device which has the tripod constantly aim the camera at the tracking device. The issue is one of these setups could cost over $500 and you would need one per robot MINIMUM. This is $3,000 dollars added onto pricing which is a lot for a system that might not even be necessary.
FIRST should always be moving to improve games and how they are run, but the last thing I want to see them do is implement an expensive solution to a problem only for it to not work or even generate more problems. Instead of jumping to the biggest change maybe we should implement more little changes to quality of life.
Big thanks for crowd control though!
What's the problem?
You absolutely DO NOT want to re-open this can of worms, I''ll leave it at that.
Jessi Kaestle
14-03-2016, 15:55
Lots of people are comparing FRC to the NFL/NHL/MLB/Nascar and saying that we should have video replay like they do. Please remember that This is a game for High School Students. As far as I am aware, High School sports do not have video replays for if/when the officials make a bad call, and if what I have heard from the stands at a high school sporting event from the parents is any indication, all the officials do is make mistakes*.
Yes I understand that bad calls can make or break a season, and it sucks. However, instead of trying to spend a lot of money and time to make this whole big procedure to make sure it doesn't happen lets instead, as mentors, use this opportunity to teach our students about making the most out of a bad situation and treating people with dignity and respect.
*I do not think that officials at high school games get everything wrong, just pointing out that that is what the parents would like you to think when a call is/isn't called for/against their child.
rich2202
14-03-2016, 16:02
Only match scoring errors (and penalties that lead to an automatic score) can be reviewed. ... Fouls are not reviewable as it is not easy to determine what fouls were assessed from a video and many involve a judgement call by a referee who has a better view than a camera or driver in their station will.
Fouls add 5 points to the other Alliance's score. Thus all fouls would need to be reviewable. Both called and uncalled fouls.
> Fouls are not reviewable as it is not easy to determine
> what fouls were assessed from a video
May not be as difficult as you think. When a Foul (or any other violation) is called, the Ref points at the offense, and waives the flag indicating the color of the Alliance causing the foul. If the flag is waived, a foul is called. If a flag is not waived (and you don't see the Ref doing anything at the panel), the no foul has been called. The tricky part is if multiple fouls were called at the same time (robot, with a boulder, crossing from the NZ to the Opponent's SP). Then you have to watch the score to see how many fouls were called. The score portion of the score board is on each video, isn't it? Time stamping is important, and that is one way to get the time stamp.
Very few things are "judgement call". It either happened, or it didn't. Either the robot broke contact with the Sally Port, or it didn't. The video may not provide convincing evidence to overturn the call.
Non-reviewable judgement call would be: Robot is likely to damage the field, so the Head Ref disabled it, if something is Strategic, or if it rises to the level of "repeated". Those are forward looking/intent judgement calls that are made real-time, so should not be reviewable.
So, what do you do if a Robot, upon review, is determined to have contacted something outside of the Field (G3), and it was not originally called by the Ref? That violation results in a Disabled. It is not a judgement call. Do you then review the rest of the match to back out all points scored by that robot?
Lil' Lavery
14-03-2016, 16:07
With regards to field automation, I would like it... provided it works. FRC games don't exactly have the greatest track record when it comes to scoring automation. The 2006 goal sensors had a slew of issues, and many (all?) events verified scores with a human scorekeeper (delays were introduced after autonomous to ensure autonomous scores were correct). The 2010 goal sensors experienced much lower volume, and were thus better performing in general, but there were still cases of sensors not working (http://www.chiefdelphi.com/forums/showpost.php?p=938943&postcount=36) and humans being used (http://www.chiefdelphi.com/forums/showpost.php?p=938958&postcount=38) to confirm scores. In 2011, there were quite (http://www.chiefdelphi.com/forums/showthread.php?t=93650)a (http://www.chiefdelphi.com/forums/showthread.php?t=94484)few (http://www.chiefdelphi.com/forums/showthread.php?t=93575)threads (http://www.chiefdelphi.com/forums/showthread.php?t=93440)covering (http://www.chiefdelphi.com/forums/showthread.php?t=93229&highlight=minibot+tower+sensor)the issues regarding the sensors on the poles registering minibots. As far as I remember, 2012's automated scoring has been easily the most accurate (ironically, so much so that thereal time scoring was removed from the screens (http://www.chiefdelphi.com/forums/showpost.php?p=1154985&postcount=18) of elimination matches to heighten excitement). 2013's weight sensors once again had issues (http://www.chiefdelphi.com/forums/showthread.php?t=114507) resulting in human confirmation of scores and untrustworthy Real Time Scoring.
A lot of that is game design, but as was pointed out earlier in the thread, making a game that is easy to score/referee often results in a game that people don't view as exciting. Excitement comes from constant action across the field, and that same constant action is what makes games more difficult to officiate. Often, this can hold true even for automated scoring (see the issues with ball jams in 2006, DOGMA penalties in 2010, and minibot sensor viability in 2011).
Not much has changed in 11 years (http://www.chiefdelphi.com/forums/showthread.php?p=420888#post420888). People are still basically arguing that because it is impractical to have a perfect replay system, every replay system is detrimental or infeasible. But the point isn't to be perfect; the point is to be better.
I'm contesting that the numerous cans of worms opened by video review aren't going to make things better. I can all but guarantee that the first case of a blind spot in video is going to result in a thread like this one, where a party objects to not being allowed to use their teams' video evidence to support overturning the call. Even worse is when two camera angles show contradictory views. Further still, instant replay does nothing to fix judgement calls, and quite frequently exacerbates the situation. I've seen plenty of open hostility towards replay officials this season in the NHL, along the lines of "if they're not going to get the call right, why bother having instant replay?" (Only less polite).
I find the dystopian vision of large teams will full video replay crews in the stands attempting to find ways to reverse the outcome of a match to influence the standings in their favor rather unsettling. And given the lengths team already go through to find an advantage, including both video scouting systems and attempting to shift the meta strategy of tournaments (http://www.chiefdelphi.com/forums/showthread.php?p=1141653) to influence rankings, I don't find this vision particularly far fetched.
This is clearly an unreasonable way of looking at the problem. What if the effect was subtle or not yet realized (e.g. the student doesn't rejoin the team the next year, and instead joins the chemistry club, which might, in a couple years, lead them to get a degree in science not engineering)? How would you propose to find such a student, given that this probably isn't a metric that anyone tracks?
If it's so clearly unreasonable, than perhaps people should stop using this example as to why FIRST needs to improve [aspect X] of the competition experience. This hypothetical has already been brought up in this thread, and many similar threads regarding perceived shortfalls of the competition aspects of FRC. So long as people cite students becoming so discouraged by a bad call that they chose not to go into a STEM field, I will continue challenging them to find me an example.
BrennanB
14-03-2016, 16:16
Please remember that This is a game for High School Students. As far as I am aware, High School sports do not have video replays for if/when the officials make a bad call, and if what I have heard from the stands at a high school sporting event from the parents is any indication, all the officials do is make mistakes*.
*I do not think that officials at high school games get everything wrong, just pointing out that that is what the parents would like you to think when a call is/isn't called for/against their child.
Worth noting is that you don't spend thousands and thousands of dollars for a few matches in any other high school sport.
Lil' Lavery
14-03-2016, 16:18
Worth noting is that you don't spend thousands and thousands of dollars for a few matches in any other high school sport.
Yes, you do.
http://www.nytimes.com/2015/01/17/your-money/rising-costs-of-youth-sports.html?_r=0
On the note of field automation, from what I have read here on cd the flags above the towers aren't even working at most events !
BrennanB
14-03-2016, 16:22
Yes, you do.
http://www.nytimes.com/2015/01/17/your-money/rising-costs-of-youth-sports.html?_r=0
However any other sport has matches longer than 2:15, or their season is more than what? 25 matches? Being generous here. Sooo 56 minutes of play time. Or like. One match of any other sport.
MrTechCenter
14-03-2016, 16:24
On the note of field automation, from what I have read here on cd the flags above the towers aren't even working at most events !
For a lot of regionals, they either didn't get the flags with the field road cases or the crew setting up the field decided not to put them up.
nrgy_blast
14-03-2016, 16:25
From what I've read the main arguments against in-house video replay (no external video, that opens a can of worms) are cost, time and "this is the way it's always been done."
I look forward to proving all of these wrong when we run a pilot of this at the Gateway Robotics Challenge in October. Expect a whitepaper that addresses all of the previously mentioned concerns.
Remember everyone - tons of people said the district system would never work in FIRST. Did Michigan take no for an answer? Now districts are the main path forward for FRC. There are a million reasons to say no, I'm proposing a solution that makes it easy to say yes, let's try this.
I greatly look forward to seeing this report! I've been on both sides of the line (judge, head judge, mentor), and I have faith we can come up with some sort of review system to help eliminate the bulk of the most impactful wrong decisions. Limit to elims only, for example.
Lil' Lavery
14-03-2016, 16:35
However any other sport has matches longer than 2:15, or their season is more than what? 25 matches? Being generous here. Sooo 56 minutes of play time. Or like. One match of any other sport.
High school football seasons are typically shorter than 10 games (not counting playoffs). While a football game lasts 2-3 hours of time in an evening and 60 minutes of game clock, there's roughly only 11 minutes (http://www.wsj.com/articles/SB10001424052748704281204575002852055561406) of actual gameplay in that span. Considering an individual athlete is (typically) only playing offense or defense (not both), cut that figure in half again. For football, we're getting into similar territories of playtime.
Plenty of other sports condense their competitions into short fire bursts. A cheerleading routine is only a few minutes long, and only done in competition a handful of times per season. Swim and track races have sprints that last under a minute (or just a matter of seconds). Shotput, javelin, long jump, and other individual track and field events are rather swift affairs for each athlete.
There are obviously some other sports with much longer match times (cross country) and higher rates of individual participation (basketball starts may play close to the entire game). But generalizing statements about playing time versus high school sports are going to have tons of exceptions. FRC certainly doesn't provide the best bang for the buck in terms of playing time, but it's more competitive than you may think.
I go back to the idea a basic video of the game that anyone can take could be used for some end of day 1 RP issues and also any elimination issues (at some point after the game except in case of elimination w/l decisions). I don't want video to slow the game play down but I think there is a reasonable option that should be used for critical calls like losses in eliminations and first/second order ranking point issues...until FRC allows review though its the way it is.
CV seemed better than SD in terms of crossing accuracy except our first match where we had to go through portcullis (which is blatantly obvious ) six times each way, I heard our SF alliance petitioned a cross in CV elims too and were denied the points as well. we talked to the head ref in Qual 1 and made them aware ..it got better. We still won and got the 3RP
Here is FRC response...
"Thank you for writing and your suggestion. However, we have made the determination that any level of video review would open the door to extensive reviews, as every team at an event could have thought they saw something that caused them to miss an RP. To determine if something was indeed missed, out of fairness, it would require a review of all concerns.
Also, it might be helpful to think of this as a typical high school sports activity, in which video reviews are usually unavailable, and it is recognized that referees are human, and will occasionally miss something. While unfortunate, this is an accepted part of the experience.
Please do not hesitate to contact us if you have any questions."
I see where they are coming from luckily from our perspective it dd not affect anything this season for us. We had great alliances each competition and a shot at it...which is all you can hope for anyhow.
Jessi Kaestle
14-03-2016, 16:38
However any other sport has matches longer than 2:15, or their season is more than what? 25 matches? Being generous here. Sooo 56 minutes of play time. Or like. One match of any other sport.
Let's analyze football, on average the pads, helmet, and uniform cost $550 per student, which comes out to ~$25,000. Each coach is paid on average 20K-25K more than they would as just a teacher. Which conservatively comes out to ~$160,000/year. So without buying any equipment, just uniforms and coach salary, schools pay ~$185,000/year for 10 games (9 if you live in Kansas).
Briansmithtown
14-03-2016, 16:41
Well said. We all know no one will stand for those reasons when put the right way. FIRST is about more than the robot, its about inspiring students and always becoming better, whether it be robot capability, gracious professionalism, or running of the competition itself.
That to me is a lesson in just giving up. Do we want our future engineers to accept defeat when they fail? No, that is not a FIRST lesson. You wished you could show a video, what if you could? We identified a problem, and are working to solve it. Being the better man is not giving up, though solid GP for not getting mad, but while part of being the better man is accepting what you can't change, the other part that is just as important is working at the things you CAN. We can't change results from past regionals but we can do our part to stop it from happening again by implementing a review system. This was never about complaining about the past. Over time, things do get better, often teams get another shot, but there are plenty of Seniors that dont get another chance. You say it can't happen but don't say why. How about you help us try? I'd love to be proven wrong, but at least give it a chance.
See earlier posts concerning the time issues, and proposals have already been made regarding limiting challenges and I fully agree, though I'd like to allow teams to continue to challenge if they get the call right. Losing your ability to challenge for being wrong would be a HUGE incentive for teams to not overuse that option, eliminating time problems resulting from too many reviews.
Again, video replays would only be called upon in contested situations, and incentives can easily be provided to make sure this privilege is not abused (described above).
Also, whether you're for or against replays, I think we should all work together to test ideas at offseason events. How else can we say with any sort of confidence which option is better? Contact your local offseason coordinators to see how you can help get this together, I know I will.
It's not a lesson of giving up, it's a lesson knowing that you've been beaten. Would I have liked their to be a rematch for my last match? Sure. Would video helped? Definitely. Definitely after an extremely rough build season where teams most likely would've folded... And it was my last year too. You're not the first team to feel like there must be video review, and you won't be the last. It's something you have to deal with. And it wouldn't matter now anyway, because it's over, and here's no going back.
Let's analyze football, on average the pads, helmet, and uniform cost $550 per student, which comes out to ~$25,000. Each coach is paid on average 20K-25K more than they would as just a teacher. Which conservatively comes out to ~$160,000/year. So without buying any equipment, just uniforms and coach salary, schools pay ~$185,000/year for 10 games (9 if you live in Kansas).
You must have a very rich school if that is true. And to my knowledge, the only football coaches that get paid are the head coaches and sometimes the assistant head coaches.
Ryan Dognaux
14-03-2016, 16:47
I'm contesting that the numerous cans of worms opened by video review aren't going to make things better.
Scenario A: The head referee makes what they thought was the correct call based on the evidence at the time, but later finds out the call was actually wrong. The call was match deciding and one alliance goes home denied of an event win.
Scenario B: The head referee makes what they think is the correct call. One alliance uses their once per elimination tournament challenge card requiring the head referee to look at the match with the captains from the alliance to explain his reason for the call. This also gives ample opportunity for the challenging alliance to provide specific rules or Q&A's to support their challenge. A match replay is issued if the head ref confirms & agree with the challenge's claims or the results stand as called if the head referee doesn't seem evidence to overturn the call.
There are definitely times where something like this could have came in handy. Look back to San Diego last year (http://www.chiefdelphi.com/forums/showthread.php?t=136476&highlight=San+Diego) where the incorrect call was made and was admitted to be wrong once the Q&A was produced. Maybe having that challenge would have given them a moment to find the Q&A and take another look at the situation?
My point is it's worth taking a look at and talking about. This could give referees another tool to help them out when making very tough calls. Why wouldn't we want to help our referees out?
How can you review something for correctness when there's no record it happened?
Folks,
FRC/VRC/FTC/VIQ matches don't proceed in little increments the way many big league sports do (USA football downs, baseball batters & pitches, Cricket ___s, Tennis volleys) .
And, they don't have long stretches of continuous action like many other big league sports (Nascar, Soccer, Basketball or Hockey).
Because of the implications of those differences, I believe that as a practical matter, untangling the spaghetti of knock-on effects that many (most?) overturned calls would create, would be a nightmare of ambiguity.
Except for the last N seconds of a match (where N is very small), everything that affects the score has an effect on how teams and alliances play the games. Neither alliance could argue that an overturned call from early in a match doesn't effect both alliances. In those many cases, I think there would be little a ref could do other than throw their hands up and initiate a replay.
I believe that Video replay successes in big-league sports are unlikely to translate into FRC successes. I predict that if video replay in FRC is successful, it will be because an FRC method is created, not because a big-league method is copied. I also believe that a successful FRC method will be a purple squirrel for most, and the great white whale of some.
I'll look forward to hearing from folks who attempt to use video replays during the upcoming off-season, to see what effect these differences create between what folks are imagining, and what is actually going to occur. I'll bet a beverage that the gulf between ambition and accomplishment is pretty wide.
Unless/until we have some experimental results to digest, I think our current posts are only making 11-12 years of "rubble" bounce.
I'm willing to stop bouncing the rubble, and wait for the results.
Blake
mhaeberli
14-03-2016, 17:25
It's not video review, but big props to 4342 did something very similar in 2014 at Chestnut Hill.
(balance snipped out).
Very cool! Great GP! And great courage!
I have to say I like Ryan Dognaux's plan to pilot this at an off-season event. I like the challenge rules (one per team, etc) that were presented in another post. I'm looking forward to seeing a series of threads on "How we did replays at XYZ event."
I'm also behind adding extra scorekeepers to the field crews to help stop some of the pain for upcoming events. It's possible that an extra set of eyes would cut the pain points way down.
In the past I've run very large VEX events. In most cases there are 2 refs watching the action on the fields. (Running two divisions at a time) Once elims start the refs double up and they pretty much get told watch a single robot's action. With a person focused on a robot it's hard to miss things. For example in VEX, the dreaded "pinning" is easier to manage since there is two refs watching (the robot doing the pinning and the robot being pinned each have a ref watching the action). Again, only for the eliminations, not possible for all matches.
I'm in a wait and see mode, happy to let Ryan and others pilot this out. Alpha test for Stronghold, beta for next year, to see how it flies.
FlyingHedgeHog
14-03-2016, 17:54
However any other sport has matches longer than 2:15, or their season is more than what? 25 matches? Being generous here. Sooo 56 minutes of play time. Or like. One match of any other sport.
If anything this seems more like an arguement in favor of implementing districts than a replay system. With districts you get 2-3 events, with 12 matches at each.
My take on this is that if FIRST were to make attempts to introduce a video review system, it would need to be piloted and slowly rolled out, like Districts, or webcasting. You can't just flip a switch and make it happen. After reading a lot of the comments in this thread, I will not be taking a dedicated stance on it since I share opinions from those on both sides of the fence. Yes, it is not OK to allow students to leave events disappointed and discouraged from their hard work because of a referee misstep - but as a soccer referee, I know what it's like to be on the other side. Learning to deal with disappointment and seeing what YOU can do to make YOUR robot better is a much better thing to get out of it than to get angry at the referees or get angry at FIRST.
If a system of video review were to be implemented, there would be several requirements and constraints that would need to be addressed.
1) It costs a lot of money to have the events record the matches. I don't know how many fields are circulating within FIRST, but a camera and recording system would need to be included with every field if this system were to be fully implemented. One way to solve the cost constraint is to get a camera company to sponsor FIRST, maybe GoPro, Samsung, or Sony, and they could work in collaboration with FIRST to supply, design, test and implement the system.
2) It delays matches and the flow of the event. I like the idea that was thrown out earlier of limiting the amount of video replays that one team can call. I would think that limiting it to 2 replays per elimination alliance is a fair number. Also to keep from delaying robot flow, you can limit the number of team members who may come to the question box to 1 or 2, such that the rest of the team can be removing the robot from the field. There's a reason you only have one timeout coupon in the eliminations. There's a reason that you only have one backup robot to call in - it just takes up too much time, and takes away from the appeal of the other teams and the visiting public, which at a certain point you must take into account.
3) There's not enough camera coverage. I don't think I've seen any FRC event this year that has less than 3 cameras covering the field on the webcast. If FIRST were to implement the system, it would need to have multiple camera angles aimed at the field - it is also possible to use the existing equipment that many events use to record their matches. These would be logistics that would need to be figured out in the pilot and testing period.
Not taking a stance, just providing solutions. If we decide against video review, what solutions will be provided to prevent referee mistakes?
Tyler Olds
14-03-2016, 19:23
We will be discussing this topic on FUN Recap tomorrow! Thank you to all who are passionate on both sides as we put together a story.
Kevin Sevcik
14-03-2016, 19:27
I'm on board with Sean that I don't think video review is going to significantly reduce complaints unless it's perfect. Whatever circumstances you confine it to, complaints will just move to the boundaries outside of them, and you'll get the new class of complaints about bad video calls. All for the low price of yet another key volunteer and an av system.
Also, I feel I should point out that whatever video replay solution you guys come up with must be implemented at every regional and district. You're planning on enshrining this in the rules, and inconsistency between events seems like just as big a complaint generator as bad calls. I think this means that any solutions that depend on streaming are non starters. Also I suggest you either need a match timer in view or a scoring overlay, or you're going to be missing at the end of auton and during the endgame.
JeremyLansing
14-03-2016, 20:03
Unless/until we have some experimental results to digest, I think our current posts are only making 11-12 years of "rubble" bounce.
I'm willing to stop bouncing the rubble, and wait for the results.
Blake
This about sums up where I'm at. I cannot wait to see the results of some offseason events that choose to implement this. Until we have some actual data, most of these arguments are speculative. Let's see how replay would work.
Ryan Dognaux
14-03-2016, 20:42
I think this means that any solutions that depend on streaming are non starters. Also I suggest you either need a match timer in view or a scoring overlay, or you're going to be missing at the end of auton and during the endgame.
The solution we're using doesn't require for you to stream. Streaming and recording are two separate functions that can happen simultaneously or you can just record the matches if you don't have internet access. This is a great point though, not every event has the internet to stream out and any solution would need to be able to operate without it.
The scoring overlay you're talking about is just the real time score graphics that the field provides. We use a cheap Monoprice Video Grabber (http://www.monoprice.com/product?p_id=5616) just to get that part to overlay on top of the GoPro's HD view. The real time graphics don't look as good as everything else, but at least you have them.
1) It costs a lot of money to have the events record the matches.
To record the events that are already being webcasted (which are a majority of the events today), it literally requires 0 more cost to record matches. You already have a PC, you're already feeding video into some piece of software, you can just record that same feed. The low cost setup we use would run you $500 if you already had a decent PC and $1000 if you didn't. For FIRST that's not a lot of money.
Folks, I'm suddenly reminded of something.
When was the first Red Card or Yellow Card issued for actions taken on the field?
Anybody who says 2010, you're right for OFFICIAL events, and wrong otherwise. It was actually IRI 2004. That's right, 5 or so seasons of testing in an offseason... And eventually, it worked.
I think, if video review is going to become a reality (note: I'd rather it didn't), that it should be thoroughly tested. Each event to try it should try something different, or slightly different from other events, and post a detailed whitepaper on setup, rules of use, actual usage, and what calls were changed. As I noted earlier, I'd be willing (if cleared by the planning committee and other interested parties) to apply it at an offseason near me, how it works TBD (I've had some ideas in the past and kind of tossed them around a bit on CD).
rich2202
14-03-2016, 21:25
Scenario A: The head referee makes what they thought was the correct call based on the evidence at the time, but later finds out the call was actually wrong. The call was match deciding and one alliance goes home denied of an event win.
Scenario B: The head referee makes what they think is the correct call. One alliance uses their once per elimination tournament challenge card requiring the head referee to look at the match with the captains from the alliance to explain his reason for the call. This also gives ample opportunity for the challenging alliance to provide specific rules or Q&A's to support their challenge. A match replay is issued if the head ref confirms & agree with the challenge's claims or the results stand as called if the head referee doesn't seem evidence to overturn the call.
There are definitely times where something like this could have came in handy. Look back to San Diego last year (http://www.chiefdelphi.com/forums/showthread.php?t=136476&highlight=San+Diego) where the incorrect call was made and was admitted to be wrong once the Q&A was produced. Maybe having that challenge would have given them a moment to find the Q&A and take another look at the situation?
My point is it's worth taking a look at and talking about. This could give referees another tool to help them out when making very tough calls. Why wouldn't we want to help our referees out?
What about scenario C: red alliance successfully challenges to get extra points to wint the match. Then blue alliance challenges a different call to get extra points to re-win the match.
What about scenario C: red alliance successfully challenges to get extra points to wint the match. Then blue alliance challenges a different call to get extra points to re-win the match.
If both are valid, and both alliances have challenges to use, then that is perfectly fine.
Wayne Doenges
14-03-2016, 21:31
The game is pretty intense for the number of refs they use.
Would it be possible to have separate refs AND scorers?
Just my $.05 :D
I think, if video review is going to become a reality, that it should be thoroughly tested. Each event to try it should try something different, or slightly different from other events, and post a detailed whitepaper on setup, rules of use, actual usage, and what calls were changed. As I noted earlier, I'd be willing (if cleared by the planning committee and other interested parties) to apply it at an offseason near me, how it works TBD (I've had some ideas in the past and kind of tossed them around a bit on CD).
Well i have some good news: This thread is all about offseason events doing just this. http://www.chiefdelphi.com/forums/showthread.php?t=145690
Kevin Sevcik
14-03-2016, 21:53
I think, if video review is going to become a reality (note: I'd rather it didn't), that it should be thoroughly tested. Each event to try it should try something different, or slightly different from other events, and post a detailed whitepaper on setup, rules of use, actual usage, and what calls were changed. As I noted earlier, I'd be willing (if cleared by the planning committee and other interested parties) to apply it at an offseason near me, how it works TBD (I've had some ideas in the past and kind of tossed them around a bit on CD).Don't forgot a survey of participants. Preferably anonymous to maximize the potential negative feedback.
Also, it probably won't be necessary, but y'all probably should go all in on getting video review at every offseason. At some point after some refinement, you're going to want to dump your system on an event that's completely unfamiliar with it to see how bad it's going to be for the first year being run by a crew with minimal training on it.
Ginger Power
14-03-2016, 22:14
Just my $.05 :D
Inflation?
Sperkowsky
14-03-2016, 23:34
Hi all!
So, I wanted to wait to post these videos and I think its calm enough to do so at this point.
Sorry for the poor video quality. I had these videos recorded so, I could go back and watch did not know they would be involved in this sort of thing.
QF-1 Was pretty clean
But QF-2 was bad. To start the big 2 things are the fact that our alliance very clearly crossed the cvr twice directly in front of 2 refs and that the red robot hit one of our blue robots during the last 20 second of the match.
If those point values were recorded correctly there would have been 35 points added to the score allowing us to win.
Here is the video for those who would like to see.
https://www.youtube.com/watch?v=qHnJSByFtV8&feature=youtu.be
QF-3 Had less clear cut issues but, you can see those aforementioned intent to tip fouls. Those pins at those angles deserve more penalty IMO. Even if there was just one more 5pt foul we would have won the match going into semis.
Here is the video for QF-3.
https://youtu.be/m2MWxJEBwpk
Some of the fouls we originally thought happened were not present but, you can clearly see enough in both matches to warrant us a win. 395 has not made CMP since 2011, we have never made it, and team 1546 has not made it since their rookie year in 2006. 395 is the only team that is even competing in a second regional. These are not elite teams who consistently make champs we are talking about here these are small teams who fight hard year after year in hopes of winning and its a shame to see all of that hard work destroyed by some bad calls. We are on the CMP waitlist and if I am not mistaken its possible to still get invited through that but regardless I am going to champs personally so see everyone there just wish my team could be there with me.
I'd like to make a quick commentary here. Just to pick on the linked videos, this is why audience videos probably shouldn't be allowed to be used for review. Zoom took time to adjust, people were in the way (down in front!), and the camera tended to follow only one robot (and, BTW, that last is fully understandable, you want to record your team's performance and I can't say I blame you there!). That being said, if the part that was needed did happen to be clear and unobstructed, it could be worth it to use that.
As far as rules go... Just remember that pins are legal up until 5 seconds (and 6' separation or chasing by the pinned robot), and tipping a robot unintentionally isn't illegal per se but a strategy to tip a robot is. Just something to think about.
I would like to hear from any referee of Stronghold so far.
No referee is going to touch this issue with a ten foot pole.
Source: Am referee. Check my signature.
Tristan Lall
15-03-2016, 04:07
I think it's worth discussing exactly what a successful replay challenge should be able to accomplish, in order to more clearly distinguish between good and bad policies.
It's probably inherently infeasible to unwind a match just because the participants' reactions to a bad call were different than they would have been had the correct call been made. That's worse than judging intent: it's almost complete subjectivity, and trying to speculate about what constitutes a correct outcome is an exercise in conjecture. Instead of expecting to unwind the match, teams should play on, with the understanding that the call on the field might be reversed on appeal. It's up to the teams to choose how they complete the match based on incomplete information, but the fact that in hindsight they should have chosen differently is of no consequence.
By contrast, a replay challenge might be warranted in situations where the nature of a discrete event is unclear, and where that event is supposed to have a defined result when determining the outcome of the match. For example, did the robot complete a game task for which points are supposed to be awarded? The resolution is to either award points or not, and that takes effect at the end of a match, no matter what else happened.
But what about games where score depends on intra-match conditions? Those fall somewhere in between, and probably need to be considered as part of the game design process. One possible resolution is to credit the points mistakenly not awarded, but not unwind the gameplay that resulted from those points not being scored at their proper time. (Is that equitable? I guess it depends on the game.) Another resolution might be to trigger a rematch for certain missed calls: perhaps they're so fundamental to the game that to miss one is to threaten the legitimacy of the event. (In fact, that's the sort of situation where a strong replay rule is beneficial: most of the audience already knows the equitable outcome, so why not give the referees the opportunity to get with the program?)
So ultimately, having a replay system doesn't need to mean that everyone will always get rematches and slow the event to a crawl. Instead, it should be tailored to the situations for which it is usually advantageous (and then applied consistently), and should be made unavailable when it would most often be detrimental.
Kevin Sevcik
15-03-2016, 09:44
By contrast, a replay challenge might be warranted in situations where the nature of a discrete event is unclear, and where that event is supposed to have a defined result when determining the outcome of the match. For example, did the robot complete a game task for which points are supposed to be awarded? The resolution is to either award points or not, and that takes effect at the end of a match, no matter what else happened.By way of example here, you have 2006, where the alliance with the highest autonomous score goes on offense first, which was considered a significant advantage. Or 2004, where knocking a ball off in auton got you your balls 40 seconds sooner.
I think the root cause of this issue is that people think there are "bad" referees that miss calls. That may be true, but fixing a bad system by introducing a new system isn't a great fix. Instead people should focus on a direct fix.
If everyone who's posted in this thread volunteered to referee, head refs would have their pick of referees. They could staff events with "good" refs instead of "bad" refs. The problem is that events can't really be selective in who they get to ref it. Some events may only have 5 applicants for the 5 ref positions. This means if one ref is known to be prone to missing calls, the event has no choice, they have to play them. If events had 10 refs volunteer for 5 positions, they would be able to select their "favorite" 5 (or 7).
So if you really want better calls, the best way to make is happen is to volunteer as a referee. Anyone who is complaining should get on VIMS right now, sign up for a week 4, 5, or 6 event and start the ref training immediately. Although it can't do anything for the event you're complaining about, you have the chance to make someone else's season much better. And then volunteer next year! You may take the spot of that really great ref, and now that they don't have to do a week 2, 3, 4, and 5 event, maybe they are willing to cover your week 1 event!
If events are able to have their pick of referees and can choose the most capable refs, and there are still problems, then I think that might be a good time to talk about video review. But getting more volunteer refs is a simpler, cheaper, more direct, and overall better approach to solving the issue of bad calls. A video review system is going to need more volunteers at the same training level as refs anyway, so why not just have more refs?
I think it's worth discussing exactly what a successful replay challenge should be able to accomplish, in order to more clearly distinguish between good and policies.
...
So ultimately, having a replay system doesn't need to mean that everyone will always get rematches and slow the event to a crawl. Instead, it should be tailored to the situations for which it is usually advantageous (and then applied consistently), and should be made unavailable when it would most often be detrimental. Tristan, I know you set out to discuss "exactly" what a successful replay challenge ... But, what you actually wrote is some high-level thoughts about how a system might work, in general.
To be successful, I think a replay system would have to describe (at the least) exactly what action the ref should take, for every single gameplay and scoring rule, whenever a challenge reverses a call involving that rule. The time when the original call occurred, and the match score when the call occurred might or might not need to be factored in for some situations.
I think there is a fairly large gap between where you ended, and what must be included in a mature, robust, system that produces consistent/predictable results.
I'm not pretending that I know the answer for this season, or for any other. It will be almost certainly be darned hard to produce a complete set of replay instructions each year. Attempting to create those instructions is an exercise left up to the readers (those readers who think reviewing video is a good idea).
IMO, the off-season experiments folks are planning for this year won't be successful unless/until they produce instructions corresponding to reach rule of this season's game. The alternative of asking the refs to fly by the seat of their pants during a tournament would not be a good idea.
Blake
Ryan Dognaux
15-03-2016, 18:44
IMO, the off-season experiments folks are planning for this year won't be successful unless/until they produce instructions corresponding to reach rule of this season's game. The alternative of asking the refs to fly by the seat of their pants during a tournament would not be a good idea.
For our testing this year we will just say if the head referee accepts the challenge request, a match replay will be issued. Keeps it really simple regardless of the play in question. May not be what would actually be implemented eventually, especially in certain cases like missing a game piece being scored, but it's what we'll try.
For our testing this year we will just say if the head referee accepts the challenge request, a match replay will be issued. Keeps it really simple regardless of the play in question. May not be what would actually be implemented eventually, especially in certain cases like missing a game piece being scored, but it's what we'll try.OK, let us know how it works out.
You probably already took into account that if I'm a team focused on winning, even if my alliance gets beaten by 100 to nothing in a match, I might decide to protest a minor call, just hoping to trigger a rematch.
Similarly, if I haven't used my protest(s) yet, I might automatically protest any/every loss that would take my alliance out of the eliminations, hoping to trigger a replay.
If I'm especially mischievous, I might purposefully create some sketchy situations, just so that I can increase my chances of having an opportunity to successfully protest a call.
If these scenarios are something you want to discourage, to avoid slow-downs, I'm not sure how you can do it.
Blake
Ryan Dognaux
15-03-2016, 19:41
OK, let us know how it works out.
You probably already took into account that if I'm a team focused on winning, even if my alliance gets beaten by 100 to nothing in a match, I might decide to protest a minor call, just hoping to trigger a rematch.
Similarly, if I haven't used my protest(s) yet, I might automatically protest any/every loss that would take my alliance out of the eliminations, hoping to trigger a replay.
If I'm especially mischievous, I might purposefully create some sketchy situations, just so that I can increase my chances of having an opportunity to successfully protest a call.
If these scenarios are something you want to discourage, to avoid slow-downs, I'm not sure how you can do it.
Blake
Just make it so the head referee has the right to take away the alliance's challenge for erroneous & repeated protests. The question box exists today and while a few teams may abuse it from time to time, I don't see all teams doing it.
Just make it so the head referee has the right to take away the alliance's challenge for erroneous & repeated protests. The question box exists today and while a few teams may abuse it from time to time, I don't see all teams doing it.
I'm not sure having every successful challenge result in a rematch is right either. If the match was a 30-100 blowout, and the challenge would net a maximum of 10 points to the 30pt team, there's no reason to replay... sort of.
Put in more "rule" type language:
After a successful challenge, the Head Referee shall take into account the match in its entirely. If, after considering the effect of a correct call on the match, the Head Referee shall determine if the correction, and likely actions of the Teams resulting thereof, could have reasonably changed the outcome of the match, the match shall be replayed. If the match is not to be replayed, the Head Referee shall adjust the score appropriately.
The wording isn't perfect, but I think you get the idea. I don't think it's enough that a score change alone would alter the winner/loser of the match, but I think the play of the match needs to be considered to trigger a reply. Eg. that 30-100 blowout, if a Robot had to attempt a 3rd crossing because the 2nd was missed by a ref and the Robot turtled as a result, there's a reasonable cause to replay the match. If it didn't turtle, but just took some time away from scoring a high-shot, no I wouldn't replay it.
smitikshah
15-03-2016, 20:11
Having been personally affected as drive coach from team 2869, I was hesitant to post as I didn't want my emotion getting in the way. However, I do like the proposed solution by many to try the video system out at any offseason comps. If anyone is planning on using this system at any events in the greater NYC region please let me know!
I would love to help out and let this emotion guide me in a positive direction in helping other teams!
By way of example here, you have 2006, where the alliance with the highest autonomous score goes on offense first, which was considered a significant advantage. Or 2004, where knocking a ball off in auton got you your balls 40 seconds sooner.
Don't remind me about 2006. Had a replay in a finals match because some field resetter didn't put the starting balls into the bins like they were supposed to, and a 10-ball auto dump got stuck on the sensor. Sensor read 15 balls (for 15 points--should have been 10). Opposing alliance drained 4 high goals for 12 points. Entire match played backwards...
I'd like to thank CVR for putting part of the problem into words... anybody in SoCal, go act on that post, please...
Folks - I think I'm right if I say that one purpose of using video is replacing fallible judgements with measurable facts. Be careful not to introduce too much heat-of-the-moment judgement back into the process you purpose/test.
Ryan Dognaux
16-03-2016, 01:24
Folks - I think I'm right if I say that one purpose of using video is replacing fallible judgements with measurable facts. Be careful not to introduce too much heat-of-the-moment judgement back into the process you purpose/test.
The purpose of video review should be to allow referees to do their jobs better - that's it. It's not there to let teams endlessly draw out a tournament. It's not there to bring up incidents that didn't really happen. I don't want to introduce opinions, I want to introduce facts. Video doesn't lie and that's a great thing.
I think everyone can agree that our head ref's have a tough job. I'd like to try and help them out by giving them an option to review the previous match. I get that it's different and hasn't been done much before (cool to read about off-seasons that have done it.) Let's try it out in the off-season. Worst that happens is we find that it doesn't support the current tournament structure and we don't pursue it any further. Best that happens is we change the FRC event structure a bit in the future with defined guidelines set by the GDC & FIRST to improve event quality overall across the program. It's all upside right now. Let's give it a go.
I'll take Blake up on that beverage bet BTW :) I have high hopes and confidence in the system we use here in St. Louis. I'd love to try and replicate it. There are plenty of people who can tell you something won't work, it's up to us to prove them that it could work - that it will work. It has to be bullet proof and simple... and I think what we have checks both of those boxes. Really excited for October (http://gatewayroboticschallenge.com/) :)
The purpose of video review should be to allow referees to do their jobs better - that's it. It's not there to let teams endlessly draw out a tournament. It's not there to bring up incidents that didn't really happen. I don't want to introduce opinions, I want to introduce facts. Video doesn't lie and that's a great thing.<Emphasis added>
This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story. Even the NFL can't find a definitive angle for every reviewed play. I don't say this to discredit the idea video review, but to remind those pushing for it that everything has its limits.
I think you and many others on this thread are well aware that video review won't resolve EVERY questionable call. Part of testing at offseason events will be determining what makes for REASONABLE review. You may otherwise find yourself down a very deep rabbit hole of "if only we had more camera angles," "if only we had higher def," "if only we had higher frame rates," "if only we allowed this/that/the other," etc.
I'm skeptical of video review, but I support those willing to invest the time/money to experiment with it. Just don't get led astray. The quest for "informative" video is a noble one. The quest for "infallible" video is quixotic.
This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story.
Heh. Reminds me of the time some friends and I were doing a video scavenger hunt--no editing allowed--and one of the items was making a half-court shot. We went for a full-court shot, and made it (on the video).
What happened, though, was that the video player was set to just play the list in order, and some quick start/stop work resulted in two videos looking close to one. Oh, and the observant folks in the audience might have noticed the leg near the rim, behind the backboard... :p (A second team member was up behind the backboard with a second basketball.)
Just to keep the robotics replays emphasis on things. The references to edited video *are* interesting, but ... remember that even unedited recordings of a robotics match can deceive an untrained eye.
Identifying, and compensating when possible, for the unavoidable distortions 2D video introduces into recording/recreating an evolving 3D (4D) scene is one real obstacle in the way of video replays.
Users would need some training to help them avoid misinterpreting what they see in the playback. That training would help keep replays in the helping-not-hindering category.
Blake
Tyler Olds
16-03-2016, 22:39
I posted in a separate thread, but if you missed it: This is what we had to say about video review on Recap (https://youtu.be/Xgy-qbfA_DY?t=3090)
Ryan Dognaux
17-03-2016, 11:00
This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story. Even the NFL can't find a definitive angle for every reviewed play. I don't say this to discredit the idea video review, but to remind those pushing for it that everything has its limits.
Yeesh - nitpicking a bit here. The system we have would make it impossible to 'cut, crop, shoot a deceptive angle' - you get what you get with it. There's no time to edit the video because you're immediately pulling it up after it was recorded. If there's not enough evidence to overturn it, the call stands. Simple.
A lot of people thinking it's not even worth trying because we can't get 100 angles and 4K resolution here. The idea is to provide the referees one more tool to help them out. It won't be perfect but it will be better than nothing, I can promise you that.
Lil' Lavery
17-03-2016, 11:32
It won't be perfect but it will be better than nothing, I can promise you that.
This is where contention is stemming from. The adverse effects may well be worse than not having a system (or investing those same resources* in alternative means of improving calls). The most debated call here is crossings. A fixed angle, full-field shot is pretty terrible at judging whether or not a team completed a crossing successfully. Those angles do a poor job at determining if a wheel (or other component) is still in contact with the outer works, if a team fully stopped touching a sallyport door, or seeing into the many blind spots around the defenses this year.
Moreover, you can't simply making scoring adjustments. When a team spends 30+ seconds of a match making sure they get a crossing that counts, that's all sorts of score impacts beyond the simple points. When a ball isn't registered properly on the tower, that has tons of impact for both alliances regarding capture strategy. Almost any circumstance where video replay is warranted also warrants a replay of a match. Even if you limit challenges to the eliminations, and only 50% of alliances use theirs, that's still 4 extra matches in the schedule. With a 7 minute cycle, at best that's 28 additional minutes added to the schedule (and likely more once you factor in 6 minute field timeouts for teams in consecutive matches).
Team experience is an important factor, but an event running on time is a part of team experience. Anyone who's been in FIRST for more than a few years has witnessed a team being called up for an award, only to find out they already left. One of the most constant complaints regarding CMP and Einstein is how late they run. On the rare occasions that events run ahead of time, you'll find plenty of posts on CD congratulating the volunteer crew for an well executed event.
*For example, one poster here is adamant about adding another key volunteer to this process. What if instead of that volunteer being a replay analyst, they were actually just another ref on the field, helping get the calls right the first time?
alephzer0
17-03-2016, 12:27
I just wanna point out that this argument has been taking place since at least 2005.
Don't believe me? Check out this thread:
http://www.chiefdelphi.com/forums/showthread.php?postid=415737#post415737
If you ask me, this is sort of a :deadhorse:
Ryan Dognaux
17-03-2016, 12:33
I just wanna point out that this argument has been taking place since at least 2005.
Don't believe me? Check out this thread:
http://www.chiefdelphi.com/forums/showthread.php?postid=415737#post415737
If you ask me, this is sort of a :deadhorse:
Technology has come a long way in 11 years. Back in 2005 I would have completely dismissed this notion. Today it's very achievable for a fraction of the cost.
There's a reason you don't see many videos from that time and if you do it looks like it was recorded using a potato.
If we're serious and FIRST is serious about delivering a product that excites young people, we need to get serious about high quality video being a staple at all of our events.
alephzer0
17-03-2016, 12:35
Technology has come a long way in 11 years. Back in 2005 I would have completely dismissed this notion. Today it's very achievable for a fraction of the cost.
There's a reason you don't see many videos from that time and if you do it looks like it was recorded using a potato.
True, but I was just pointing out that this isn't a new argument.
Ryan Dognaux
17-03-2016, 12:38
True, but I was just pointing out that this isn't a new argument.
Agree - but it's not beating a dead horse. We should revisit topics like this from time to time as things become cheaper, easier and more obtainable.
alephzer0
17-03-2016, 12:45
Agree - but it's not beating a dead horse. We should revisit topics like this from time to time as things become cheaper, easier and more obtainable.
Honestly I just wanted an excuse to use that emoji :p
Technology has come a long way in 11 years. Back in 2005 I would have completely dismissed this notion. Today it's very achievable for a fraction of the cost.
There's a reason you don't see many videos from that time and if you do it looks like it was recorded using a potato.Yes - Technology has changed - but trigonometry, basic game theory math, the number of hours in a day, and human nature haven't.
Few arguments against using video replays rest on technological foundations (some do, but the majority do not).
I look at this as one of those circumstances where "technology" can either be the final puzzle piece that enables great things, or be the shiny trickster that seduces users into great folly. We already know I lean toward the trickster side of the spectrum for this topic.
In one important sense, if technology is truly no longer a problem, it shouldn't appear much (except as minor footnotes) in this thread. Instead our conversation should be focusing on the rest of the strong arguments that have existed since 2005.
Let's hope enough off-season experiments produce enough repeatable, hard-evidence measurements against important criteria (not collections of fuzzy anecdotes from a few inconsistent, small sample-size, experiences) to move the needle in one direction or the other.
Blake
PS: Clear (1976) evidence that imagery doesn't lie. ;)
What are Dave Lavery's rovers really doing up there??? (http://science.nasa.gov/science-news/science-at-nasa/2001/ast24may_1/)
:eek: :rolleyes: :D
Standing this topic on its head ...
What if inexpensive video (and image processing) became the foundation for both most of the scoring and some of the penalties, in future FRC games?
I think it is technically feasible (I'm being seduced by the "technology trickster") to design interesting FRC games, and to equip FRC fields, to permit/do this.
Cameras would track the state of objects, the location/movement(s) of some objects, and the locations of robots.
Humans would still be needed to enforce some rules (certainly, some penalties), and/or to step in if the field equipment/detectors misfire in some way.
After 2-3 years of behind the scenes R&D, would introducing games designed with this approach in mind, along with introducing fields that attempt to implement those games, be a net improvement or a lead balloon?
Blake
PS: Whatever the answers/opinions, it will be interesting to ask ourselves, "Why?", and then bring the answers to bear on this thread's original topic.
Ryan Dognaux
17-03-2016, 17:44
I'm bowing out of this discussion for now. There's a lot of doubt, negativity and misinformation about what video review could be. These opinions won't change until we see some positive examples of it being used during the off-season. I absolutely cannot wait to implement it and see what happens (hopefully proving the naysayers wrong.)
Remember students - just because someone tells you something won't work doesn't mean you shouldn't go and find that out for yourself, especially if you think you can make it work. Often times the people telling you it won't work have biased opinions from bad experiences of their own or just don't know what they're talking about.
I can see an adjunct competition where you are given the game and you need to design the scoring system for it. You get to deploy your scoring system with others at the event. Winner is the one that matches the humans the most at an event.
Side benefit is that in case of a question about the score they can go to the computer scores and see what they think.
Ryan, you and the others are on the right track.
Remember students - just because someone tells you something won't work doesn't mean you shouldn't go and find that out for yourself, especially if you think you can make it work. Often times the people telling you it won't work have biased opinions from bad experiences of their own or just don't know what they're talking about.
This is CD gold. For the last decade I've often been surprised and impressed with what roboteers can do. I never discount a motivated roboteers ability.
I for one welcome my new computer scoring overlords. :rolleyes:
I can see an adjunct competition where you are given the game and you need to design the scoring system for it. You get to deploy your scoring system with others at the event. Winner is the one that matches the humans the most at an event.
Side benefit is that in case of a question about the score they can go to the computer scores and see what they think.
That is NOT a bad idea at all. Could be an interesting offseason competition: Given game X, design a scoring system with constraints A, B, and C--most likely portability (including ease of assembly into the field), durability (including fault tolerance/"idiot-proofing"), and cost. Must match or beat humans.
Is anyone in the mood to put to together an accurate enough model of this year's field (or a previous one), then add some similar fidelity robot and game piece models, and then explore the accuracy and obstruction (and misinterpretation of camera imagery) topics connected to using replays to answer questions?
I'm betting someone among us has the models, motivations, domain-knowledge, and free-time necessary.
Any takers?
Blake
Folks,
This interesting article can probably be used to bolster the positions pn all sides of this topic. It has some info about how sensors and computing are (and aren't) being used to supplement/replace human refereeing in some big league sports.
I enjoyed reading it.
Blake
Folks,
This interesting article can probably be used to bolster the positions pn all sides of this topic. It has some info about how sensors and computing are (and aren't) being used to supplement/replace human refereeing in some big league sports.
I enjoyed reading it.
Blake
Did you forget a link?
That being said, for a number of reasons automated scoring can be very useful. Ball scoring has been almost completely automated on several occasions ('10, '12, '16--'06 was automated but required manual backup due to system design). It's GREAT to not have to worry about scoring balls with the barrage that most alliances can unleash. OTOH, the DOGMA penalty in '10, which was automatic, was kind of a nuisance--and on at least one occasion, a team did exactly what they were supposed to and still got a penalty series from that (ball missed the counter on the way down). '08 had automatic robot lap counting, if that actually worked--I don't recall hearing anything either way on that.
But there are still a number of cases that automation would make particularly "interesting". Here's a couple related ones: Could a sensor detect whether a boulder crossed legally this year, under the case where a robot did not finish (or start) a Crossing but the boulder went across? I think so... but I would argue that that would have to be a very good sensor system to detect that, particularly live on the field. The question is, is the added cost* worth it for a system that would only need to be used some small portion of the time, for a non-profit competition system? And that I would say that it depends.
Many things can be automated. Some things cannot be automated. I happen to like the MLB replay model where it's basically a second set of eyes, detached from the game in question, looking at the various viewpoints, and not everything is actually reviewable.
*Not necessarily money--time, training, etc are all factors in the cost.
Hitchhiker 42
22-08-2016, 21:44
Did you forget a link?
But there are still a number of cases that automation would make particularly "interesting". Here's a couple related ones: Could a sensor detect whether a boulder crossed legally this year, under the case where a robot did not finish (or start) a Crossing but the boulder went across? I think so... but I would argue that that would have to be a very good sensor system to detect that, particularly live on the field.
Just as a design thing (not really related, but I'm interested), it'd probably be easier to put a location sensor within each bolder. Location can be accelerometer/gyro combo with a tiny controller. Of course, I'm not a fan of making those boulders even more expensive...
Doh! Doh! and Double-Doh!
This is the link I forgot to include earlier today.
The missing link (https://www.hpematter.com/sports-tech-issue/5-reasons-why-robots-are-the-future-of-sports-officiating)
Blake
Video review made it's debut in Olympics Beach Volleyball this year with specific rules on what may be reviewed, such as line violations and in/out calls, but not judgement calls such as net violations and lifts, and additionally there's only a set amount of time after a play (5 seconds) in which they're allowed to request a challenge.
I think this is a good example of how to implement video review in the context of FRC. Not everything should be allowed to be challenged, only certain black or white calls, such as crossing a zone in 2014. A time limit on requesting a challenge should be implemented.
I think this is a good example of how to implement video review in the context of FRC. Not everything should be allowed to be challenged, only certain black or white calls, such as crossing a zone in 2014. A time limit on requesting a challenge should be implemented.
And that's a reasonable implementation, too. The game changes year-to-year, but there's almost always some hard-and-fast "do this like this" area that can/should be reviewed.
If I were to implement replay, judgement calls would be out of the question for team-implemented review. (I'd give the head refs the option to use it for judgment calls, if they thought it necessary, and I'd guess that most head refs wouldn't. The bigger the judgement call, the more refs are in the huddle.) To review a call, it'd have to be something that could be seen and quantified from the video, and isn't covered by any automated systems or placement at the end of the match. (End-of-match placement, you can get 2-3 opinions in a matter of seconds.) Sounds like auto line violations, secret passage calls, and defense crossings for 2016; zone entry in 2014; contact around the protected zones in 2012 and 2013.
Folks,
Now that the summer of 2016 is almost over, does anyone want to report any new (previously unreported) results of experimenting with video replay ("hard" data or anecdotes) during off-season tournaments/scrimmages?
What about any non-tournament / non-scrimmage experiments to determine numbers of cameras needed for good results (for Stronghold), or best camera locations (for Stronghold), or minimum useful frame rates (for fast robots and game pieces), or estimated time needed to accurately review each/any protested ruling (for Stronghold (for each rule and/or scoring event)) or ...
Blake
Hitchhiker 42
23-08-2016, 22:49
Again, as a theoretical - no hard data here - but there are easier ways than cameras. Requiring each robot to carry a small sensor packet that can tell where the robot is at any point in time would have fixed the crossings issue this year. Maybe in the future, it could even be built into the RIO.
Again, as a theoretical - no hard data here - but there are easier ways than cameras. Requiring each robot to carry a small sensor packet that can tell where the robot is at any point in time would have fixed the crossings issue this year. Maybe in the future, it could even be built into the RIO.
Actually, it wouldn't have totally fixed that issue. You'd have to completely redefine Crossing for that to work automatically--Crossing is determined by bumpers, and making a foolproof system that could tell that the bumpers were clear from a sensor packet (from whom? team/FIRST/other?) that would need to be inside the bumpers would require a lot of complexity. Need to know size + shape + location + orientation of the robot with bumpers, the last two in three dimensions each. It'd be a lot easier for Scales--mount a device or two at the bottom of your bumpers, if they're all above a sensor mounted in the castle wall at the end of a match you're good. (Reasons to deny a Crossing included starting in contact with a defense--see also bad Sally Port crossing--as well as bumpers not finishing clearing and bumpers going over/above the shields.)
Now, they did do something like that in '08, where each robot carried a small IR beacon (or something like that--it was field-supplied with another piece of required equipment) mounted in a known area of the robot--pass under a receiver, get credit. But I don't recall how well that worked.
Can it be done, yes. But to be honest, the second-best tool we have right now is actually cameras, properly pointed, if the rules allow it. The best tool? The 6 pairs of eyes in the heads of the referees. Without those, there is not a good way to tell if the electronics are acting up and not counting stuff properly. And, of course, we sure hope the game is designed so it's a little easier on the refs...
ATannahill
24-08-2016, 06:59
Now, they did do something like that in '08, where each robot carried a small IR beacon (or something like that--it was field-supplied with another piece of required equipment) mounted in a known area of the robot--pass under a receiver, get credit. But I don't recall how well that worked.
As I remember, poorly. I do not believe they were used at all events.
Chris is me
24-08-2016, 09:10
Folks,
Now that the summer of 2016 is almost over, does anyone want to report any new (previously unreported) results of experimenting with video replay ("hard" data or anecdotes) during off-season tournaments/scrimmages?
What about any non-tournament / non-scrimmage experiments to determine numbers of cameras needed for good results (for Stronghold), or best camera locations (for Stronghold), or minimum useful frame rates (for fast robots and game pieces), or estimated time needed to accurately review each/any protested ruling (for Stronghold (for each rule and/or scoring event)) or ...
Blake
I would ask again in the fall. Most offseasons happen in the spring or fall; relatively few in the summer.
While I think nobody can expect perfect camera coverage of the field, even a single full field view results in some things being reviewable. If someone tries to get a review and there isn't clear video, the outcome doesn't change and the situation is no worse than having no video review at all.
I feel like a "it has to be perfect or it isn't worth doing at all" attitude here will really hold people back from experimenting with this.
XaulZan11
24-08-2016, 13:06
. If someone tries to get a review and there isn't clear video, the outcome doesn't change and the situation is no worse than having no video review at all.
.
Other than the additional time wasted.
Hitchhiker 42
24-08-2016, 13:11
I feel like a "it has to be perfect or it isn't worth doing at all" attitude here will really hold people back from experimenting with this.
Nothing's holding off-seasons from experimenting with this. The real problem lies with real competitions, where we should be wary in implementing something that can make them even longer without having a clear and proven positive effect (and a streamlined process for doing so).
I would ask again in the fall. Most offseasons happen in the spring or fall; relatively few in the summer.Sure - I'm aware, I thought a few were going to do some trials this summer, so I asked. I or someone should give this thread another poke in late Fall.
I feel like a "it has to be perfect or it isn't worth doing at all" attitude here will really hold people back from experimenting with this.I certainly hope not.
Conducting experiments is how you quantify "better, but not perfect". When a discussion reaches that stage, it can leave endless cycles of hand-waving behind.
Imperfect is fine, if the imperfections are known, and well-understood; and if the decision-makers agree that the imperfect system is the one they want (because it's better (in some hopefully well-defined sense) than the system it replaces).
Imperfect, and poorly understood, when/if it were implemented, ... Well, that would be a different kettle of fish.
Well designed experiments are exactly what this topic needs.
Blake
Sperkowsky
24-08-2016, 14:17
Other than the additional time wasted.
During the time the video review is happening. Pre match things can be going on. Ie connecting robots, and reseting the field. I can't see very much time wasted at all unless the system to replay matches is not working at all.
Besides I would rather spend 30 minutes longer at ever Frc event then see anyone leave an event how MY team did this year. (go look at the start of the thread my team was on that alliance). If there was video review we would have atleast been semi finalists. But, being our alliance set a higher score then that semi finals alliance there was a good chance if video review existed we could have been finalists. I have thought through every scenario I could have done to make the outcome different. Design choices, driving choices, repair choices, ect. But the bottom line is our season ended because of a bad ref call. I don't fault the refs. This game was ridiculously hard to referee. But, the fact of the situation is still there.
Video review would help the teams, and help the refs. It doesn't cost that much to implement and maybe it can be integrated into an frc dedicated livestream setup. Which would be a huge win.
...
Video review would help the teams, and help the refs. It doesn't cost that much to implement and maybe it can be integrated into an frc dedicated livestream setup. Which would be a huge win.In what well-designed experiments has this been qauntified?
... leave endless cycles of hand-waving behind.
...
Well designed experiments are exactly what this topic needs.
Without proof that the net effect (the entire gestalt) will be better (in some well-defined sense) outcomes, both you and I are just announcing our opinions while figuratively waving our hands in the air.
Blake
Hitchhiker 42
24-08-2016, 14:50
I think that one of the biggest things we always need to keep in mind is the goal of FIRST and FRC. What are we really trying to achieve? Inspiration.
The next question we have to ask is "Is it really worth it to implement video review systems, or will the goal of inspiration* be just as attainable even with the few (and far between) errors of the ref?"
*Yes I understand that having teams' hard work recognized plays a part in this. That's why this is a question, not a statement.
Tyler Olds
24-08-2016, 14:54
"Is it really worth it to implement video review systems, or will the goal of inspiration* be just as attainable even with the few (and far between) errors of the ref?"
The term inspiration is very subjective and to me having a higher percentage of calls correct is much more inspiring.
I think that one of the biggest things we always need to keep in mind is the goal of FIRST and FRC. What are we really trying to achieve? Inspiration.
I think the point trying to be made is video replay may not be inspiring, on it's own, but it's a tool to prevent the kids from getting discouraged by bad calls. Thus it's a net positive to inspiration overall.
Hitchhiker 42
24-08-2016, 15:11
The term inspiration is very subjective and to me having a higher percentage of calls correct is much more inspiring.
I think the point trying to be made is video replay may not be inspiring, on it's own, but it's a tool to prevent the kids from getting discouraged by bad calls. Thus it's a net positive to inspiration overall.
I definitly agree with you both. My question is whether this (however amount) increase in inspiration is worth the longer waits, and possible teams leaving to prior awards, which, in my mind, decreases inspiration to those members (trying to quantify an unquantifiable thing :) ).
Ryan Dognaux
24-08-2016, 15:16
I definitly agree with you both. My question is whether this (however amount) increase in inspiration is worth the longer waits, and possible teams leaving to prior awards, which, in my mind, decreases inspiration to those members (trying to quantify an unquantifiable thing :) ).
The one example of video replay so far at TRI literally happened during the setup for the next match and didn't hold up the match at all.
The argument saying time would be wasted is a bad one because it can be done so quickly that very little time is wasted.
waialua359
24-08-2016, 15:17
tl;dr.
I am a full supporter of video review.
If matches/events take longer as a result, then let it go longer.
I can see where subjective calls not being reviewed is fine.
But for missed calls such as breaching defenses in 2016, definitely could have used some video review.
I cant see anyone arguing or being upset, if a call was overturned to get it right, where an alliance ends up losing a match instead of winning. Getting a right call trumps that.
Hitchhiker 42
24-08-2016, 15:20
The one example of video replay so far at TRI literally happened during the setup for the next match and didn't hold up the match at all.
The argument saying time would be wasted is a bad one because it can be done so quickly that very little time is wasted.
Fair enough. All I'm saying is that benefits/costs should be weighed carefully. If it takes up little extra time and allows better calls, I'm all for it.
My question is whether this (however amount) increase in inspiration is worth the longer waits, and possible teams leaving to prior awards, which, in my mind, decreases inspiration to those members (trying to quantify an unquantifiable thing :) ).
(First - /agree with the earlier comments about not necessarily causing a delay but...)
This is basically a thought exercise, but if you ignore team sizes for a moment, which is more de-inspirational, potentially ending your season on a bad call, or having to leave the event that's run late a few minutes early?
Another thought here is that wouldn't Gracious Professionalism suggest that a short delay, if needed, to get the call correct, be the right thing to do?
...
Another thought here is that wouldn't Gracious Professionalism suggest that a short delay, if needed, to get the call correct, be the right thing to do?
Follow the rules.
Encourage or discourage FIRST to change the rules from one season to the next, but once they are established, follow them.
bigbeezy
24-08-2016, 16:18
The one example of video replay so far at TRI literally happened during the setup for the next match and didn't hold up the match at all.
The argument saying time would be wasted is a bad one because it can be done so quickly that very little time is wasted.
Can you elaborate on what happened? What was the call on the field/what was the challenge?
Ryan Dognaux
24-08-2016, 16:37
Can you elaborate on what happened? What was the call on the field/what was the challenge?
More info here: https://www.chiefdelphi.com/forums/showthread.php?t=146411&page=2
...
The argument saying time would be wasted is a bad one because it can be done so quickly that very little time is wasted.Given all the opinions (so far) about what Video Review should or shouldn't be, the word "it" in this sentence is very poorly defined.
For that reason, still color me undecided.
YMMV.
Blake
s_forbes
24-08-2016, 19:52
I like the concept of video reviews being used to make sure the correct call is made, using technology to make the competition better seems like a very FIRSTish thing to do. We have loads of regionals, maybe FIRST could do a trial run at a few and see how it turns out?
One potential downside is that there might be more people in the front rows waving their iPads in the air. If that's a sacrifice we'd have to make, then I don't wanna do it.
One potential downside is that there might be more people in the front rows waving their iPads in the air. If that's a sacrifice we'd have to make, then I don't wanna do it.
If there's video review at an event, it's gonna be from official cameras or camera stations. I don't see any other way to make it fair to the teams whose entire team + supporters are on drive vs the teams with 5 students and a mentor on video.
Honestly, if I was to do a replay--and this is what I'd be pitching to offseason events--I'd figure out where the most non-judgement calls* were going to be. For 2016, that's the Defenses (and possibly the Secret Passage). It'll vary by year. That's where any cameras get pointed, probably one per area. If whatever is being contested is in view, then it's review time. If not, you're right out of luck, better luck next time.
One other element that I'd be considering would be a case of "what if something else got missed?" My opinion would be that whatever gets caught on video but missed by the refs should be taken into account when determining if the outcome of the match was affected--and unless the outcome is ruled to be affected, the only thing that is reviewed is the original request.
*Defined as a call where, if it's seen, it happened, and there is no room for interpretation. Examples are non-judgement calls; Counter-Examples are judgement calls. Example, defense crossing. Example, two balls in the robot. Example, Secret Passage violation. Counter-Example, intentional tipping. Counter-Example, egregious behavior. Counter-Example, robot with ball bumps another ball out of the way or drives over it. Example, zone entry in 2014. You get the idea.
Ryan Dognaux
24-08-2016, 22:04
Given all the opinions (so far) about what Video Review should or shouldn't be, the word "it" in this sentence is very poorly defined.
My definition of 'it' is what I already posted in this thread on what we will be using at GRC in October. Link in case you need to go read it again -
https://www.chiefdelphi.com/forums/showpost.php?p=1556392&postcount=49
GaryVoshol
25-08-2016, 06:08
The one example of video replay so far at TRI literally happened during the setup for the next match and didn't hold up the match at all.
The argument saying time would be wasted is a bad one because it can be done so quickly that very little time is wasted.
This presumes every game has a long reset period and robots take a long time to sync up. Sure, if that's the case, then there could be time for a review. I know I've spent many a reset period minute discussing situations with students in the question box.
But really, shouldn't we be looking to make matches run quicker, with less down time?
Lil' Lavery
27-08-2016, 02:31
The one example of video replay so far at TRI literally happened during the setup for the next match and didn't hold up the match at all.
The argument saying time would be wasted is a bad one because it can be done so quickly that very little time is wasted.
The argument regarding additional time is not centered around the time to conduct the video review (hopefully there is a definite time cap on that), but rather the fact that blown calls lead to replayed matches. A single replay at a single event that did not change the outcome of a match does not change that factor in the least.
Ryan's point about 200 posts ago was that events could trial video replay to give some HARD data on what they did, how many reviews, outcomes, issues, etc.
One data point at TRI. One coming up in Oct from an event that Ryan is running.
Any other off seasons done or planning to do replays?
Sperkowsky
27-08-2016, 13:38
Ryan's point about 200 posts ago was that events could trial video replay to give some HARD data on what they did, how many reviews, outcomes, issues, etc.
One data point at TRI. One coming up in Oct from an event that Ryan is running.
Any other off seasons done or planning to do replays?
I'm going to try to convince a local offseason we attend and do the Webcast for to do them. Its not my team that hosts the event though so I'm not too sure If it will get the green light.
I'm going to try to convince a local offseason we attend and do the Webcast for to do them. Its not my team that hosts the event though so I'm not too sure If it will get the green light.
Asking is most of the battle, thanks for trying!!
JohnBoucher
27-08-2016, 14:55
Has there been a ref offering an opinion on this? I don't want to see a mass exodus of refs over this.
If everyone can call for a match review, would you be expected to review the match for the 6 objections made every match?
The refs all take this very seriously. Let the refs do their job.
On the flip side, perhaps I should start wearing a body camera to work. That will make my day go easier !!!
Sperkowsky
27-08-2016, 14:58
Has there been a ref offering an opinion on this? I don't want to see a mass exodus of refs over this.
If everyone can call for a match review, would you be expected to review the match for the 6 objections made every match?
The refs all take this very seriously. Let the refs do their job.
On the flip side, perhaps I should start wearing a body camera to work. That will make my day go easier !!!
There have been refs in this thread.
Here is the way I see it. A ref wants the right call to be made. No one wants their mistake to hurt a team. A safety net stopping some mistakes from hurting teams is something I can't see anyone mad about.
Hitchhiker 42
27-08-2016, 15:05
Has there been a ref offering an opinion on this? I don't want to see a mass exodus of refs over this.
If everyone can call for a match review, would you be expected to review the match for the 6 objections made every match?
The refs all take this very seriously. Let the refs do their job.
On the flip side, perhaps I should start wearing a body camera to work. That will make my day go easier !!!
The concern about 6 objections every match is unjustified. At TRI, only one objection was made the whole event. I think that teams will understand that asking for silly reviews is a waste of time for everyone, and will only ask when needed. Six objections in one match (not to mention, every match) is going to be very very rare.
BrennanB
27-08-2016, 15:19
Something to add that watching volleyball in the Olympics reminded me of. They have a review system for balls being in/out. And you have as many challenges as you wish as long as your claim isn't proven to be incorrect.
Perhaps a system could be implemented so that all teams get as many vid reviews as they wish until a claim they have made is denied and thus lose the privilege to contest via video review.
To me this is a win win. Discourages claims without certainty, and if a team has two or more valid issues, they aren't punished for it.
Has there been a ref offering an opinion on this? I don't want to see a mass exodus of refs over this.
If everyone can call for a match review, would you be expected to review the match for the 6 objections made every match?
The refs all take this very seriously. Let the refs do their job.
On the flip side, perhaps I should start wearing a body camera to work. That will make my day go easier !!!This ref's opinion*: I'm good with it provided the FTA side is on board there are clear bounds to it. Official video only, set situations each year that can be appealed, published mechanisms for doing so, teams cannot mandate reviews of judgment calls (they can question box it and/or refs can decide look ourselves). Maybe we need to cap the challenges per team; I'd be okay experimenting with either plus a blue box warning of possible changes mid-season. It also needs to be piloted more, not because I think I'll change my mind but because there are far too many nuances, procedures, and edge cases left to work out. We do not want to be fumbling over something like this in Week 1 on top of everything else when it could've been honed in off-season. Trust me, you don't want us to be either.
*I feel I can speak with some credibility on both how difficult this is and how much it can suck, but that does make me not a "pure" referee. I am a veteran ref and off-season head ref but also a veteran coach and have been burned by reviewable missed calls before. This includes a district champs playoff loss, though without lingering bitterness (we made Einstein that year). Consume with however much salt you want.
This ref's opinion*: I'm good with it provided the FTA side is on board there are clear bounds to it. Official video only, set situations each year that can be appealed, published mechanisms for doing so, teams cannot mandate reviews of judgment calls (they can question box it and/or refs can decide look ourselves). Maybe we need to cap the challenges per team; I'd be okay experimenting with either plus a blue box warning of possible changes mid-season. It also needs to be piloted more, not because I think I'll change my mind but because there are far too many nuances, procedures, and edge cases left to work out. We do not want to be fumbling over something like this in Week 1 on top of everything else when it could've been honed in off-season. Trust me, you don't want us to be either.
*I feel I can speak with some credibility on both how difficult this is and how much it can suck, but that does make me not a "pure" referee. I am a veteran ref and off-season head ref but also a veteran coach and have been burned by reviewable missed calls before. This includes a district champs playoff loss, though without lingering bitterness (we made Einstein that year). Consume with however much salt you want.
No salt needed, I'm good
Side no review: Too hard, no tech, refs are volunteers, match play times, adds too much to the competition, we all do the best we can
Side review: Tech isn't that hard, less than 1/12 of a Dean speech to the schedule, we all do the best we can.
As a VEX event producer I've said these words more often than I can count. "The referees rule is final. This is a robot competition that has referees watching each second. The referees rule, and their ruling is final. But in the event they are wrong, and that is a possibility, pull me aside and we will talk it over, as long as you want, but the last words out of my mouth will be 'the referees rule is final'.
But like Siri, I've been on the wrong side of calls and have blown calls DRAT YOU CELL PHONE VIDEO !!!
But this thread is on possibilities. Can we do it, can it work, can it be a positive method, how do we make it work?
Most of the thread has posts of "No", "Umm no", "Don't want to" "Will kill refs" "Will end civilization as we know it" (and worse)
So again, the fall is busy with off season as we build interest, try it out.
As the lone science guy in this thread :rolleyes: hard data beats feelings every day.
Siri asked for some boundary conditions (FTA's, process, etc) and I think thats a key thing. But FIRST isn't going to even test fly without some data points. Lets get the data points. And if the data points are "will end civilization as we know it" then we can close this off.
JohnBoucher
27-08-2016, 16:34
Can anyone cite a real world example where this is used and works?
I am struggling to find a situation that this would apply.
vBulletin® v3.6.4, Copyright ©2000-2017, Jelsoft Enterprises Ltd.