Re: Video Review Needs to Happen Now
 

^^^Champions right here.

Re: Video Review Needs to Happen Now
 

Thank you all so much for keeping the discussion going, I've never been the OP for anything nearly as heavily discussed or important. Anyone keeping the subject open makes the discussion bigger and thereby attracting more views, but I really have to hand it to people volunteering to put in the effort through refereeing. I also plan on volunteering as a ref as soon as I'm in a position to (haven't been able to attend an event without teams I'm involved with for years), but those of you designing and building setups and running off season events to test them have the potential to prove to FIRST that this can work, and I know if we are wrong we will admit it, but at least then we'll know, and be much more able to accept the current system. Debate has gotten heated, but people on both sides are really making me proud to be a part of the FIRST community.

TLDR you guys are pretty great.

Re: Video Review Needs to Happen Now
 

I want to address this. Inspecting is nothing like reffing. I have done both of these jobs (and many more) in the past two years. Reffing puts you in arguably one of the toughest decision making positions many times more than any other position, often without having the ability for someone else (let alone your volenteer lead (LRI, Head Ref, etc.)) to see it happen. An inspector can call another inspector or the CSA over to look at a problem during inspection, refs rarely have the ability to have someone else see what a robot did in the first six seconds of tele-op. Don't get me started on how vastly different the training/tests are for the positions.

I'm not going to totally disregard anyone's claim that calls have been missed or that ref quality could be improved, but it will come with much less weight than the experience of people that have the experience of being in the role for many years.

Re: Video Review Needs to Happen Now
 

+1

Best way to find out whether it is practical or not.

Re: Video Review Needs to Happen Now
 

I recognize that I'm a bit late to this thread, but I'd like to clarify the situation at TVR last year. As the team's drive team mentor for that year, I sent our student coach to the question box after we realized that there had been a scoring error and stood at the side of the field while the head ref talked to our students and then called a meeting with the other referees to discuss the match. While one of our alliance partners had a picture of the field at the end of the match with the correct number of stacks and we had a video, we neither asked the referees to look at them nor did the refs use them to make their decision. Luckily for us, the TVR refs did the right thing, both in terms of making a fair call and following the letter of the rules. Had that call gone the other way, I'm sure the aforementioned picture would have ended up on CD in a thread very similar to this one. I fully understand the sentiment behind the OP, as the desperation and lack of control our alliance felt trying to get the match rescored isn't something I'd wish on any other team. However, it seems to me that posts like these are best made a couple of days after the event once everyone has gotten some sleep and emotions aren't running as high on either side.

Re: Video Review Needs to Happen Now
 

Agree entirely. We will do this at the off-season event I run here in St. Louis this year. We're already doing the "hard part" - recording 1080P full field video in real-time. The easy part is opening the video file and just watching it again. I don't see a reason not to try this out.

Re: Video Review Needs to Happen Now
 

What if reviews were only allowed for SCORING disputes and not fouls/no-calls? I was quite shocked by how many matches were just simply scored incorrectly this weekend. Most of the time, the outcome of the match wouldn't have changed in these instances, but we shouldn't be giving teams points that they didn't earn. It's just wrong. I saw a few instances of the referees crediting an alliance with three robots on the batter at the end lf the match when the third robot on that alliance was on the complete opposite side of the outer works (I'm not even kidding, this actually happened more than once). I can give tge referees some slack on foul calls because most of those are based on judgement but how can you even mess up the match score like that?

Re: Video Review Needs to Happen Now
 

But fouls are a part of scores. So they would have to be reviewable too.

Re: Video Review Needs to Happen Now
 

I apologize for not reading 8 pages of posts, but reviewing videos is a bad idea.

The Original Poster brought up a valid point: They potentially lost 25 points and the match.

What about other fouls that were not called against their alliance? What about crossings for the other alliance the Ref's may have missed? Did the robot really start fully in the Neutral Zone and fully cross into the Courtyard?

Is this going to be like the NFL where each team gets to identify a play for review (throw an instant replay flag)? How long will the have to identify the point in the game? What is the penalty if the original ruling is upheld?

Not all fouls are on the field itself. What about cameras in the Castle area to enforce the 6 boulder limit? No touching across the line during Autonomous?

That Team delayed the match 2 minutes 37 seconds, and should have been called for delay. Robot broke contact with the Sally Port for 0.2 seconds and should have been credited with a cross.

The only source of instant replay video is FRC Cameras only, and one field camera from above won't be sufficient for all the rules. You can't allow an opportune video by a bystander, because that would be an unfair advantage. One team could have dozens of cameras, and only show the video when it helps their case.

The Ref's make the call which rule is enforced when. It happens real-time, and that is a major consistency of the enforcement of the rules between matches.

Re: Video Review Needs to Happen Now
 

I'd say the best way to figure this out is to establish clear communication between Offseason Event Organizers dedicated enough to implement review systems, and intentionally use different rulesets at each, preferably picking one set to be the "control" (not sure what to use, thoughts on this?) changing one aspect we're not sure about between each event, so we'll figure out what works better in each case that way. We're already experimenting with doing video review in general, why not try to learn as much as we can?

Re: Video Review Needs to Happen Now
 

Stop right there. Please read the posts, as I'm pretty sure your concerns were addressed in previous posts. However, a few of the concerns you're citing are things none of us know how to answer because FRC has never had official video review. These are things that I think we should test at offseasons.

Can't take you offseason organizers for granted but knowing the community, I have faith that there will be people that put in more effort than I can imagine to get this done for the benefit of future students. I know I'm going to do all I can, but this is going to take a community effort.

Re: Video Review Needs to Happen Now
 

As an offseason event organizer (more of a planning comittee member now), I would totally have us be the lab rat for a match review system and I'm sure the rest of the committee would be as well. The only challenge would be finding a head referee willing to do it.

Re: Video Review Needs to Happen Now
 

^^^ Well we've got one (THANK YOU SIR), any more takers? The more events that are willing to test ideas, the more ideas that get to be tested/the more general examples of using video review the community has to show FIRST. Anybody know people? IRI? Chezy Champs? Suffield Shakedown? Week 0s? R2OC? Between Preseason and Offseason, Blue Alliance shows that for the 2015 season there were 61 events. If less than half of that number of events offered to try video we'd have more than enough data to work with.

Re: Video Review Needs to Happen Now
 

Ok, I read through all the posts.

First of all, one camera can only solve one problem, and only partially. To be a "fair" Instant Replay (IR) system, it needs to handle most problems. That, one camera cannot solve (i.e. how many boulders are behind the castle wall).

You don't necessarily need one camera on a catwalk. There could be multiple cameras mounted on each castle wall (with at least 2 being really high up). While that would be more feasible for more events, it increases the complexity of the IR system. Don't forget, you also need to game time stamp each frame of the footage so you can know if the potential infraction happened at 21 seconds prior to the end of the game, or at 20 seconds.

That said, I think you might need a camera following each robot for some of the trickier calls where a general field camera would not provide enough detail. In Major League sports, that is easier when you really need to follow only one ball.

While limiting IR to Eliminations, and only potentially game changing calls, in close games, every call is potentially game changing (114 vs 110, when each foul is 5 points). What about Foul Cards? Do you allow IR for Foul Cards (both earned it, and it wasn't given; or it was given, but did they really earn it?). If a team has a Card, it can really affect how they play subsequent matches (more than just a game changer).

IMHO, only a Ref can be the IR initial reviewer. They have to know the game rules in detail (what constitutes a valid crossing).

I was chatting with a Ref at the KC regional this weekend. He said the Outer Works Ref was the most challenging position (the position that watches crossings). IMHO, the genesis of the problem is the Boulder Rules. Not only is the Ref watching crossings of the Robots, but they also have to watch crossings of the Boulders. A lot of Boulders illegally crossed. With the complexity of the Boulder rules, details is everything. That's when I joked on another thread that they will need 18 Ref's, one to watch each Boulder, and remember where it came from.

A few years ago, there were 2 Ref's at each panel (vs. last year when there was a Ref and a Field Reset person to help count totes). IMHO, this year, there should be 2 refs at each panel, at least for the Outer Works Position.

IMHO, increasing the number of Ref's is easier than IR. If you reduce the pressure on an individual Ref, maybe more people would be willing to be a Ref.

Remember: This is High School Competition. How many High School Football games have Instant Replay?

Re: Video Review Needs to Happen Now
 

I just wanted to clarify that the "GoPro on a stick" setup wasn't what I was considering cost or labor prohibitive. I was talking about a pure top down shot like John Fogarty was suggesting.

And, I don't want to speak for Sean Lavery, but I believe his point about that particular setup being prone to blind spots on this field still stands.

That being said, let's give it a shot at an offseason! I'm always for more and better webcasts from events and if the side effect is that we may have a way to improve team experiences, i'll take it.

I think the main disagreement we're having here is between people who think a system that will be able to catch most missed calls is too expensive so why bother, and people who think an affordable system would have flaws (blind spots) but is worth trying to improve the experience at least a little.

We should be able to come up with something in between the two that fixes some problems but still bothers everybody equally in it's own way.

Oh and this:

is why we started the RoboShow in the first place. I disagree that the GoPro on the stick is the best way to go about it in the long run, but it'd certainly be a start. Would love to talk about getting better coverage to more regionals.

Re: Video Review Needs to Happen Now
 

I've been skimming through these posts (there are a lot), and after being on an alliance that lost out on a ranking point today because of one of these types of errors, I would like to hear from any referee of Stronghold so far.

Do the refs simply have too much on their plates to be watching for defense crossings, fouls, and making sure stuff generally doesn't go sour? It would be good to get input on the people actually making the calls as they will have the best insight towards this issue.

Re: Video Review Needs to Happen Now
 

Agree that one camera will miss some things, especially with the height of drawbridges and sally ports this year. I would think instant replay would be used for major missed items such as defense crossings, scored boulders and egregious fouls for tipping and such. Maybe the head ref should reserve the right to deny the challenge if it won't affect the outcome over the match overall.

The GoPro definitely isn't on the level of what RoboShow, GameSense, IRI Live, etc. are doing but it's a start. Crawl > walk > run.

Re: Video Review Needs to Happen Now
 

Not much has changed in 11 years. People are still basically arguing that because it is impractical to have a perfect replay system, every replay system is detrimental or infeasible. But the point isn't to be perfect; the point is to be better.

Part of the process of implementing a replay system is establishing the rules for its use. If you're concerned that too many replays will slow down a regional, then limit the number of replays and the circumstances in which replays are available to teams. Maybe every team can have one replay in the qualifying matches, and one in the eliminations, and all the alliance partners have to spend their replays together. I would postulate that much of the benefit of a replay system is to give everyone the certainty that it's there in the rare occasions where it would clearly be beneficial—but making replays artificially scarce conveys the strong implication that they should be used wisely, and would put a tight cap on the likely quantity of delay. Obviously a good technical implementation would also serve to control delay.

Maybe a given replay scheme wouldn't address the rare situation where the referees are so terrible or overworked that nearly every match is questionable. But it would start to address the far likelier scenario that a team feels bad about an entire event (and perhaps the competition in general) because one major, game-changing call was blown. And even if the replay footage was inconclusive, the act of a review will serve to placate the team. Something was done, so it can't be as easily argued that the team is being ignored. And the team and the referee have an explicit common frame of reference to guide the conversation—even if the challenge is unsuccessful, that will cut down on speculation about motive or competence. In this way, the referees appear deferential instead of capricious (without compromising their authority), and it therefore makes the competition appear more credible.


I can't get behind that. Referees know that they're subject to scrutiny, and should expect commentary substantiated by fact. And competitors should not be made to feel like they're not welcome to contribute their assessments of the problem—because obviously it's they, and not the referees, that are most affected by officiating errors.


This is clearly an unreasonable way of looking at the problem. What if the effect was subtle or not yet realized (e.g. the student doesn't rejoin the team the next year, and instead joins the chemistry club, which might, in a couple years, lead them to get a degree in science not engineering)? How would you propose to find such a student, given that this probably isn't a metric that anyone tracks?

Re: Video Review Needs to Happen Now
 

Tristan's second link contained this Gem http://www.chiefdelphi.com/forums/sh...4&postcount=15 but also both threads made me realize there's a problem: People have actually been saying the same things we are on this thread for over a decade. Anyone from FIRST know what's causing the decision to be made against video review?

Re: Video Review Needs to Happen Now
 

I love the FIRST Robotics Competition. Some people say "Wil, you complain a lot about FRC, why do you do something you obviously hate?" This is a program that has entirely enraptured me and to the chagrin of other obvious high priorities, I put a lot of time into my team. I know a lot of people put a lot of time into their teams as well. Don't we all want FIRST, an organization that by its very nature must always be sprinting forward instead of dragging its feet, to constantly be improving and tweaking the status quo to make the program better?

Some disagreements lie in both parties wanting to make the organization better in fundamentally different ways (2 champs), some disagreements lie in two parties wanting some change but have differences in methods (Chairman's feedback). The worst ones, and the ones that get me really fired up, are ones where under an almost instinctive impetus to make a change, the controlling party wants to actively fight against the change (Cali districts).

I think piloting a replay system at an event like IRI and other offseasons should lead FRC down the path to pilot it throughout the season in certain regions or globally, and see if it works by 2champs.

Raise your hand if you think missed calls are not a buzzkill and something no one ever feels bad about.

Stand and be counted if you think field disconnects make FRC better.

Shout from the high horse on which you ride, proclaiming "yes, I love to be in a program that says 'No! We cannot come up with any way of improving a problem that has existed for years! There is not even a problem! Scorekeeping errors enable teams, not disable them!"

I know who some of you are, I just want to see how this line of thinking will be swiftly dragged into the mud. FIRST can be all things to all people. It can be a program that transforms people of all ages and backgrounds. It can always be better.

Re: Video Review Needs to Happen Now
 

Heres my input on this...

Back in 2014, out last match (match 90) at NYC was.... horrible to say the least. 2 of out alliances failed in the first 3 seconds, and we were the only running robot (you would not want to hear the anger that was behind the alliance wall) During the match, we pushed an opponent bot into our ball, and the ball scored low goal... that caused mass confusion with the field crew... 14 seconds later, we finally got the ball, and we have video of it taking that long... now this is where i flip the table. No, there should not be video reviewing, and you should just accept the results. BUT if there were video reviewing, the rule should be that you get one per regional/district. Now why one? Because there is simply no time. Especially this year when the field reset crew has a lot to do. Yes, me and my team were angry with the results, and did it help make us lose? Sure it did. But over time you just learn to take one on the chin, raise your head up high, and be the better man. You can sorta turn it into a life lesson if you want, by saying its like the real world where a mistake happens, and it doest work out for you at all, but you learn to deal with it and just keep moving on. Because trust me, there have been plenty of times where I wish I could show video and get things right, but you simply just can't. Thats a FIRST lesson.

Re: Video Review Needs to Happen Now
 

I agree that something needs to be done to address situations where the referees miss a score by an alliance. In 2014, we were sent home from two separate events when referees missed an assist. Both times we had video evidence refuting the calls. It's ridiculous to invest hundreds of man-hours, pay thousands of dollars, and then be sent packing by a distracted referee.

Re: Video Review Needs to Happen Now
 

While it has its merits, I think that a full video replay system is only going to be yet another stressor placed on event organizers, referees and FIRST.

Initially I was thinking: Why not just change the current rule? Often, teams go to the question box with a complaint about the way a foul was called or match was scored, and they try to argue their case without video evidence. If teams that go to the question box could simply provide the replay of the exact moment of the missed call, would it be too unreasonable for the refs to use that as evidence to overturn a call?

But this only introduces the potential for more delays. The Alamo Regional was a full 2 hours behind schedule, and lord knows how much more time would be lost scrutinizing every detail of a match just played.

One other (smaller) factor: video replay of every match would mean that scores would be delayed 2-3 minutes after a match. I can't see that being well-received by the audience :)

Re: Video Review Needs to Happen Now
 

Just be careful with the inflammatory language there. I agree that it's frustrating, but to call the referees distracted should be very far from the truth, and taken as a little bit of unnecessary "taunting" (for lack of a better word).


That being said: I can't say for sure, but I might be thinking about trying something involving replay at the local offseason out here. Too many variables need to fall into place, though, to make a call one way or the other at this point--including who ends up being the head ref at that event.

Re: Video Review Needs to Happen Now
 

I'll bite. I was a student Coach and Driver for 3 years, mentor Coach for 2, and have reffed the last 2 years (I'll be a Coach at AZ West later this year also). As alicen and EricH noted I'm sure there are a growing number of alumni who have been on both sides.

Despite the missed calls that have happened FIRST actually has made some strides to improve referee calls. The refs do not have to watch Boulder scores at all this year (automated scoring is the best way to not miss scores even if past failures like 2006 make us wary of it) and the training has very detailed questions on difficult scenarios that could happen in match. There was also a referee update after week 1 to put more focus on Outer Works Crossings.

I think video review could help with getting calls correct, but it needs to be used in limited cases to keep events running smooth and to limit its use to fixing errors that are significant and can be reasonably evaluated. Building off of other suggestions in this thread I would propose:

• Each Alliance is allowed one challenge/review in the playoffs. Unlimited challenges will result in reviewing every somewhat close match since no one wants their event/season to end. If the system works it could be expanded to each team having one challenge for all qualification matches also but not more to limit review quantities.
• Head Ref leads the review process and delivers the final call, with other refs who were involved in that call assisting if necessary. The rest of the refs keep the next match getting setup correctly so that minimal time is added to normal field reset. Adding a review process should not require an increase in referee headcount given the difficulty in finding referee volunteers already.
• Only match scoring errors (and penalties that lead to an automatic score) can be reviewed. This year that would mean defense crossings, challenges, scales, autonomous points, G13, G28, and boulders (though I am not sure if allowing review of an automatically scored element is reasonable to review, counting balls/disks scored in a match from a video is time consuming and more prone to error). Fouls are not reviewable as it is not easy to determine what fouls were assessed from a video and many involve a judgement call by a referee who has a better view than a camera or driver in their station will.
• The score or lack of being reviewed must be significant enough to affect the outcome of the match (or an RP being awarded for games like this year). Reviewing whether a crossing was awarded in autonomous or teleop in a 40 point blowout is a waste of time and the implications on ranking tiebreakers are not significant enough to justify the resources for that.
• Video evidence must be indisputable to change a call. The point is to receive credit for an obviously missed score, not debate further a close call that a referee already used their best judgement on (such as barely breaking contact with the Outer Works and Sally Port door).

I'm not sure that anything other than a full field view all match should be allowed, as it would be difficult to determine anything from a video flying all over the field following one robot. I don't have good suggestions on implementing a video system.

Overall I think that a review system would only be helpful in a small handful of circumstances, such as a missed stack last year or an obvious defense crossing this year. A video is not going to give a good view of things like bumpers being barely in or out of the Outer Works for when a crossing was not awarded as complete, and those are the more common scenarios that as a driver I would think "we should have scored for that" but as a referee I call a no score because of a different view on field.

Also I thought I'd throw in that I love Stronghold, the referee jobs are hard this year but this is one of the best games (especially for spectators) I've seen in FRC.

Re: Video Review Needs to Happen Now
 

The OP wrote My reply is, "No."

Re: Video Review Needs to Happen Now
 

Well said. We all know no one will stand for those reasons when put the right way. FIRST is about more than the robot, its about inspiring students and always becoming better, whether it be robot capability, gracious professionalism, or running of the competition itself.



That to me is a lesson in just giving up. Do we want our future engineers to accept defeat when they fail? No, that is not a FIRST lesson. You wished you could show a video, what if you could? We identified a problem, and are working to solve it. Being the better man is not giving up, though solid GP for not getting mad, but while part of being the better man is accepting what you can't change, the other part that is just as important is working at the things you CAN. We can't change results from past regionals but we can do our part to stop it from happening again by implementing a review system. This was never about complaining about the past. Over time, things do get better, often teams get another shot, but there are plenty of Seniors that dont get another chance. You say it can't happen but don't say why. How about you help us try? I'd love to be proven wrong, but at least give it a chance.

See earlier posts concerning the time issues, and proposals have already been made regarding limiting challenges and I fully agree, though I'd like to allow teams to continue to challenge if they get the call right. Losing your ability to challenge for being wrong would be a HUGE incentive for teams to not overuse that option, eliminating time problems resulting from too many reviews.


Again, video replays would only be called upon in contested situations, and incentives can easily be provided to make sure this privilege is not abused (described above).

Also, whether you're for or against replays, I think we should all work together to test ideas at offseason events. How else can we say with any sort of confidence which option is better? Contact your local offseason coordinators to see how you can help get this together, I know I will.

Re: Video Review Needs to Happen Now
 

^^^EXPERIENCED REF AND DRIVER

This ruleset is great, and I think this could be our base rules, minus defining where cameras are to be placed and what video feeds can be used. Some variations to try would be changing how many reviews a team gets (especially allowing them to keep reviewing if they aren't ever wrong), allowing foul reviews, implementing other penalities for baseless video requests, and whether or not to allow review in quals vs elims. Great start.

For camera placement, I'd suggest using existing stream setups to start. Variations could be placing cameras at or as close to bird's eye as possible, and not adding new cameras but allowing fans/media reps to submit video, and referee body cams?


No chute door?

Re: Video Review Needs to Happen Now
 

Super insightful, thanks Blake. Added a lot to this discussion. /s

Re: Video Review Needs to Happen Now
 

Folks you can PM me if you care to. A longer post by me would repeat what other experienced people already wrote.

Whatever faith people put in my opinions should simply be added to the "No" column, if anyone is keeping score.

Hopefully this added context explains the purpose of my earlier, succint, no-soapbox, horse-is-already-dead post.

Blake

Re: Video Review Needs to Happen Now
 

Horse is both alive and dead until we open the box. We should get this tested and if we're wrong we're wrong, if not FIRST can be made better. Why not take that chance?

Re: Video Review Needs to Happen Now
 

Hi there, I was a ref this last week at the PNW Wilsonville district event. Personally I think while being an outerworks ref is challenging, it's still doable. The biggest issue in my experience has been the Sallyport, which opens TOWARDS the referee, such that it is often difficult to determine if a robot began their crossing free of contact with said defense.

That said, there have been many instances at my event alone where teams were in the question box after talking about missed crossings, when many attempts were not valid in the first place. I remember several specific instances where a team either didn't begin the crossing entirely within the neutral zone, or never completed the crossing entirely into the courtyard before backing up and crossing again. While there are certainly missed calls, as with every game, I don't think there are nearly the number of uncalled crossing that teams are asserting there are. Many teams seem to be confused as to the exact definition of a crossing, and it's often hard to see exact robot positioning across the field. As such, teams should make it incredibly clear where they start and end their crossings, to help avoid any problems on both ends.

Re: Video Review Needs to Happen Now
 

For all the reasons other people I agree with have already explained accurately.

Re: Video Review Needs to Happen Now
 

I've seen reasons it might not work, but nothing refuting trying the system over the off-season

Re: Video Review Needs to Happen Now
 

Being on drive team this year, the validity of this argument is quite clear... It's not a matter of if but when. We clearly crossed a defense 2 times but the refs didn't see it/ made it unclear if we crossed to we had to waste more time going back through it. This clearly could be solved if an "over head" camera was installed (use the top of either tower). But the case is when to use a video replay. I believe that if the refs themselves are unsure of a call or if a valid question is raised (post match, in the question area) that they can call a match under review and sort it out.

Re: Video Review Needs to Happen Now
 

From what I've read the main arguments against in-house video replay (no external video, that opens a can of worms) are cost, time and "this is the way it's always been done."

I look forward to proving all of these wrong when we run a pilot of this at the Gateway Robotics Challenge in October. Expect a whitepaper that addresses all of the previously mentioned concerns.

Remember everyone - tons of people said the district system would never work in FIRST. Did Michigan take no for an answer? Now districts are the main path forward for FRC. There are a million reasons to say no, I'm proposing a solution that makes it easy to say yes, let's try this.

Re: Video Review Needs to Happen Now
 

I have had experience with refs making the wrong call, effecting the winner of the match in 2014. I was driving for 3324, the Metrobots, in the semi-finals against 16. In the first match, 16 won, but we put up a good fight. In the second match, I actually managed to defend enough against them to win that match, however, they went to the judge's table afterward, and somehow convinced them that we cheated/broke a rule, so they redid the match entirely, when video evidence would have proven we did nothing wrong. The third (second official) match, 16 won, and our season was over. I am still extremely proud of beating mentor built, swerve drive with pneumatic shooter, with a student built mecanum drive with a sketchy kicker, and it's really unfortunate to not even acknowledge officially that we won that match.

That being said, I still think video evidence should not be shown to refs. If a video proves a ref wrong, people begin to trust the ref less and less, just ensuring more drama and chaos. I know the feeling of refs making mistakes, and it is extremely frustrating and makes you hate the system at the time, but it's really the most logical thing to do. Videos don't always show what happens, such as a drawbridge blocking a view, so it can and will just bring up more controversy.

Re: Video Review Needs to Happen Now
 

Oh boy...

Re: Video Review Needs to Happen Now
 

What's the problem?

Re: Video Review Needs to Happen Now
 

My issue with it is that the post comes across as accusatory unnecessarily, and makes assumptions about another team's robot and team as a whole without any sort of validation or proof.

Re: Video Review Needs to Happen Now
 

Ok guys. We're taking a breather for now.

2:25pm EDT: Reopened. Please take extra care with your posts.

Re: Video Review Needs to Happen Now
 

I've been waiting for this.

So instead of jumping to video review I think the FIRST needs to make the fields smarter overall. If they moved the burden of decisions away from Ref's I would think most people would be happy. Adding moar sensors to the field and having the field keep track and the refs verify would make for a more accurate game overall, however this is a slow, expensive process. For a video system IMO the most perfect awesome amazing thingy would be SOLOSHOT. Its a camera, a smart tripod, and a tracking device which has the tripod constantly aim the camera at the tracking device. The issue is one of these setups could cost over $500 and you would need one per robot MINIMUM. This is $3,000 dollars added onto pricing which is a lot for a system that might not even be necessary.
FIRST should always be moving to improve games and how they are run, but the last thing I want to see them do is implement an expensive solution to a problem only for it to not work or even generate more problems. Instead of jumping to the biggest change maybe we should implement more little changes to quality of life.

Big thanks for crowd control though!

Re: Video Review Needs to Happen Now
 

You absolutely DO NOT want to re-open this can of worms, I''ll leave it at that.

Re: Video Review Needs to Happen Now
 

Lots of people are comparing FRC to the NFL/NHL/MLB/Nascar and saying that we should have video replay like they do. Please remember that This is a game for High School Students. As far as I am aware, High School sports do not have video replays for if/when the officials make a bad call, and if what I have heard from the stands at a high school sporting event from the parents is any indication, all the officials do is make mistakes*.

Yes I understand that bad calls can make or break a season, and it sucks. However, instead of trying to spend a lot of money and time to make this whole big procedure to make sure it doesn't happen lets instead, as mentors, use this opportunity to teach our students about making the most out of a bad situation and treating people with dignity and respect.

*I do not think that officials at high school games get everything wrong, just pointing out that that is what the parents would like you to think when a call is/isn't called for/against their child.

Re: Video Review Needs to Happen Now
 

Fouls add 5 points to the other Alliance's score. Thus all fouls would need to be reviewable. Both called and uncalled fouls.

> Fouls are not reviewable as it is not easy to determine
> what fouls were assessed from a video

May not be as difficult as you think. When a Foul (or any other violation) is called, the Ref points at the offense, and waives the flag indicating the color of the Alliance causing the foul. If the flag is waived, a foul is called. If a flag is not waived (and you don't see the Ref doing anything at the panel), the no foul has been called. The tricky part is if multiple fouls were called at the same time (robot, with a boulder, crossing from the NZ to the Opponent's SP). Then you have to watch the score to see how many fouls were called. The score portion of the score board is on each video, isn't it? Time stamping is important, and that is one way to get the time stamp.

Very few things are "judgement call". It either happened, or it didn't. Either the robot broke contact with the Sally Port, or it didn't. The video may not provide convincing evidence to overturn the call.

Non-reviewable judgement call would be: Robot is likely to damage the field, so the Head Ref disabled it, if something is Strategic, or if it rises to the level of "repeated". Those are forward looking/intent judgement calls that are made real-time, so should not be reviewable.

So, what do you do if a Robot, upon review, is determined to have contacted something outside of the Field (G3), and it was not originally called by the Ref? That violation results in a Disabled. It is not a judgement call. Do you then review the rest of the match to back out all points scored by that robot?

Re: Video Review Needs to Happen Now
 

With regards to field automation, I would like it... provided it works. FRC games don't exactly have the greatest track record when it comes to scoring automation. The 2006 goal sensors had a slew of issues, and many (all?) events verified scores with a human scorekeeper (delays were introduced after autonomous to ensure autonomous scores were correct). The 2010 goal sensors experienced much lower volume, and were thus better performing in general, but there were still cases of sensors not working and humans being used to confirm scores. In 2011, there were quite a few threads covering the issues regarding the sensors on the poles registering minibots. As far as I remember, 2012's automated scoring has been easily the most accurate (ironically, so much so that thereal time scoring was removed from the screens of elimination matches to heighten excitement). 2013's weight sensors once again had issues resulting in human confirmation of scores and untrustworthy Real Time Scoring.

A lot of that is game design, but as was pointed out earlier in the thread, making a game that is easy to score/referee often results in a game that people don't view as exciting. Excitement comes from constant action across the field, and that same constant action is what makes games more difficult to officiate. Often, this can hold true even for automated scoring (see the issues with ball jams in 2006, DOGMA penalties in 2010, and minibot sensor viability in 2011).

I'm contesting that the numerous cans of worms opened by video review aren't going to make things better. I can all but guarantee that the first case of a blind spot in video is going to result in a thread like this one, where a party objects to not being allowed to use their teams' video evidence to support overturning the call. Even worse is when two camera angles show contradictory views. Further still, instant replay does nothing to fix judgement calls, and quite frequently exacerbates the situation. I've seen plenty of open hostility towards replay officials this season in the NHL, along the lines of "if they're not going to get the call right, why bother having instant replay?" (Only less polite).

I find the dystopian vision of large teams will full video replay crews in the stands attempting to find ways to reverse the outcome of a match to influence the standings in their favor rather unsettling. And given the lengths team already go through to find an advantage, including both video scouting systems and attempting to shift the meta strategy of tournaments to influence rankings, I don't find this vision particularly far fetched.

If it's so clearly unreasonable, than perhaps people should stop using this example as to why FIRST needs to improve [aspect X] of the competition experience. This hypothetical has already been brought up in this thread, and many similar threads regarding perceived shortfalls of the competition aspects of FRC. So long as people cite students becoming so discouraged by a bad call that they chose not to go into a STEM field, I will continue challenging them to find me an example.

Re: Video Review Needs to Happen Now
 

Worth noting is that you don't spend thousands and thousands of dollars for a few matches in any other high school sport.

Re: Video Review Needs to Happen Now
 

Yes, you do.
http://www.nytimes.com/2015/01/17/yo...orts.html?_r=0

On the note of field automation, from what I have read here on cd the flags above the towers aren't even working at most events !

Re: Video Review Needs to Happen Now
 

However any other sport has matches longer than 2:15, or their season is more than what? 25 matches? Being generous here. Sooo 56 minutes of play time. Or like. One match of any other sport.

Re: Video Review Needs to Happen Now
 

For a lot of regionals, they either didn't get the flags with the field road cases or the crew setting up the field decided not to put them up.

Re: Video Review Needs to Happen Now
 

I greatly look forward to seeing this report! I've been on both sides of the line (judge, head judge, mentor), and I have faith we can come up with some sort of review system to help eliminate the bulk of the most impactful wrong decisions. Limit to elims only, for example.

Re: Video Review Needs to Happen Now
 

High school football seasons are typically shorter than 10 games (not counting playoffs). While a football game lasts 2-3 hours of time in an evening and 60 minutes of game clock, there's roughly only 11 minutes of actual gameplay in that span. Considering an individual athlete is (typically) only playing offense or defense (not both), cut that figure in half again. For football, we're getting into similar territories of playtime.

Plenty of other sports condense their competitions into short fire bursts. A cheerleading routine is only a few minutes long, and only done in competition a handful of times per season. Swim and track races have sprints that last under a minute (or just a matter of seconds). Shotput, javelin, long jump, and other individual track and field events are rather swift affairs for each athlete.

There are obviously some other sports with much longer match times (cross country) and higher rates of individual participation (basketball starts may play close to the entire game). But generalizing statements about playing time versus high school sports are going to have tons of exceptions. FRC certainly doesn't provide the best bang for the buck in terms of playing time, but it's more competitive than you may think.

Re: Video Review Needs to Happen Now
 

I go back to the idea a basic video of the game that anyone can take could be used for some end of day 1 RP issues and also any elimination issues (at some point after the game except in case of elimination w/l decisions). I don't want video to slow the game play down but I think there is a reasonable option that should be used for critical calls like losses in eliminations and first/second order ranking point issues...until FRC allows review though its the way it is.

CV seemed better than SD in terms of crossing accuracy except our first match where we had to go through portcullis (which is blatantly obvious ) six times each way, I heard our SF alliance petitioned a cross in CV elims too and were denied the points as well. we talked to the head ref in Qual 1 and made them aware ..it got better. We still won and got the 3RP

Here is FRC response...

"Thank you for writing and your suggestion. However, we have made the determination that any level of video review would open the door to extensive reviews, as every team at an event could have thought they saw something that caused them to miss an RP. To determine if something was indeed missed, out of fairness, it would require a review of all concerns.
Also, it might be helpful to think of this as a typical high school sports activity, in which video reviews are usually unavailable, and it is recognized that referees are human, and will occasionally miss something. While unfortunate, this is an accepted part of the experience.
Please do not hesitate to contact us if you have any questions."

I see where they are coming from luckily from our perspective it dd not affect anything this season for us. We had great alliances each competition and a shot at it...which is all you can hope for anyhow.

Re: Video Review Needs to Happen Now
 

Let's analyze football, on average the pads, helmet, and uniform cost $550 per student, which comes out to ~$25,000. Each coach is paid on average 20K-25K more than they would as just a teacher. Which conservatively comes out to ~$160,000/year. So without buying any equipment, just uniforms and coach salary, schools pay ~$185,000/year for 10 games (9 if you live in Kansas).

Re: Video Review Needs to Happen Now
 

It's not a lesson of giving up, it's a lesson knowing that you've been beaten. Would I have liked their to be a rematch for my last match? Sure. Would video helped? Definitely. Definitely after an extremely rough build season where teams most likely would've folded... And it was my last year too. You're not the first team to feel like there must be video review, and you won't be the last. It's something you have to deal with. And it wouldn't matter now anyway, because it's over, and here's no going back.

Re: Video Review Needs to Happen Now
 

You must have a very rich school if that is true. And to my knowledge, the only football coaches that get paid are the head coaches and sometimes the assistant head coaches.

Re: Video Review Needs to Happen Now
 

Scenario A: The head referee makes what they thought was the correct call based on the evidence at the time, but later finds out the call was actually wrong. The call was match deciding and one alliance goes home denied of an event win.

Scenario B: The head referee makes what they think is the correct call. One alliance uses their once per elimination tournament challenge card requiring the head referee to look at the match with the captains from the alliance to explain his reason for the call. This also gives ample opportunity for the challenging alliance to provide specific rules or Q&A's to support their challenge. A match replay is issued if the head ref confirms & agree with the challenge's claims or the results stand as called if the head referee doesn't seem evidence to overturn the call.

There are definitely times where something like this could have came in handy. Look back to San Diego last year where the incorrect call was made and was admitted to be wrong once the Q&A was produced. Maybe having that challenge would have given them a moment to find the Q&A and take another look at the situation?

My point is it's worth taking a look at and talking about. This could give referees another tool to help them out when making very tough calls. Why wouldn't we want to help our referees out?

Re: Video Review Needs to Happen Now
 

How can you review something for correctness when there's no record it happened?

Re: Video Review Needs to Happen Now
 

Folks,

FRC/VRC/FTC/VIQ matches don't proceed in little increments the way many big league sports do (USA football downs, baseball batters & pitches, Cricket ___s, Tennis volleys) .

And, they don't have long stretches of continuous action like many other big league sports (Nascar, Soccer, Basketball or Hockey).

Because of the implications of those differences, I believe that as a practical matter, untangling the spaghetti of knock-on effects that many (most?) overturned calls would create, would be a nightmare of ambiguity.

Except for the last N seconds of a match (where N is very small), everything that affects the score has an effect on how teams and alliances play the games. Neither alliance could argue that an overturned call from early in a match doesn't effect both alliances. In those many cases, I think there would be little a ref could do other than throw their hands up and initiate a replay.

I believe that Video replay successes in big-league sports are unlikely to translate into FRC successes. I predict that if video replay in FRC is successful, it will be because an FRC method is created, not because a big-league method is copied. I also believe that a successful FRC method will be a purple squirrel for most, and the great white whale of some.

I'll look forward to hearing from folks who attempt to use video replays during the upcoming off-season, to see what effect these differences create between what folks are imagining, and what is actually going to occur. I'll bet a beverage that the gulf between ambition and accomplishment is pretty wide.

Unless/until we have some experimental results to digest, I think our current posts are only making 11-12 years of "rubble" bounce.

I'm willing to stop bouncing the rubble, and wait for the results.

Blake

Re: Video Review Needs to Happen Now
 

(balance snipped out).
Very cool! Great GP! And great courage!

Re: Video Review Needs to Happen Now
 

I have to say I like Ryan Dognaux's plan to pilot this at an off-season event. I like the challenge rules (one per team, etc) that were presented in another post. I'm looking forward to seeing a series of threads on "How we did replays at XYZ event."

I'm also behind adding extra scorekeepers to the field crews to help stop some of the pain for upcoming events. It's possible that an extra set of eyes would cut the pain points way down.

In the past I've run very large VEX events. In most cases there are 2 refs watching the action on the fields. (Running two divisions at a time) Once elims start the refs double up and they pretty much get told watch a single robot's action. With a person focused on a robot it's hard to miss things. For example in VEX, the dreaded "pinning" is easier to manage since there is two refs watching (the robot doing the pinning and the robot being pinned each have a ref watching the action). Again, only for the eliminations, not possible for all matches.

I'm in a wait and see mode, happy to let Ryan and others pilot this out. Alpha test for Stronghold, beta for next year, to see how it flies.

Re: Video Review Needs to Happen Now
 

If anything this seems more like an arguement in favor of implementing districts than a replay system. With districts you get 2-3 events, with 12 matches at each.

Re: Video Review Needs to Happen Now
 

My take on this is that if FIRST were to make attempts to introduce a video review system, it would need to be piloted and slowly rolled out, like Districts, or webcasting. You can't just flip a switch and make it happen. After reading a lot of the comments in this thread, I will not be taking a dedicated stance on it since I share opinions from those on both sides of the fence. Yes, it is not OK to allow students to leave events disappointed and discouraged from their hard work because of a referee misstep - but as a soccer referee, I know what it's like to be on the other side. Learning to deal with disappointment and seeing what YOU can do to make YOUR robot better is a much better thing to get out of it than to get angry at the referees or get angry at FIRST.

If a system of video review were to be implemented, there would be several requirements and constraints that would need to be addressed.

1) It costs a lot of money to have the events record the matches. I don't know how many fields are circulating within FIRST, but a camera and recording system would need to be included with every field if this system were to be fully implemented. One way to solve the cost constraint is to get a camera company to sponsor FIRST, maybe GoPro, Samsung, or Sony, and they could work in collaboration with FIRST to supply, design, test and implement the system.

2) It delays matches and the flow of the event. I like the idea that was thrown out earlier of limiting the amount of video replays that one team can call. I would think that limiting it to 2 replays per elimination alliance is a fair number. Also to keep from delaying robot flow, you can limit the number of team members who may come to the question box to 1 or 2, such that the rest of the team can be removing the robot from the field. There's a reason you only have one timeout coupon in the eliminations. There's a reason that you only have one backup robot to call in - it just takes up too much time, and takes away from the appeal of the other teams and the visiting public, which at a certain point you must take into account.

3) There's not enough camera coverage. I don't think I've seen any FRC event this year that has less than 3 cameras covering the field on the webcast. If FIRST were to implement the system, it would need to have multiple camera angles aimed at the field - it is also possible to use the existing equipment that many events use to record their matches. These would be logistics that would need to be figured out in the pilot and testing period.

Not taking a stance, just providing solutions. If we decide against video review, what solutions will be provided to prevent referee mistakes?

Re: Video Review Needs to Happen Now
 

We will be discussing this topic on FUN Recap tomorrow! Thank you to all who are passionate on both sides as we put together a story.

Re: Video Review Needs to Happen Now
 

I'm on board with Sean that I don't think video review is going to significantly reduce complaints unless it's perfect. Whatever circumstances you confine it to, complaints will just move to the boundaries outside of them, and you'll get the new class of complaints about bad video calls. All for the low price of yet another key volunteer and an av system.

Also, I feel I should point out that whatever video replay solution you guys come up with must be implemented at every regional and district. You're planning on enshrining this in the rules, and inconsistency between events seems like just as big a complaint generator as bad calls. I think this means that any solutions that depend on streaming are non starters. Also I suggest you either need a match timer in view or a scoring overlay, or you're going to be missing at the end of auton and during the endgame.

Re: Video Review Needs to Happen Now
 

This about sums up where I'm at. I cannot wait to see the results of some offseason events that choose to implement this. Until we have some actual data, most of these arguments are speculative. Let's see how replay would work.

Re: Video Review Needs to Happen Now
 

The solution we're using doesn't require for you to stream. Streaming and recording are two separate functions that can happen simultaneously or you can just record the matches if you don't have internet access. This is a great point though, not every event has the internet to stream out and any solution would need to be able to operate without it.

The scoring overlay you're talking about is just the real time score graphics that the field provides. We use a cheap Monoprice Video Grabber just to get that part to overlay on top of the GoPro's HD view. The real time graphics don't look as good as everything else, but at least you have them.

To record the events that are already being webcasted (which are a majority of the events today), it literally requires 0 more cost to record matches. You already have a PC, you're already feeding video into some piece of software, you can just record that same feed. The low cost setup we use would run you $500 if you already had a decent PC and $1000 if you didn't. For FIRST that's not a lot of money.

Re: Video Review Needs to Happen Now
 

Folks, I'm suddenly reminded of something.

When was the first Red Card or Yellow Card issued for actions taken on the field?



Anybody who says 2010, you're right for OFFICIAL events, and wrong otherwise. It was actually IRI 2004. That's right, 5 or so seasons of testing in an offseason... And eventually, it worked.


I think, if video review is going to become a reality (note: I'd rather it didn't), that it should be thoroughly tested. Each event to try it should try something different, or slightly different from other events, and post a detailed whitepaper on setup, rules of use, actual usage, and what calls were changed. As I noted earlier, I'd be willing (if cleared by the planning committee and other interested parties) to apply it at an offseason near me, how it works TBD (I've had some ideas in the past and kind of tossed them around a bit on CD).

Re: Video Review Needs to Happen Now
 

What about scenario C: red alliance successfully challenges to get extra points to wint the match. Then blue alliance challenges a different call to get extra points to re-win the match.

Re: Video Review Needs to Happen Now
 

If both are valid, and both alliances have challenges to use, then that is perfectly fine.

Re: Video Review Needs to Happen Now
 

The game is pretty intense for the number of refs they use.
Would it be possible to have separate refs AND scorers?
Just my $.05 :D

Re: Video Review Needs to Happen Now
 

Well i have some good news: This thread is all about offseason events doing just this. http://www.chiefdelphi.com/forums/sh...d.php?t=145690

Re: Video Review Needs to Happen Now
 

Don't forgot a survey of participants. Preferably anonymous to maximize the potential negative feedback.

Also, it probably won't be necessary, but y'all probably should go all in on getting video review at every offseason. At some point after some refinement, you're going to want to dump your system on an event that's completely unfamiliar with it to see how bad it's going to be for the first year being run by a crew with minimal training on it.

Re: Video Review Needs to Happen Now
 

Inflation?

Re: Video Review Needs to Happen Now
 

Hi all!

So, I wanted to wait to post these videos and I think its calm enough to do so at this point.

Sorry for the poor video quality. I had these videos recorded so, I could go back and watch did not know they would be involved in this sort of thing.

QF-1 Was pretty clean

But QF-2 was bad. To start the big 2 things are the fact that our alliance very clearly crossed the cvr twice directly in front of 2 refs and that the red robot hit one of our blue robots during the last 20 second of the match.
If those point values were recorded correctly there would have been 35 points added to the score allowing us to win.
Here is the video for those who would like to see.
https://www.youtube.com/watch?v=qHnJ...ature=youtu.be

QF-3 Had less clear cut issues but, you can see those aforementioned intent to tip fouls. Those pins at those angles deserve more penalty IMO. Even if there was just one more 5pt foul we would have won the match going into semis.
Here is the video for QF-3.
https://youtu.be/m2MWxJEBwpk

Some of the fouls we originally thought happened were not present but, you can clearly see enough in both matches to warrant us a win. 395 has not made CMP since 2011, we have never made it, and team 1546 has not made it since their rookie year in 2006. 395 is the only team that is even competing in a second regional. These are not elite teams who consistently make champs we are talking about here these are small teams who fight hard year after year in hopes of winning and its a shame to see all of that hard work destroyed by some bad calls. We are on the CMP waitlist and if I am not mistaken its possible to still get invited through that but regardless I am going to champs personally so see everyone there just wish my team could be there with me.

Re: Video Review Needs to Happen Now
 

I'd like to make a quick commentary here. Just to pick on the linked videos, this is why audience videos probably shouldn't be allowed to be used for review. Zoom took time to adjust, people were in the way (down in front!), and the camera tended to follow only one robot (and, BTW, that last is fully understandable, you want to record your team's performance and I can't say I blame you there!). That being said, if the part that was needed did happen to be clear and unobstructed, it could be worth it to use that.


As far as rules go... Just remember that pins are legal up until 5 seconds (and 6' separation or chasing by the pinned robot), and tipping a robot unintentionally isn't illegal per se but a strategy to tip a robot is. Just something to think about.

Re: Video Review Needs to Happen Now
 

No referee is going to touch this issue with a ten foot pole.

Source: Am referee. Check my signature.

Re: Video Review Needs to Happen Now
 

I think it's worth discussing exactly what a successful replay challenge should be able to accomplish, in order to more clearly distinguish between good and bad policies.

It's probably inherently infeasible to unwind a match just because the participants' reactions to a bad call were different than they would have been had the correct call been made. That's worse than judging intent: it's almost complete subjectivity, and trying to speculate about what constitutes a correct outcome is an exercise in conjecture. Instead of expecting to unwind the match, teams should play on, with the understanding that the call on the field might be reversed on appeal. It's up to the teams to choose how they complete the match based on incomplete information, but the fact that in hindsight they should have chosen differently is of no consequence.

By contrast, a replay challenge might be warranted in situations where the nature of a discrete event is unclear, and where that event is supposed to have a defined result when determining the outcome of the match. For example, did the robot complete a game task for which points are supposed to be awarded? The resolution is to either award points or not, and that takes effect at the end of a match, no matter what else happened.

But what about games where score depends on intra-match conditions? Those fall somewhere in between, and probably need to be considered as part of the game design process. One possible resolution is to credit the points mistakenly not awarded, but not unwind the gameplay that resulted from those points not being scored at their proper time. (Is that equitable? I guess it depends on the game.) Another resolution might be to trigger a rematch for certain missed calls: perhaps they're so fundamental to the game that to miss one is to threaten the legitimacy of the event. (In fact, that's the sort of situation where a strong replay rule is beneficial: most of the audience already knows the equitable outcome, so why not give the referees the opportunity to get with the program?)

So ultimately, having a replay system doesn't need to mean that everyone will always get rematches and slow the event to a crawl. Instead, it should be tailored to the situations for which it is usually advantageous (and then applied consistently), and should be made unavailable when it would most often be detrimental.

Re: Video Review Needs to Happen Now
 

By way of example here, you have 2006, where the alliance with the highest autonomous score goes on offense first, which was considered a significant advantage. Or 2004, where knocking a ball off in auton got you your balls 40 seconds sooner.

Re: Video Review Needs to Happen Now
 

I think the root cause of this issue is that people think there are "bad" referees that miss calls. That may be true, but fixing a bad system by introducing a new system isn't a great fix. Instead people should focus on a direct fix.

If everyone who's posted in this thread volunteered to referee, head refs would have their pick of referees. They could staff events with "good" refs instead of "bad" refs. The problem is that events can't really be selective in who they get to ref it. Some events may only have 5 applicants for the 5 ref positions. This means if one ref is known to be prone to missing calls, the event has no choice, they have to play them. If events had 10 refs volunteer for 5 positions, they would be able to select their "favorite" 5 (or 7).

So if you really want better calls, the best way to make is happen is to volunteer as a referee. Anyone who is complaining should get on VIMS right now, sign up for a week 4, 5, or 6 event and start the ref training immediately. Although it can't do anything for the event you're complaining about, you have the chance to make someone else's season much better. And then volunteer next year! You may take the spot of that really great ref, and now that they don't have to do a week 2, 3, 4, and 5 event, maybe they are willing to cover your week 1 event!

If events are able to have their pick of referees and can choose the most capable refs, and there are still problems, then I think that might be a good time to talk about video review. But getting more volunteer refs is a simpler, cheaper, more direct, and overall better approach to solving the issue of bad calls. A video review system is going to need more volunteers at the same training level as refs anyway, so why not just have more refs?

Re: Video Review Needs to Happen Now
 

Tristan, I know you set out to discuss "exactly" what a successful replay challenge ... But, what you actually wrote is some high-level thoughts about how a system might work, in general.

To be successful, I think a replay system would have to describe (at the least) exactly what action the ref should take, for every single gameplay and scoring rule, whenever a challenge reverses a call involving that rule. The time when the original call occurred, and the match score when the call occurred might or might not need to be factored in for some situations.

I think there is a fairly large gap between where you ended, and what must be included in a mature, robust, system that produces consistent/predictable results.

I'm not pretending that I know the answer for this season, or for any other. It will be almost certainly be darned hard to produce a complete set of replay instructions each year. Attempting to create those instructions is an exercise left up to the readers (those readers who think reviewing video is a good idea).

IMO, the off-season experiments folks are planning for this year won't be successful unless/until they produce instructions corresponding to reach rule of this season's game. The alternative of asking the refs to fly by the seat of their pants during a tournament would not be a good idea.

Blake

Re: Video Review Needs to Happen Now
 

For our testing this year we will just say if the head referee accepts the challenge request, a match replay will be issued. Keeps it really simple regardless of the play in question. May not be what would actually be implemented eventually, especially in certain cases like missing a game piece being scored, but it's what we'll try.

Re: Video Review Needs to Happen Now
 

OK, let us know how it works out.

You probably already took into account that if I'm a team focused on winning, even if my alliance gets beaten by 100 to nothing in a match, I might decide to protest a minor call, just hoping to trigger a rematch.

Similarly, if I haven't used my protest(s) yet, I might automatically protest any/every loss that would take my alliance out of the eliminations, hoping to trigger a replay.

If I'm especially mischievous, I might purposefully create some sketchy situations, just so that I can increase my chances of having an opportunity to successfully protest a call.

If these scenarios are something you want to discourage, to avoid slow-downs, I'm not sure how you can do it.

Blake

Re: Video Review Needs to Happen Now
 

Just make it so the head referee has the right to take away the alliance's challenge for erroneous & repeated protests. The question box exists today and while a few teams may abuse it from time to time, I don't see all teams doing it.

Re: Video Review Needs to Happen Now
 

I'm not sure having every successful challenge result in a rematch is right either. If the match was a 30-100 blowout, and the challenge would net a maximum of 10 points to the 30pt team, there's no reason to replay... sort of.

Put in more "rule" type language:

The wording isn't perfect, but I think you get the idea. I don't think it's enough that a score change alone would alter the winner/loser of the match, but I think the play of the match needs to be considered to trigger a reply. Eg. that 30-100 blowout, if a Robot had to attempt a 3rd crossing because the 2nd was missed by a ref and the Robot turtled as a result, there's a reasonable cause to replay the match. If it didn't turtle, but just took some time away from scoring a high-shot, no I wouldn't replay it.

Re: Video Review Needs to Happen Now
 

Having been personally affected as drive coach from team 2869, I was hesitant to post as I didn't want my emotion getting in the way. However, I do like the proposed solution by many to try the video system out at any offseason comps. If anyone is planning on using this system at any events in the greater NYC region please let me know!

I would love to help out and let this emotion guide me in a positive direction in helping other teams!

Re: Video Review Needs to Happen Now
 

Don't remind me about 2006. Had a replay in a finals match because some field resetter didn't put the starting balls into the bins like they were supposed to, and a 10-ball auto dump got stuck on the sensor. Sensor read 15 balls (for 15 points--should have been 10). Opposing alliance drained 4 high goals for 12 points. Entire match played backwards...


I'd like to thank CVR for putting part of the problem into words... anybody in SoCal, go act on that post, please...

Re: Video Review Needs to Happen Now
 

Folks - I think I'm right if I say that one purpose of using video is replacing fallible judgements with measurable facts. Be careful not to introduce too much heat-of-the-moment judgement back into the process you purpose/test.

Re: Video Review Needs to Happen Now
 

The purpose of video review should be to allow referees to do their jobs better - that's it. It's not there to let teams endlessly draw out a tournament. It's not there to bring up incidents that didn't really happen. I don't want to introduce opinions, I want to introduce facts. Video doesn't lie and that's a great thing.

I think everyone can agree that our head ref's have a tough job. I'd like to try and help them out by giving them an option to review the previous match. I get that it's different and hasn't been done much before (cool to read about off-seasons that have done it.) Let's try it out in the off-season. Worst that happens is we find that it doesn't support the current tournament structure and we don't pursue it any further. Best that happens is we change the FRC event structure a bit in the future with defined guidelines set by the GDC & FIRST to improve event quality overall across the program. It's all upside right now. Let's give it a go.

I'll take Blake up on that beverage bet BTW :) I have high hopes and confidence in the system we use here in St. Louis. I'd love to try and replicate it. There are plenty of people who can tell you something won't work, it's up to us to prove them that it could work - that it will work. It has to be bullet proof and simple... and I think what we have checks both of those boxes. Really excited for October :)

Re: Video Review Needs to Happen Now
 

This last statement isn't true. Video can be cut, cropped, shot a deceptive angles, etc., all to tell an untruthful (or at least incomplete) story. Even the NFL can't find a definitive angle for every reviewed play. I don't say this to discredit the idea video review, but to remind those pushing for it that everything has its limits.

I think you and many others on this thread are well aware that video review won't resolve EVERY questionable call. Part of testing at offseason events will be determining what makes for REASONABLE review. You may otherwise find yourself down a very deep rabbit hole of "if only we had more camera angles," "if only we had higher def," "if only we had higher frame rates," "if only we allowed this/that/the other," etc.

I'm skeptical of video review, but I support those willing to invest the time/money to experiment with it. Just don't get led astray. The quest for "informative" video is a noble one. The quest for "infallible" video is quixotic.

Re: Video Review Needs to Happen Now
 

Heh. Reminds me of the time some friends and I were doing a video scavenger hunt--no editing allowed--and one of the items was making a half-court shot. We went for a full-court shot, and made it (on the video).

What happened, though, was that the video player was set to just play the list in order, and some quick start/stop work resulted in two videos looking close to one. Oh, and the observant folks in the audience might have noticed the leg near the rim, behind the backboard... :p (A second team member was up behind the backboard with a second basketball.)

Re: Video Review Needs to Happen Now
 

Just to keep the robotics replays emphasis on things. The references to edited video *are* interesting, but ... remember that even unedited recordings of a robotics match can deceive an untrained eye.

Identifying, and compensating when possible, for the unavoidable distortions 2D video introduces into recording/recreating an evolving 3D (4D) scene is one real obstacle in the way of video replays.

Users would need some training to help them avoid misinterpreting what they see in the playback. That training would help keep replays in the helping-not-hindering category.

Blake

Re: Video Review Needs to Happen Now
 

I posted in a separate thread, but if you missed it: This is what we had to say about video review on Recap

Re: Video Review Needs to Happen Now
 

Yeesh - nitpicking a bit here. The system we have would make it impossible to 'cut, crop, shoot a deceptive angle' - you get what you get with it. There's no time to edit the video because you're immediately pulling it up after it was recorded. If there's not enough evidence to overturn it, the call stands. Simple.

A lot of people thinking it's not even worth trying because we can't get 100 angles and 4K resolution here. The idea is to provide the referees one more tool to help them out. It won't be perfect but it will be better than nothing, I can promise you that.

Re: Video Review Needs to Happen Now
 

This is where contention is stemming from. The adverse effects may well be worse than not having a system (or investing those same resources* in alternative means of improving calls). The most debated call here is crossings. A fixed angle, full-field shot is pretty terrible at judging whether or not a team completed a crossing successfully. Those angles do a poor job at determining if a wheel (or other component) is still in contact with the outer works, if a team fully stopped touching a sallyport door, or seeing into the many blind spots around the defenses this year.

Moreover, you can't simply making scoring adjustments. When a team spends 30+ seconds of a match making sure they get a crossing that counts, that's all sorts of score impacts beyond the simple points. When a ball isn't registered properly on the tower, that has tons of impact for both alliances regarding capture strategy. Almost any circumstance where video replay is warranted also warrants a replay of a match. Even if you limit challenges to the eliminations, and only 50% of alliances use theirs, that's still 4 extra matches in the schedule. With a 7 minute cycle, at best that's 28 additional minutes added to the schedule (and likely more once you factor in 6 minute field timeouts for teams in consecutive matches).

Team experience is an important factor, but an event running on time is a part of team experience. Anyone who's been in FIRST for more than a few years has witnessed a team being called up for an award, only to find out they already left. One of the most constant complaints regarding CMP and Einstein is how late they run. On the rare occasions that events run ahead of time, you'll find plenty of posts on CD congratulating the volunteer crew for an well executed event.

*For example, one poster here is adamant about adding another key volunteer to this process. What if instead of that volunteer being a replay analyst, they were actually just another ref on the field, helping get the calls right the first time?

Re: Video Review Needs to Happen Now
 

I just wanna point out that this argument has been taking place since at least 2005.

Don't believe me? Check out this thread:
http://www.chiefdelphi.com/forums/sh...737#post415737
If you ask me, this is sort of a :deadhorse:

Re: Video Review Needs to Happen Now
 

Technology has come a long way in 11 years. Back in 2005 I would have completely dismissed this notion. Today it's very achievable for a fraction of the cost.

There's a reason you don't see many videos from that time and if you do it looks like it was recorded using a potato.

If we're serious and FIRST is serious about delivering a product that excites young people, we need to get serious about high quality video being a staple at all of our events.

Re: Video Review Needs to Happen Now
 

True, but I was just pointing out that this isn't a new argument.

Re: Video Review Needs to Happen Now
 

Agree - but it's not beating a dead horse. We should revisit topics like this from time to time as things become cheaper, easier and more obtainable.