Video Review Needs to Happen Now

[patar8746]

Originally Posted by gblake

Folks you can PM me if you care to. A longer post by me would repeat what other experienced people already wrote.

Whatever faith people put in my opinions should simply be added to the "No" column, if anyone is keeping score.

Hopefully this added context explains the purpose of my earlier, succint, no-soapbox, horse-is-already-dead post.

Blake

Horse is both alive and dead until we open the box. We should get this tested and if we're wrong we're wrong, if not FIRST can be made better. Why not take that chance?

[gblake]

Originally Posted by patar8746

... Why not take that chance?

For all the reasons other people I agree with have already explained accurately.

[patar8746]

Originally Posted by gblake

For all the reasons other people I agree with have already explained accurately.

I've seen reasons it might not work, but nothing refuting trying the system over the off-season

[Ryan Dognaux]

Originally Posted by gblake

For all the reasons other people I agree with have already explained accurately.

From what I've read the main arguments against in-house video replay (no external video, that opens a can of worms) are cost, time and "this is the way it's always been done."

I look forward to proving all of these wrong when we run a pilot of this at the Gateway Robotics Challenge in October. Expect a whitepaper that addresses all of the previously mentioned concerns.

Remember everyone - tons of people said the district system would never work in FIRST. Did Michigan take no for an answer? Now districts are the main path forward for FRC. There are a million reasons to say no, I'm proposing a solution that makes it easy to say yes, let's try this.

[caume]

I have had experience with refs making the wrong call, effecting the winner of the match in 2014. I was driving for 3324, the Metrobots, in the semi-finals against 16. In the first match, 16 won, but we put up a good fight. In the second match, I actually managed to defend enough against them to win that match, however, they went to the judge's table afterward, and somehow convinced them that we cheated/broke a rule, so they redid the match entirely, when video evidence would have proven we did nothing wrong. The third (second official) match, 16 won, and our season was over. I am still extremely proud of beating mentor built, swerve drive with pneumatic shooter, with a student built mecanum drive with a sketchy kicker, and it's really unfortunate to not even acknowledge officially that we won that match.

That being said, I still think video evidence should not be shown to refs. If a video proves a ref wrong, people begin to trust the ref less and less, just ensuring more drama and chaos. I know the feeling of refs making mistakes, and it is extremely frustrating and makes you hate the system at the time, but it's really the most logical thing to do. Videos don't always show what happens, such as a drawbridge blocking a view, so it can and will just bring up more controversy.

[Sam_Mills]

Originally Posted by caume

I am still extremely proud of beating mentor built, swerve drive with pneumatic shooter, with a student built mecanum drive with a sketchy kicker, and it's really unfortunate to not even acknowledge officially that we won that match.

Oh boy...

[caume]

Originally Posted by Sam_Mills

Oh boy...

What's the problem?

[rzoeller]

Originally Posted by caume

What's the problem?

My issue with it is that the post comes across as accusatory unnecessarily, and makes assumptions about another team's robot and team as a whole without any sort of validation or proof.

[Jessica Boucher]

Ok guys. We're taking a breather for now.

2:25pm EDT: Reopened. Please take extra care with your posts.

[IronicDeadBird]

Originally Posted by Jessica Boucher

Ok guys. We're taking a breather for now.

2:25pm EDT: Reopened. Please take extra care with your posts.

I've been waiting for this.

So instead of jumping to video review I think the FIRST needs to make the fields smarter overall. If they moved the burden of decisions away from Ref's I would think most people would be happy. Adding moar sensors to the field and having the field keep track and the refs verify would make for a more accurate game overall, however this is a slow, expensive process. For a video system IMO the most perfect awesome amazing thingy would be SOLOSHOT. Its a camera, a smart tripod, and a tracking device which has the tripod constantly aim the camera at the tracking device. The issue is one of these setups could cost over $500 and you would need one per robot MINIMUM. This is $3,000 dollars added onto pricing which is a lot for a system that might not even be necessary.
FIRST should always be moving to improve games and how they are run, but the last thing I want to see them do is implement an expensive solution to a problem only for it to not work or even generate more problems. Instead of jumping to the biggest change maybe we should implement more little changes to quality of life.

Big thanks for crowd control though!

[Lil' Lavery]

With regards to field automation, I would like it... provided it works. FRC games don't exactly have the greatest track record when it comes to scoring automation. The 2006 goal sensors had a slew of issues, and many (all?) events verified scores with a human scorekeeper (delays were introduced after autonomous to ensure autonomous scores were correct). The 2010 goal sensors experienced much lower volume, and were thus better performing in general, but there were still cases of sensors not working and humans being used to confirm scores. In 2011, there were quite a few threads covering the issues regarding the sensors on the poles registering minibots. As far as I remember, 2012's automated scoring has been easily the most accurate (ironically, so much so that thereal time scoring was removed from the screens of elimination matches to heighten excitement). 2013's weight sensors once again had issues resulting in human confirmation of scores and untrustworthy Real Time Scoring.

A lot of that is game design, but as was pointed out earlier in the thread, making a game that is easy to score/referee often results in a game that people don't view as exciting. Excitement comes from constant action across the field, and that same constant action is what makes games more difficult to officiate. Often, this can hold true even for automated scoring (see the issues with ball jams in 2006, DOGMA penalties in 2010, and minibot sensor viability in 2011).

Originally Posted by Tristan Lall

Not much has changed in 11 years. People are still basically arguing that because it is impractical to have a perfect replay system, every replay system is detrimental or infeasible. But the point isn't to be perfect; the point is to be better.

I'm contesting that the numerous cans of worms opened by video review aren't going to make things better. I can all but guarantee that the first case of a blind spot in video is going to result in a thread like this one, where a party objects to not being allowed to use their teams' video evidence to support overturning the call. Even worse is when two camera angles show contradictory views. Further still, instant replay does nothing to fix judgement calls, and quite frequently exacerbates the situation. I've seen plenty of open hostility towards replay officials this season in the NHL, along the lines of "if they're not going to get the call right, why bother having instant replay?" (Only less polite).

I find the dystopian vision of large teams will full video replay crews in the stands attempting to find ways to reverse the outcome of a match to influence the standings in their favor rather unsettling. And given the lengths team already go through to find an advantage, including both video scouting systems and attempting to shift the meta strategy of tournaments to influence rankings, I don't find this vision particularly far fetched.

Originally Posted by Tristan Lall

This is clearly an unreasonable way of looking at the problem. What if the effect was subtle or not yet realized (e.g. the student doesn't rejoin the team the next year, and instead joins the chemistry club, which might, in a couple years, lead them to get a degree in science not engineering)? How would you propose to find such a student, given that this probably isn't a metric that anyone tracks?

If it's so clearly unreasonable, than perhaps people should stop using this example as to why FIRST needs to improve [aspect X] of the competition experience. This hypothetical has already been brought up in this thread, and many similar threads regarding perceived shortfalls of the competition aspects of FRC. So long as people cite students becoming so discouraged by a bad call that they chose not to go into a STEM field, I will continue challenging them to find me an example.

[Ryan Dognaux]

Originally Posted by Lil' Lavery

I'm contesting that the numerous cans of worms opened by video review aren't going to make things better.

Scenario A: The head referee makes what they thought was the correct call based on the evidence at the time, but later finds out the call was actually wrong. The call was match deciding and one alliance goes home denied of an event win.

Scenario B: The head referee makes what they think is the correct call. One alliance uses their once per elimination tournament challenge card requiring the head referee to look at the match with the captains from the alliance to explain his reason for the call. This also gives ample opportunity for the challenging alliance to provide specific rules or Q&A's to support their challenge. A match replay is issued if the head ref confirms & agree with the challenge's claims or the results stand as called if the head referee doesn't seem evidence to overturn the call.

There are definitely times where something like this could have came in handy. Look back to San Diego last year where the incorrect call was made and was admitted to be wrong once the Q&A was produced. Maybe having that challenge would have given them a moment to find the Q&A and take another look at the situation?

My point is it's worth taking a look at and talking about. This could give referees another tool to help them out when making very tough calls. Why wouldn't we want to help our referees out?

[gblake]

Folks,

FRC/VRC/FTC/VIQ matches don't proceed in little increments the way many big league sports do (USA football downs, baseball batters & pitches, Cricket ___s, Tennis volleys) .

And, they don't have long stretches of continuous action like many other big league sports (Nascar, Soccer, Basketball or Hockey).

Because of the implications of those differences, I believe that as a practical matter, untangling the spaghetti of knock-on effects that many (most?) overturned calls would create, would be a nightmare of ambiguity.

Except for the last N seconds of a match (where N is very small), everything that affects the score has an effect on how teams and alliances play the games. Neither alliance could argue that an overturned call from early in a match doesn't effect both alliances. In those many cases, I think there would be little a ref could do other than throw their hands up and initiate a replay.

I believe that Video replay successes in big-league sports are unlikely to translate into FRC successes. I predict that if video replay in FRC is successful, it will be because an FRC method is created, not because a big-league method is copied. I also believe that a successful FRC method will be a purple squirrel for most, and the great white whale of some.

I'll look forward to hearing from folks who attempt to use video replays during the upcoming off-season, to see what effect these differences create between what folks are imagining, and what is actually going to occur. I'll bet a beverage that the gulf between ambition and accomplishment is pretty wide.

Unless/until we have some experimental results to digest, I think our current posts are only making 11-12 years of "rubble" bounce.

I'm willing to stop bouncing the rubble, and wait for the results.

Blake

[rich2202]

Originally Posted by Ryan Dognaux

Scenario A: The head referee makes what they thought was the correct call based on the evidence at the time, but later finds out the call was actually wrong. The call was match deciding and one alliance goes home denied of an event win.

Scenario B: The head referee makes what they think is the correct call. One alliance uses their once per elimination tournament challenge card requiring the head referee to look at the match with the captains from the alliance to explain his reason for the call. This also gives ample opportunity for the challenging alliance to provide specific rules or Q&A's to support their challenge. A match replay is issued if the head ref confirms & agree with the challenge's claims or the results stand as called if the head referee doesn't seem evidence to overturn the call.

There are definitely times where something like this could have came in handy. Look back to San Diego last year where the incorrect call was made and was admitted to be wrong once the Q&A was produced. Maybe having that challenge would have given them a moment to find the Q&A and take another look at the situation?

My point is it's worth taking a look at and talking about. This could give referees another tool to help them out when making very tough calls. Why wouldn't we want to help our referees out?

What about scenario C: red alliance successfully challenges to get extra points to wint the match. Then blue alliance challenges a different call to get extra points to re-win the match.

[Lij2015]

Originally Posted by caume

What's the problem?

You absolutely DO NOT want to re-open this can of worms, I''ll leave it at that.