Video Review Needs to Happen Now

[EricH]

Originally Posted by cgmv123

If the score is 100-45, the losing alliance challenging to get an extra 5 or 10 points is a waste of time (assuming total score is not part of the ranking system). I think requiring the review have an effect outside of just the match is not unreasonable.

I agree on that. Anything from missing one breach ranking point on up is reasonable. If total score is part of the ranking system, that could be re-thought, but I'd suggest any score changes less than X amount (determined by the game and ranking, but let's call it one penalty of TBD type) would be unreviewable if the score differential was greater than 2X.

I disagree on the Finals being an automatic review, primarily because that means a minimum of 3 minutes where any refs involved aren't doing their between-match stuff (traffic control, overall monitoring), and also because if there's something tough in the Finals every ref is going to be in the huddle discussing the calls--we want to get the calls right the first time. What I'd do instead would be to reset challenges (I'm in favor of LIMITED challenges, and I'll explain why in a minute) to full for finals regardless of prior usage.


The reason I prefer limited challenges (probably 1/alliance in playoffs, with a second if the first is successful) is that by the second challenge from the same alliance, if the Head Ref hasn't shuffled the crew, he or she probably needs to. And it may be obvious on the reviews that one ref or another needs to be shuffled to break or another field position if possible. For those that aren't refs, the ref crews tend to find their weak links quickly and strengthen them as needed. If there's two challenges, chances are that there's a ref that needs more strength--or it's possible that the alliance is just trying to game the system.

[cgmv123]

Originally Posted by Siri

For quals, the requirement that it change one of the aggregates in the ranking algorithm (though I'm not positive that's what cgmv means) would be pretty loose in most years. This seems like a reasonable standard, since it's easy enough to know beforehand ("that would raise our auto score total") and speaks directly to the impact quals are supposed to have.

My intent was that ranking points (or the first order ranking sort in subsequent games) would need to be affected. For Stronghold, this would have meant changing the outcome of the match or awarding a point for a missed breach/capture. I'm reconsidering this now, because you correctly point out that the ranking tiebreakers can also matter. I'm not sure where to draw the line, though, because for Stronghold, the only scores that don't factor into at least one tiebreaker are foul points, and I don't think allowing a challenge to every single Qualification match score is worth it. I'm fine with Eliminations going deep into the evening if it takes that much time to get all of the calls right, but I think Qualifications should be expected to stick to the schedule a bit more.

Not sure how I feel about requiring every member of a qual alliance to still have their challenge coupon. I think I'd be okay with "any" instead of "all". It likely means more challenges, but then normal teams can't be burned by being randomly assigned a trigger-happy alliance partner who wasted their coupon on match 1.

Yeah, I wasn't sure how fair/unfair it was for teams to be able to jeopardize future alliance partners by losing their challenges early. I do think it's fair to limit challenges in qualification matches to blatantly obvious errors with ranking implications, and the rules I proposed are written with that in mind. They're just a suggestion that I expect the GDC (Hi GDC!) to tweak before placing in the 2017 manual. :wink:

Originally Posted by EricH

I disagree on the Finals being an automatic review, primarily because that means a minimum of 3 minutes where any refs involved aren't doing their between-match stuff (traffic control, overall monitoring), and also because if there's something tough in the Finals every ref is going to be in the huddle discussing the calls--we want to get the calls right the first time.

The automatic review wouldn't be done by a referee. It would be done by a "replay official" who would "confirm" most of the calls in the match right away and only call over a ref if they see something questionable.

[bdaroz]

Quick thought with regard to the 'must affect ranking points'....

If there is a) a time limit to make the call b) you need all 3 teams on the alliance to agree and c) need the ranking points to change then:

1) Is there enough time as robots are being pulled from the field to coordinate with the other two teams?
2) AND in the event that a 100-90 loss was a missed breech and would have resulted in a match point tie, is there still enough time to find and calculate the tie-breaker while coordinating with two other teams?

End of match is hectic as it is... If a team/alliance is feeling slighted by a call (or non-call), it's even more so. While I agree there should be reasonable limits the challenging team should have sufficient opportunity to avail itself of the rule.

To that end I would change the rule to be that the challenge, if successful, must either change the ranking points awarded, or change the win/loss/tie result of the match, without regard to the tie-breaking formula.

[EricH]

Originally Posted by cgmv123

My intent was that ranking points (or the first order ranking sort in subsequent games) would need to be affected. For Stronghold, this would have meant changing the outcome of the match or awarding a point for a missed breach/capture. I'm reconsidering this now, because you correctly point out that the ranking tiebreakers can also matter.

I'd stick with first-order sort, possibly 2nd-order. Most times the sorting doesn't seem to go beyond 2nd order. 2nd-order sort generally being auto, those would be quick reviews (15 seconds or less).

The automatic review wouldn't be done by a referee. It would be done by a "replay official" who would "confirm" most of the calls in the match right away and only call over a ref if they see something questionable.

And this is where you need to remember something: the replay official is going to need to have at least some referee training. Otherwise, how are they going to know a questionable call? I know we're taking judgement out of it as much as possible, but, for example, when we're looking at crossings, there's an awful lot of nuances to crossings (for robot and for boulder) that sometimes people don't quite grasp--I once had a team complain during practice day after I called them for not finishing a Crossing before returning to the NZ from scoring in the high goal. So if they're going to need to have referee training anyways, then you may as well make them a referee. And if they're a referee, you may as well make them work the field for a few matches, particularly if you're short-handed. So that means that several refs may as well have training on the replay system.

I think the best way to handle replay would be to have the "off" ref in the rotation handle it, or have two "off" refs (one assigned to replay at any given time). Now, finding the refs over and above the field crew can sometimes be difficult. But I think with enough effort, someone could be found... And actually, that would speed up replays a bit--if you've got an off-field referee going through them during a match, then the only thing they need to do is to advise the head ref (NOT a regular ref, BTW, that's another thing--this is a call reversal or not) that this-that-and-the-other is the case, or that he needs to take a look as somebody's asking for a judgement call, or what-have-you.

[scca229]

Originally Posted by cgmv123

I'm not sure where to draw the line, though, because for Stronghold, the only scores that don't factor into at least one tiebreaker are foul points

In Quals, yes, fouls don't factor, but in Playoffs they are the 1st tie-break in a tied score. Cleaner played match wins. [5.4.4]

[Siri]

Originally Posted by cgmv123

My intent was that ranking points (or the first order ranking sort in subsequent games) would need to be affected. For Stronghold, this would have meant changing the outcome of the match or awarding a point for a missed breach/capture. I'm reconsidering this now, because you correctly point out that the ranking tiebreakers can also matter. I'm not sure where to draw the line, though, because for Stronghold, the only scores that don't factor into at least one tiebreaker are foul points, and I don't think allowing a challenge to every single Qualification match score is worth it. I'm fine with Eliminations going deep into the evening if it takes that much time to get all of the calls right, but I think Qualifications should be expected to stick to the schedule a bit more.

We could hedge it and say "changes Nth (likely 1st/2nd) order ranking points or changes rank". Presumably any team looking at a deeper ranking aggregate is on-the-ball enough to determine whether a changed call would move them. It leaves out edge cases where it'd move you closer to jumping someone without actually passing them, but it does have to stop somewhere (I'd argue).

[Foster]

Caveat -- I'm not trying to reopen the battle about if there should be a video review or not. I just wanted to ask a simple question.

There has been much discussion about the technology needed to do video reviews and some teams have said that they have that level of equipment.

Since we are in the final weeks of off season events, are there any events that are planning to allow for video review?

Thanks!

[gblake]

Ryan's folks have one coming up sometime in October.

[EricH]

I asked about doing one at one of the local offseasons, as a pilot. No-go with the planning committee.

On the other hand, they did approve one of my other ideas... If that goes well, I may find myself needing to write a report on it.

[gblake]

Eric H, if the rest of us can volunteer you for some work, maybe the planning committee will let you place a few cameras at overlapping locations so that you could collect data able to shed light on some of the more technical topics raised earlier in this thread.

Topics like "How many points of view are needed to give an accurate record of things like line-crossings, or of contact between robots, or of contact/positions among/of any other physical parts of a match."

And/or maybe produce some screen shots illustrating:
- How much moving object blur/tearing exists in a single frame of the imagery captured by the cameras you happen to use.
- How far robots and game pieces travel between successive frames of the video at the frame rates you try.
- How much real world area each pixel represents in the images you capture (length of the field, worth of the field), given the camera settings you try (see next bullet)
- How much the resolution varies throughout the cameras' depth of field, given the focus & aperture settings, and lenses you choose to use.
Etc.

You would do a little scurrying around, and maybe pose some robots during down time, but your effect on the event would be hardly noticeable compared to a full replay experiment.

Blake

[AllenGregoryIV]

Originally Posted by gblake

Eric H, if the rest of us can volunteer you for some work, maybe the planning committee will let you place a few cameras at overlapping locations so that you could collect data able to shed light on some of the more technical topics raised earlier in this thread.

Topics like "How many points of view are needed to give an accurate record of things like line-crossings, or of contact between robots, or of contact/positions among/of any other physical parts of a match."

And/or maybe produce some screen shots illustrating:
- How much moving object blur/tearing exists in a single frame of the imagery captured by the cameras you happen to use.
- How far robots and game pieces travel between successive frames of the video at the frame rates you try.
- How much real world area each pixel represents in the images you capture (length of the field, worth of the field), given the camera settings you try (see next bullet)
- How much the resolution varies throughout the cameras' depth of field, given the focus & aperture settings, and lenses you choose to use.
Etc.

You would do a little scurrying around, and maybe pose some robots during down time, but your effect on the event would be hardly noticeable compared to a full replay experiment.

Blake

Feel free to analyze any of the video we have posted on our channel, we have two events worth from this summer with three different camera angles in every shot.

https://www.youtube.com/playlist?lis...ff6pk35HJXrr9n

[EricH]

Originally Posted by gblake

Eric H, if the rest of us can volunteer you for some work, maybe the planning committee will let you place a few cameras at overlapping locations so that you could collect data able to shed light on some of the more technical topics raised earlier in this thread.

If the event wasn't tomorrow, that might theoretically be possible. But I'd need to clone myself at least once. My effect on the event is rather noticeable in my current role...

However, if you're interested in the typical camera setup, I've put the stream links to TBA (Fall Classic).

Like I said, there's something else I'm up to that has some relation to this topic, but is in a somewhat different vein. If I remember to write it up afterwards, I'll post it.

[gblake]

Originally Posted by AllenGregoryIV

Feel free to analyze any of the video we have posted on our channel, we have two events worth from this summer with three different camera angles in every shot.

https://www.youtube.com/playlist?lis...ff6pk35HJXrr9n

Not today, but maybe sometime reasonably soon. Doing that would give me a good reason to recreate a good-enough (bare) field model.

What can you tell us about the cameras? Lenses, F Stop, Virtual shutter speed & other "shutter" settings, frame rate, Model number (video chipset), resolution chosen, and, of course, locations & orientations (plus whatever else I'm forgetting)?

Some results can be evaluated reasonably well without that info, some not so much.

Also, I would have guessed that videos posted on the Web for ordinary consumption would have gone through some lossy compression. Is the raw, 100% uncompressed frame-by-frame imagery available for download from the YouTube page at that link (I'm definitely not a YouTube guru)? Thumbs up if it is. If not, do you have a link to a site where I can download some of it?

Blake

[EricH]

Originally Posted by EricH

Like I said, there's something else I'm up to that has some relation to this topic, but is in a somewhat different vein. If I remember to write it up afterwards, I'll post it.

As it turns out, nobody took advantage... But I'll post the write-up anyways, or something like one.

The offer: Any person who could plausibly be a ref within the next couple of years (high school seniors, parents, mentors) was offered a chance to shadow a referee for a few matches. Now, obviously they wouldn't actually be making any calls, but being with the referees and able to ask questions is big for both the people and the refs, as normally their interaction is one or two kids talking to the head ref in the question box. (I also had an "open box" policy as the head ref for the event: The box is where I am, and the person in it is the person talking to me.)

There were no takers.

And I had a team ask me if they could set up a camera to check the crossings, and at least two teams tried to have me look at a replay. I told them all that I couldn't look at a replay--not that they can't set up a camera, mind you, but that I can't look at a replay.

I did have a team or two ask about crossings being counted--my usual response was to turn to the FMS operator and ask for the appropriate sheet for that match (we used a paper scoring system, and we had scorers working with the refs--HQ, take note, refs could use some scorers just to enter data), then check with the appropriate ref(s) and make corrections as necessary.

[gblake]

Hey off-season folks,

What news do you have for us?

What are the new anecdotes?

What are the new hard data?

Blake