Video Review Needs to Happen Now

[Lil' Lavery]

Originally Posted by Ryan Dognaux

It won't be perfect but it will be better than nothing, I can promise you that.

This is where contention is stemming from. The adverse effects may well be worse than not having a system (or investing those same resources* in alternative means of improving calls). The most debated call here is crossings. A fixed angle, full-field shot is pretty terrible at judging whether or not a team completed a crossing successfully. Those angles do a poor job at determining if a wheel (or other component) is still in contact with the outer works, if a team fully stopped touching a sallyport door, or seeing into the many blind spots around the defenses this year.

Moreover, you can't simply making scoring adjustments. When a team spends 30+ seconds of a match making sure they get a crossing that counts, that's all sorts of score impacts beyond the simple points. When a ball isn't registered properly on the tower, that has tons of impact for both alliances regarding capture strategy. Almost any circumstance where video replay is warranted also warrants a replay of a match. Even if you limit challenges to the eliminations, and only 50% of alliances use theirs, that's still 4 extra matches in the schedule. With a 7 minute cycle, at best that's 28 additional minutes added to the schedule (and likely more once you factor in 6 minute field timeouts for teams in consecutive matches).

Team experience is an important factor, but an event running on time is a part of team experience. Anyone who's been in FIRST for more than a few years has witnessed a team being called up for an award, only to find out they already left. One of the most constant complaints regarding CMP and Einstein is how late they run. On the rare occasions that events run ahead of time, you'll find plenty of posts on CD congratulating the volunteer crew for an well executed event.

*For example, one poster here is adamant about adding another key volunteer to this process. What if instead of that volunteer being a replay analyst, they were actually just another ref on the field, helping get the calls right the first time?

[alephzer0]

I just wanna point out that this argument has been taking place since at least 2005.

Don't believe me? Check out this thread:
http://www.chiefdelphi.com/forums/sh...737#post415737
If you ask me, this is sort of a

[Ryan Dognaux]

Originally Posted by alephzer0

I just wanna point out that this argument has been taking place since at least 2005.

Don't believe me? Check out this thread:
http://www.chiefdelphi.com/forums/sh...737#post415737
If you ask me, this is sort of a

Technology has come a long way in 11 years. Back in 2005 I would have completely dismissed this notion. Today it's very achievable for a fraction of the cost.

There's a reason you don't see many videos from that time and if you do it looks like it was recorded using a potato.

If we're serious and FIRST is serious about delivering a product that excites young people, we need to get serious about high quality video being a staple at all of our events.

[alephzer0]

Originally Posted by Ryan Dognaux

Technology has come a long way in 11 years. Back in 2005 I would have completely dismissed this notion. Today it's very achievable for a fraction of the cost.

There's a reason you don't see many videos from that time and if you do it looks like it was recorded using a potato.

True, but I was just pointing out that this isn't a new argument.

[Ryan Dognaux]

Originally Posted by alephzer0

True, but I was just pointing out that this isn't a new argument.

Agree - but it's not beating a dead horse. We should revisit topics like this from time to time as things become cheaper, easier and more obtainable.

[alephzer0]

Originally Posted by Ryan Dognaux

Agree - but it's not beating a dead horse. We should revisit topics like this from time to time as things become cheaper, easier and more obtainable.

Honestly I just wanted an excuse to use that emoji

[gblake]

Originally Posted by Ryan Dognaux

Technology has come a long way in 11 years. Back in 2005 I would have completely dismissed this notion. Today it's very achievable for a fraction of the cost.

There's a reason you don't see many videos from that time and if you do it looks like it was recorded using a potato.

Yes - Technology has changed - but trigonometry, basic game theory math, the number of hours in a day, and human nature haven't.

Few arguments against using video replays rest on technological foundations (some do, but the majority do not).

I look at this as one of those circumstances where "technology" can either be the final puzzle piece that enables great things, or be the shiny trickster that seduces users into great folly. We already know I lean toward the trickster side of the spectrum for this topic.

In one important sense, if technology is truly no longer a problem, it shouldn't appear much (except as minor footnotes) in this thread. Instead our conversation should be focusing on the rest of the strong arguments that have existed since 2005.

Let's hope enough off-season experiments produce enough repeatable, hard-evidence measurements against important criteria (not collections of fuzzy anecdotes from a few inconsistent, small sample-size, experiences) to move the needle in one direction or the other.

Blake
PS: Clear (1976) evidence that imagery doesn't lie.
What are Dave Lavery's rovers really doing up there???

[gblake]

Standing this topic on its head ...

What if inexpensive video (and image processing) became the foundation for both most of the scoring and some of the penalties, in future FRC games?

I think it is technically feasible (I'm being seduced by the "technology trickster") to design interesting FRC games, and to equip FRC fields, to permit/do this.

Cameras would track the state of objects, the location/movement(s) of some objects, and the locations of robots.

Humans would still be needed to enforce some rules (certainly, some penalties), and/or to step in if the field equipment/detectors misfire in some way.

After 2-3 years of behind the scenes R&D, would introducing games designed with this approach in mind, along with introducing fields that attempt to implement those games, be a net improvement or a lead balloon?

Blake
PS: Whatever the answers/opinions, it will be interesting to ask ourselves, "Why?", and then bring the answers to bear on this thread's original topic.

[Ryan Dognaux]

I'm bowing out of this discussion for now. There's a lot of doubt, negativity and misinformation about what video review could be. These opinions won't change until we see some positive examples of it being used during the off-season. I absolutely cannot wait to implement it and see what happens (hopefully proving the naysayers wrong.)

Remember students - just because someone tells you something won't work doesn't mean you shouldn't go and find that out for yourself, especially if you think you can make it work. Often times the people telling you it won't work have biased opinions from bad experiences of their own or just don't know what they're talking about.

[Foster]

I can see an adjunct competition where you are given the game and you need to design the scoring system for it. You get to deploy your scoring system with others at the event. Winner is the one that matches the humans the most at an event.

Side benefit is that in case of a question about the score they can go to the computer scores and see what they think.

Ryan, you and the others are on the right track.

Remember students - just because someone tells you something won't work doesn't mean you shouldn't go and find that out for yourself, especially if you think you can make it work. Often times the people telling you it won't work have biased opinions from bad experiences of their own or just don't know what they're talking about.

This is CD gold. For the last decade I've often been surprised and impressed with what roboteers can do. I never discount a motivated roboteers ability.

I for one welcome my new computer scoring overlords.

[EricH]

Originally Posted by Foster

I can see an adjunct competition where you are given the game and you need to design the scoring system for it. You get to deploy your scoring system with others at the event. Winner is the one that matches the humans the most at an event.

Side benefit is that in case of a question about the score they can go to the computer scores and see what they think.

That is NOT a bad idea at all. Could be an interesting offseason competition: Given game X, design a scoring system with constraints A, B, and C--most likely portability (including ease of assembly into the field), durability (including fault tolerance/"idiot-proofing"), and cost. Must match or beat humans.

[gblake]

Is anyone in the mood to put to together an accurate enough model of this year's field (or a previous one), then add some similar fidelity robot and game piece models, and then explore the accuracy and obstruction (and misinterpretation of camera imagery) topics connected to using replays to answer questions?

I'm betting someone among us has the models, motivations, domain-knowledge, and free-time necessary.

Any takers?

Blake

[gblake]

Folks,

This interesting article can probably be used to bolster the positions pn all sides of this topic. It has some info about how sensors and computing are (and aren't) being used to supplement/replace human refereeing in some big league sports.

I enjoyed reading it.

Blake

[EricH]

Originally Posted by gblake

Folks,

This interesting article can probably be used to bolster the positions pn all sides of this topic. It has some info about how sensors and computing are (and aren't) being used to supplement/replace human refereeing in some big league sports.

I enjoyed reading it.

Blake

Did you forget a link?


That being said, for a number of reasons automated scoring can be very useful. Ball scoring has been almost completely automated on several occasions ('10, '12, '16--'06 was automated but required manual backup due to system design). It's GREAT to not have to worry about scoring balls with the barrage that most alliances can unleash. OTOH, the DOGMA penalty in '10, which was automatic, was kind of a nuisance--and on at least one occasion, a team did exactly what they were supposed to and still got a penalty series from that (ball missed the counter on the way down). '08 had automatic robot lap counting, if that actually worked--I don't recall hearing anything either way on that.

But there are still a number of cases that automation would make particularly "interesting". Here's a couple related ones: Could a sensor detect whether a boulder crossed legally this year, under the case where a robot did not finish (or start) a Crossing but the boulder went across? I think so... but I would argue that that would have to be a very good sensor system to detect that, particularly live on the field. The question is, is the added cost* worth it for a system that would only need to be used some small portion of the time, for a non-profit competition system? And that I would say that it depends.

Many things can be automated. Some things cannot be automated. I happen to like the MLB replay model where it's basically a second set of eyes, detached from the game in question, looking at the various viewpoints, and not everything is actually reviewable.

*Not necessarily money--time, training, etc are all factors in the cost.

[Hitchhiker 42]

Originally Posted by EricH

Did you forget a link?


But there are still a number of cases that automation would make particularly "interesting". Here's a couple related ones: Could a sensor detect whether a boulder crossed legally this year, under the case where a robot did not finish (or start) a Crossing but the boulder went across? I think so... but I would argue that that would have to be a very good sensor system to detect that, particularly live on the field.

Just as a design thing (not really related, but I'm interested), it'd probably be easier to put a location sensor within each bolder. Location can be accelerometer/gyro combo with a tiny controller. Of course, I'm not a fan of making those boulders even more expensive...