Call Inconsistencies Between Regionals

[Brandon Holley]

I agree with many of the posters in this thread who agree that "they are just volunteers" excuse is no longer valid.

Something MUST be done, because what would a major league sport be if the crowned champions were crowned (or not crowned) because of a poor referee decision?

Unfortunately, reffing a sport, or being an umpire or any of this, is not usually something that can be picked up over a online courses test. I was an umpire in little league baseball, which believe it or not can have its fair share of confrontation. There were umpires that simply could not cut it. They didn't have the eye, or the "nack" to do it. Even though they are volunteers, in a game as complicated as that, you may have to look at getting professional help. The head referee cannot look at the entire field at once...



My suggestions:
1. There absolutely must be a way to train these referees better than just a test. Corey balint had suggested a video of possible scenarios played out via animation or even people robots like done in the kickoff examples, and do this pre regional events.

2. Even if it means raising the regional fee a hundred dollars extra per team, it would be worth it to have all the referees in person, together, going over the rules and some of the scenarios that might occur. Teams pay so much money to go to an event, its simply unfair to not have consistent refereeing event to event. Even if this means paying for these referees to fly around the country to keep it consistent.

3. The last suggestion I have isn't really a suggestion, but more of an observation of things that I've seen at competitions that unease me. Specifically in the case of talking to referees in the "box", I have seen white shirted volunteers turn students away without giving them the chance to state their case. This is obviously unacceptable. The box is there for a reason, it is there so that a student can talk to a referee about what had just occurred in a match. There is no reason for a disconnect between the students and the referees. Regardless of whether its about the robots or not, its always about the students. If it means the schedule needs to wait for 45 seconds, then let the schedule wait. Everyone is supposed to be there for the kids, so lets actually let the kids talk. Going from FLL in '01 to FRC '02 to '05 to FRC college mentor I have definitely seen the disconnect between the students/teams and FIRST as a unit. It saddens me to see students turned away from expressing their opinion, this isn't what FIRST is about.

[Bob Steele]

In looking at all of this an idea occurred to me.

What about taking another page from the new rulebook of sports.

The challenge flag... Each team would be allowed one per event.
Throw it for little things.... waste it....
Save it for something really important.... perhaps a change that would make a difference in a match score and tip the scales...

Perhaps use it as a third tie breaker for qualifications...

Perhaps, Each alliance would have one (only one) of these for the elimination rounds.

Of course this would not be used this year.... just something to kick around for next year...

I applaud the referees for the work they have done this year....
This is an extremely difficult game to referee compared to many others we have played.

thanks
Have a great year!!!

[David Brinza]

Originally Posted by Brandon Holley

I agree with many of the posters in this thread who agree that "they are just volunteers" excuse is no longer valid.


3. The last suggestion I have isn't really a suggestion, but more of an observation of things that I've seen at competitions that unease me. Specifically in the case of talking to referees in the "box", I have seen white shirted volunteers turn students away without giving them the chance to state their case. This is obviously unacceptable. The box is there for a reason, it is there so that a student can talk to a referee about what had just occurred in a match. There is no reason for a disconnect between the students and the referees. Regardless of whether its about the robots or not, its always about the students. If it means the schedule needs to wait for 45 seconds, then let the schedule wait. Everyone is supposed to be there for the kids, so lets actually let the kids talk. Going from FLL in '01 to FRC '02 to '05 to FRC college mentor I have definitely seen the disconnect between the students/teams and FIRST as a unit. It saddens me to see students turned away from expressing their opinion, this isn't what FIRST is about.

Would it at least be acceptable to allow coaches to front off these volunteers that just want the issues to "go away" so the show can go on? If directed to leave the challenge box by an adult volunteer, many students will do so because it's GP to respect authority. Let the students get access (for however brief) to the referee to present their case.

[Brandon Holley]

Originally Posted by David Brinza

Would it at least be acceptable to allow coaches to front off these volunteers that just want the issues to "go away" so the show can go on? If directed to leave the challenge box by an adult volunteer, many students will do so because it's GP to respect authority. Let the students get access (for however brief) to the referee to present their case.

David,

You hit the nail on the head in my eyes. As a high schooler on the team I felt that if an adult volunteer told me not to be somewhere, I should abide and walk away.

However now as a mentor, if I saw one of my students turned away, my first instinct would be to politely confront the volunteer to find out why they were turned away.

While i see the point why adults can cause heated conflict, I do see a place for them in just GETTING the student to the REFEREE. I have full faith that any of my kids could explain what the situation was we were contesting, however if my kids cannot even get to a referee this is moot.

[Richard Wallace]

Originally Posted by David Brinza

Would it at least be acceptable to allow coaches to front off these volunteers that just want the issues to "go away" so the show can go on? If directed to leave the challenge box by an adult volunteer, many students will do so because it's GP to respect authority. Let the students get access (for however brief) to the referee to present their case.

Which volunteers do you mean?

Only the Head Referee can make a team's post-match clarification request "go away". Head Referees I have worked with prefer to interact with pre-college team members who are standing in an area that has been designated (e.g., with tape on the floor) for that purpose.

Originally Posted by 2008 FRC Manual, Section 9.6.2

9.6.2 Referee Interaction Rules

<T03> The Head Referee has the ultimate authority on the field during the competition. THE HEAD REFEREE RULINGS ARE FINAL! The referee will not review recorded replays under any circumstances.

<T04> If a team needs clarification on a ruling or score, a pre-college student from that team should address the Head Referee after a field reset has been signaled. Depending on timing, the Head Referee may postpone any requested discussion until the end of the subsequent match. Head Referees will only discuss calls, scores, penalties or match outcomes with pre-college team members.

[David Brinza]

Originally Posted by Richard

Which volunteers do you mean?

Only the Head Referee can make a team's post-match clarification request "go away". Head Referees I have worked with prefer to interact with pre-college team members who are standing in an area that has been designated (e.g., with tape on the floor) for that purpose.

I'm referring to non-referee volunteers trying to quickly usher the teams away from field-side after their match, sometimes before the scores are announced. We know that it can be a challenge to keep on schedule, especially if there are field faults, repairs, referee discussions, etc.

Teams are entitled to get clarification of rulings from the referee in accordance with <T04> and <G53>. This needs to be made clear to all of the field-side volunteers, who should point them to the challenge box. The teams need to use discretion in seeking clarification, no whining about a judgment call, but understanding what action triggered the penalty should be fine. Most certainly, questions about correct application of the rule should be explained.

[jgannon]

Originally Posted by Richard

Which volunteers do you mean? Only the Head Referee can make a team's post-match clarification request "go away".

Officially, yes, but sometimes some headstrong field personnel feel like they can usher you away from the box because you're slowing things down. This happened to us once this season, and this same volunteer told me later that we were lucky she didn't tell the judges about the incident, because she felt that our resistance to her demands should exclude us from eligibility for awards. It doesn't surprise me to hear that others are being turned away from the challenge box by people other than the head ref.

[Ryan Dognaux]

Originally Posted by jgannon

Officially, yes, but sometimes some headstrong field personnel feel like they can usher you away from the box because you're slowing things down. This happened to us once this season, and this same volunteer told me later that we were lucky she didn't tell the judges about the incident, because she felt that our resistance to her demands should exclude us from eligibility for awards. It doesn't surprise me to hear that others are being turned away from the challenge box by people other than the head ref.

In the event that this happens, it's almost worth it to try and talk to the volunteer coordinator and report these remarks. A crew volunteer should have no control over the awards selection, that's for the judges to decide only. Stories like this make me angry as most volunteers are amazingly cool people who are there to make your regional run smoothly. I guess there's room for bad apples even in FIRST.

Originally Posted by Paul Copioli

1. While the NASCAR theme is cool, forget about this "look at the robots right at zero nonsense." Just assess the points when everything comes to rest: robots and balls. Why? It makes it easier for the refs and the spectators.

I can't tell you how much I agree with this point. After attending BMR, I saw so many matches where balls were knocked off the overpass, but remained as counted on the overpass because they didn't hit the ground in time. This is lame. It confuses the crowd and even threw me for a loop at first. Little things like this make a big difference for the "wow factor" of an event. Many ends of matches that could've been exciting were quickly extinguished because of this flaw.

[Ken Streeter]

Originally Posted by Ryan Dognaux

After attending BMR, I saw so many matches where balls were knocked off the overpass, but remained as counted on the overpass because they didn't hit the ground in time. ...

Hmm. In cases of unrestrained motion, the trackball positions are scored based upon where they do finally come to rest, not where they are when the clock reaches zero. If BMR was scoring all trackballs based upon where they were when the clock reached zero, that was incorrect based upon Team Update #13, which resulted in rule G14 being updated as follows:

Originally Posted by Section 7, The Game, Rev G

<G14>
When the MATCH ends, each TRACKBALL that is at least partially supported by the OVERPASS and not in contact with any ROBOT of the same ALLIANCE will earn a 12-point bonus. If a TRACKBALL is in unrestrained motion (i.e. not in contact with a Robot) when the clock reaches zero, its contribution to the score will be based on when it comes to rest.

[Ryan Dognaux]

Originally Posted by Ken Streeter

Hmm. In cases of unrestrained motion, the trackball positions are scored based upon where they do finally come to rest, not where they are when the clock reaches zero. If BMR was scoring all trackballs based upon where they were when the clock reached zero, that was incorrect based upon Team Update #13, which resulted in rule G14 being updated as follows:

Maybe they weren't counting them because they were knocked off after the clock hit zero. Not exactly sure, but it could be the case. At least in one of our matches, our partner (1018 I think) knocked a ball off right at the end and it was still counted as being on the overpass. The officiating was pretty consistent either way. Tons of penalties of course.

[IndySam]

Originally Posted by Ryan Dognaux

I can't tell you how much I agree with this point. After attending BMR, I saw so many matches where balls were knocked off the overpass, but remained as counted on the overpass because they didn't hit the ground in time. This is lame. It confuses the crowd and even threw me for a loop at first. Little things like this make a big difference for the "wow factor" of an event. Many ends of matches that could've been exciting were quickly extinguished because of this flaw.

I didn't see that, what I saw were balls that were knocked off right at the buzzer and were rightfully counted because they were in contact with the robot knocking them off at the time of the buzzer.

[Ken Leung]

Originally Posted by Anne Shade

2. Provide for a method of evaluating the referees, specifically the head referees. Training is only half of the equation. Training provides the knowledge but is it being applied correctly? Referees in all professional sports are evaluated by their organization after all games. If too many mistakes are made, they are re-trained or released. Where is the evaluation for FIRST referees? How do we know that the training received was effective if their is no evaluation of performance? Some people are just not cut out for the job but we have no way to determine that until something goes grossly wrong.

Originally Posted by Brandon Holley

My suggestions:
1. There absolutely must be a way to train these referees better than just a test. Corey balint had suggested a video of possible scenarios played out via animation or even people robots like done in the kickoff examples, and do this pre regional events.

Perhaps we can take care of both of issues with one stone, maybe 3 issues with one stone. Ever thought about focus groups of FIRST Referees?

The difficulties of using video footages to train refs is an excellent idea, but I don't think animation is going to cut it. We literally need footages from the first week regionals. It's true that there will be no footages until the first week, but that is not that different from how things are now.

The Refs and the GDC go into week 1 without a clear vision of how things will played out, but their vision is refined after that, and refined again after another week, and so on. So, its a matter of how you can capture the first week to help you train the Refs better in the weeks that follow. Besides, there are certain elements that will continue to come back year after year, such as entanglement, tipping, and high speed ramping.

Anyway, back to the focus group. Here is the idea. You look at videos from week one. You look for instances where rule violations occur. And you look for them so much that you begin build up a spectrum of instances between clearly acceptable acts and clear violations. Say you get 5 video clips of ramming that give you an understanding of, on a scale of 1 to 5, what's acceptable ramming and what's unacceptable, and the gray area in between.

Here is how you apply these videos to training and evaluation. During the training, you show 1 clip as an example of clear violation of a certain rule. After the training is over, you show 5 clips of the same rule, and you ask the person in training which ones they feel are violation and which ones aren't. You give them a chance to make mistakes so they have a better understanding of the variation.

Then you show them another 5 clips for another evaluation. After you repeat that once or twice, you should have a good idea whether that person has caught on to what's acceptable and what isn't.

For training purposes, it is very important for the head Referee at the headquarter to decide, on scales of 1 to 5, what is acceptable and what isn't. Once a line is drawn, you drill that line into all referees' head in the weeks that follow. Consistency is what's important here. And you continue to use these clips and newer ones to train and evaluate refs before and after the events, and before and after the years.

Won't be for every rule of course, just the difficult ones.

Here are some side bonuses:

1. You release some of these clips to the teams so they too will get a better understanding of what's acceptable and what isn't.
2. You collect data as people evaluate various clips. The more data you collect, the better you are at understanding, in general, how people are calling certain rules. This understanding will help you decide whether a rule need to be better written, or if the training need to be improved.


Sounds complicated, right? You betcha! There is probably a simplier idea in here somewhere. On the other hand, there are many people watching many videos of many competition during the weeks. Won't be hard to find volunteers whose job is to identify these clips for the GDC and the head Ref at headquarter. Once a system is in place, Refs and refs in training just need to go to the training site and watch some videos.

[Anne Shade]

Ken, I think you have some valid points but the evaluation I was refering to is an evaluation of a head referees' performance at a regional. There needs to be a way to say that a head ref did a good or bad job and to determine who is not right for the job. Tests and questionaires only get you so far in the evaluation of a person's skills and performance.

Part of that evaluation could be team and key regional personel feedback but most really should be video review or in person evaluation. Why not have the committee of championship head referees review random matches from each regional and evaluate the play calling and the refereeing procedures utilized by the head referee? Head referees would receive feedback on how they can improve and those that are not performing up to par can be replaced.

This is the second part of training. Without the evaluation, the training has very little value.

[GaryVoshol]

Originally Posted by Anne Shade

There needs to be a way to say that a head ref did a good or bad job and to determine who is not right for the job. Tests and questionaires only get you so far in the evaluation of a person's skills and performance.

Part of that evaluation could be team and key regional personel feedback but most really should be video review or in person evaluation. Why not have the committee of championship head referees review random matches from each regional and evaluate the play calling and the refereeing procedures utilized by the head referee? Head referees would receive feedback on how they can improve and those that are not performing up to par can be replaced.

This is the second part of training. Without the evaluation, the training has very little value.

And how do you know whether that is or is not happening now? (I don't mean to be harsh here. I'm just pointing out that we don't know everything that goes on in the background.)

Head Referees have been replaced in the past.

[Anne Shade]

Gary,

I have been a referee for the past three years and a head referee the past two. I have not been a head ref this year (started a new team). Video review has not occured in the past. I've spoken to Aidan on this before.

Head Referees have been replaced in the past, that is true. They were replaced when the situation got so bad that FIRST had no choice but to act. We need to diagnose problems before that point. Hence my comment, evaluations are the next step after training.