Call Inconsistencies Between Regionals

[Swan217]

Originally Posted by Paul Copioli

1. While the NASCAR theme is cool, forget about this "look at the robots right at zero nonsense." Just assess the points when everything comes to rest: robots and balls. Why? It makes it easier for the refs and the spectators.

As far as problems go, I see this as one of the least. On the contrary, I think it's much more exciting seeing if that robot can beat the buzzer to the finish line. (As a robot-scorer referee as well)

Originally Posted by Paul Copioli

2. Don't use referees for the scoring, use scorekeepers. Scorekeepers only need to know what counts and what doesn't with respect to balls and robots crossing. Use 4 scorekeepers per arena.

We already do this. There are 4 referees dedicated to scoring. They do not call penalties or do anything else besides scoring. You may think of them as linesmen.

Originally Posted by Paul Copioli

3. Along with number 2, use 6 referees per match plus the head referee. Each ref is assigned one robot the entire match. They asses every penalty associated with that robot as they are actually watching that robot. The zone reffing makes it almost impossible.

You may want to ask the Great Lakes/Detroit Head Referee about this. In previous years, he had a system where your idea would be the norm. Unfortunately, with all of the problems last year, orders from On High were to have no more than 8+1 referees on the field (including 4 linesmen & Head).

Originally Posted by Paul Copioli

4. Use examples in the rule book. Let's take the multiple configurations rule as an example. A simple "if it looks like the two mechanisms could be used without each other on the field as a moveable robot, then they are not considered mechanisms; they are considered robots" would stop the majority of the debate (there will always be FIRSTers who try to wiggle through every word (aka lawyering) even though they will swear they are not lawyering).

Agreed - examples would make it easier for teams, and harder for lawyers on said teams.

Originally Posted by Paul Copioli

5. Clearly state the reason for the yellow flag. Is it for dangerous play or not? I thought it was reserved for dangerous play. How does an offensive robot get a yellow card while it is being aggressively defended?

<T05> Adequately describes the purpose of yellow flags. The trouble arises when <G#> rules such as <G34> or <G40> blur those lines without any clear middle ground.

Originally Posted by Paul Copioli

7. Don't blame the game of the refs for certain deficiencies in our skill. G22, for example, is a clear rule that is being called as per the rule. Hurdler interference is a penalty that has to be called. You are stopping a team from scoring 10 points, so the penalty is 10 points. It is the "pass interference" of Overdrive. We make every decision based on the rules. If there was a rule against guarding the trackball, they we wouldn't consider it a viable strategy. However, even though we know this is supposed to be an offensive game, we are using the fact that there is not a rule about guarding the trackball as a reason to guard the trackball. Because of us, the rulebook will keep getting bigger.

Agreed - Pass interference = good rule. Guarding trackball = good non-rule & great strategy

Originally Posted by Paul Copioli

8. Those of you complaining about the quality of reffing, become a ref. I will do this next year as I have never reffed beofer and will now start. At Detroit, the reason the reffing is much better than other places I have witnessed is the refs are mostly involved with FIRST teams or are FIRST team alumni. We need to increase the talent level of the referees from within. We must become a part of the solution.

I relish the day when some uppity team coach comes up and yells at referee Paul Copioli for some bogus call he just made. But yes, as I mention in FIRSTruth, the great regionals are the ones where the referees are biased - towards fair play and the love of the game.

[Anne Shade]

There have been attempts to improve refereeing made this year. I agree that the online training and "certification" process was the start of a good idea. At the very least, it forces the referees to look at the rules before they show up to the competition. This should help the head referees come competition. BUt how does it help if the referees are allowed to retake the test 53 times before they pass? At one point, you'll end up guessing the right answers with still no understanding of the rules.

To continue the list of improvements to the refereeing:

1. Limit the number of times the referees are allowed to take the test before they fail (I think this may have already been done but want to make sure it's mentioned in case it has not).

2. Provide for a method of evaluating the referees, specifically the head referees. Training is only half of the equation. Training provides the knowledge but is it being applied correctly? Referees in all professional sports are evaluated by their organization after all games. If too many mistakes are made, they are re-trained or released. Where is the evaluation for FIRST referees? How do we know that the training received was effective if their is no evaluation of performance? Some people are just not cut out for the job but we have no way to determine that until something goes grossly wrong.

3. This has been said already but to reiterate, provide hands on, face-to-face training for all the head referees in FIRST. It should be no different than what FIRST does with FTAs...

4. Give Aidan some help during the game design process for development of the referee process and during the regionals to evaluate performance of the other head referees. Maybe the Championship head refs should be determined at the Championship the prior year so they can form a referee committee to help Aidan? It is way too big of a job for one person.

[Brandon Holley]

I agree with many of the posters in this thread who agree that "they are just volunteers" excuse is no longer valid.

Something MUST be done, because what would a major league sport be if the crowned champions were crowned (or not crowned) because of a poor referee decision?

Unfortunately, reffing a sport, or being an umpire or any of this, is not usually something that can be picked up over a online courses test. I was an umpire in little league baseball, which believe it or not can have its fair share of confrontation. There were umpires that simply could not cut it. They didn't have the eye, or the "nack" to do it. Even though they are volunteers, in a game as complicated as that, you may have to look at getting professional help. The head referee cannot look at the entire field at once...



My suggestions:
1. There absolutely must be a way to train these referees better than just a test. Corey balint had suggested a video of possible scenarios played out via animation or even people robots like done in the kickoff examples, and do this pre regional events.

2. Even if it means raising the regional fee a hundred dollars extra per team, it would be worth it to have all the referees in person, together, going over the rules and some of the scenarios that might occur. Teams pay so much money to go to an event, its simply unfair to not have consistent refereeing event to event. Even if this means paying for these referees to fly around the country to keep it consistent.

3. The last suggestion I have isn't really a suggestion, but more of an observation of things that I've seen at competitions that unease me. Specifically in the case of talking to referees in the "box", I have seen white shirted volunteers turn students away without giving them the chance to state their case. This is obviously unacceptable. The box is there for a reason, it is there so that a student can talk to a referee about what had just occurred in a match. There is no reason for a disconnect between the students and the referees. Regardless of whether its about the robots or not, its always about the students. If it means the schedule needs to wait for 45 seconds, then let the schedule wait. Everyone is supposed to be there for the kids, so lets actually let the kids talk. Going from FLL in '01 to FRC '02 to '05 to FRC college mentor I have definitely seen the disconnect between the students/teams and FIRST as a unit. It saddens me to see students turned away from expressing their opinion, this isn't what FIRST is about.

[Bob Steele]

In looking at all of this an idea occurred to me.

What about taking another page from the new rulebook of sports.

The challenge flag... Each team would be allowed one per event.
Throw it for little things.... waste it....
Save it for something really important.... perhaps a change that would make a difference in a match score and tip the scales...

Perhaps use it as a third tie breaker for qualifications...

Perhaps, Each alliance would have one (only one) of these for the elimination rounds.

Of course this would not be used this year.... just something to kick around for next year...

I applaud the referees for the work they have done this year....
This is an extremely difficult game to referee compared to many others we have played.

thanks
Have a great year!!!

[David Brinza]

Originally Posted by Brandon Holley

I agree with many of the posters in this thread who agree that "they are just volunteers" excuse is no longer valid.


3. The last suggestion I have isn't really a suggestion, but more of an observation of things that I've seen at competitions that unease me. Specifically in the case of talking to referees in the "box", I have seen white shirted volunteers turn students away without giving them the chance to state their case. This is obviously unacceptable. The box is there for a reason, it is there so that a student can talk to a referee about what had just occurred in a match. There is no reason for a disconnect between the students and the referees. Regardless of whether its about the robots or not, its always about the students. If it means the schedule needs to wait for 45 seconds, then let the schedule wait. Everyone is supposed to be there for the kids, so lets actually let the kids talk. Going from FLL in '01 to FRC '02 to '05 to FRC college mentor I have definitely seen the disconnect between the students/teams and FIRST as a unit. It saddens me to see students turned away from expressing their opinion, this isn't what FIRST is about.

Would it at least be acceptable to allow coaches to front off these volunteers that just want the issues to "go away" so the show can go on? If directed to leave the challenge box by an adult volunteer, many students will do so because it's GP to respect authority. Let the students get access (for however brief) to the referee to present their case.

[Brandon Holley]

Originally Posted by David Brinza

Would it at least be acceptable to allow coaches to front off these volunteers that just want the issues to "go away" so the show can go on? If directed to leave the challenge box by an adult volunteer, many students will do so because it's GP to respect authority. Let the students get access (for however brief) to the referee to present their case.

David,

You hit the nail on the head in my eyes. As a high schooler on the team I felt that if an adult volunteer told me not to be somewhere, I should abide and walk away.

However now as a mentor, if I saw one of my students turned away, my first instinct would be to politely confront the volunteer to find out why they were turned away.

While i see the point why adults can cause heated conflict, I do see a place for them in just GETTING the student to the REFEREE. I have full faith that any of my kids could explain what the situation was we were contesting, however if my kids cannot even get to a referee this is moot.

[Richard Wallace]

Originally Posted by David Brinza

Would it at least be acceptable to allow coaches to front off these volunteers that just want the issues to "go away" so the show can go on? If directed to leave the challenge box by an adult volunteer, many students will do so because it's GP to respect authority. Let the students get access (for however brief) to the referee to present their case.

Which volunteers do you mean?

Only the Head Referee can make a team's post-match clarification request "go away". Head Referees I have worked with prefer to interact with pre-college team members who are standing in an area that has been designated (e.g., with tape on the floor) for that purpose.

Originally Posted by 2008 FRC Manual, Section 9.6.2

9.6.2 Referee Interaction Rules

<T03> The Head Referee has the ultimate authority on the field during the competition. THE HEAD REFEREE RULINGS ARE FINAL! The referee will not review recorded replays under any circumstances.

<T04> If a team needs clarification on a ruling or score, a pre-college student from that team should address the Head Referee after a field reset has been signaled. Depending on timing, the Head Referee may postpone any requested discussion until the end of the subsequent match. Head Referees will only discuss calls, scores, penalties or match outcomes with pre-college team members.

[David Brinza]

Originally Posted by Richard

Which volunteers do you mean?

Only the Head Referee can make a team's post-match clarification request "go away". Head Referees I have worked with prefer to interact with pre-college team members who are standing in an area that has been designated (e.g., with tape on the floor) for that purpose.

I'm referring to non-referee volunteers trying to quickly usher the teams away from field-side after their match, sometimes before the scores are announced. We know that it can be a challenge to keep on schedule, especially if there are field faults, repairs, referee discussions, etc.

Teams are entitled to get clarification of rulings from the referee in accordance with <T04> and <G53>. This needs to be made clear to all of the field-side volunteers, who should point them to the challenge box. The teams need to use discretion in seeking clarification, no whining about a judgment call, but understanding what action triggered the penalty should be fine. Most certainly, questions about correct application of the rule should be explained.

[jgannon]

Originally Posted by Richard

Which volunteers do you mean? Only the Head Referee can make a team's post-match clarification request "go away".

Officially, yes, but sometimes some headstrong field personnel feel like they can usher you away from the box because you're slowing things down. This happened to us once this season, and this same volunteer told me later that we were lucky she didn't tell the judges about the incident, because she felt that our resistance to her demands should exclude us from eligibility for awards. It doesn't surprise me to hear that others are being turned away from the challenge box by people other than the head ref.

[Ryan Dognaux]

Originally Posted by jgannon

Officially, yes, but sometimes some headstrong field personnel feel like they can usher you away from the box because you're slowing things down. This happened to us once this season, and this same volunteer told me later that we were lucky she didn't tell the judges about the incident, because she felt that our resistance to her demands should exclude us from eligibility for awards. It doesn't surprise me to hear that others are being turned away from the challenge box by people other than the head ref.

In the event that this happens, it's almost worth it to try and talk to the volunteer coordinator and report these remarks. A crew volunteer should have no control over the awards selection, that's for the judges to decide only. Stories like this make me angry as most volunteers are amazingly cool people who are there to make your regional run smoothly. I guess there's room for bad apples even in FIRST.

Originally Posted by Paul Copioli

1. While the NASCAR theme is cool, forget about this "look at the robots right at zero nonsense." Just assess the points when everything comes to rest: robots and balls. Why? It makes it easier for the refs and the spectators.

I can't tell you how much I agree with this point. After attending BMR, I saw so many matches where balls were knocked off the overpass, but remained as counted on the overpass because they didn't hit the ground in time. This is lame. It confuses the crowd and even threw me for a loop at first. Little things like this make a big difference for the "wow factor" of an event. Many ends of matches that could've been exciting were quickly extinguished because of this flaw.

[Ken Streeter]

Originally Posted by Ryan Dognaux

After attending BMR, I saw so many matches where balls were knocked off the overpass, but remained as counted on the overpass because they didn't hit the ground in time. ...

Hmm. In cases of unrestrained motion, the trackball positions are scored based upon where they do finally come to rest, not where they are when the clock reaches zero. If BMR was scoring all trackballs based upon where they were when the clock reached zero, that was incorrect based upon Team Update #13, which resulted in rule G14 being updated as follows:

Originally Posted by Section 7, The Game, Rev G

<G14>
When the MATCH ends, each TRACKBALL that is at least partially supported by the OVERPASS and not in contact with any ROBOT of the same ALLIANCE will earn a 12-point bonus. If a TRACKBALL is in unrestrained motion (i.e. not in contact with a Robot) when the clock reaches zero, its contribution to the score will be based on when it comes to rest.

[Ryan Dognaux]

Originally Posted by Ken Streeter

Hmm. In cases of unrestrained motion, the trackball positions are scored based upon where they do finally come to rest, not where they are when the clock reaches zero. If BMR was scoring all trackballs based upon where they were when the clock reached zero, that was incorrect based upon Team Update #13, which resulted in rule G14 being updated as follows:

Maybe they weren't counting them because they were knocked off after the clock hit zero. Not exactly sure, but it could be the case. At least in one of our matches, our partner (1018 I think) knocked a ball off right at the end and it was still counted as being on the overpass. The officiating was pretty consistent either way. Tons of penalties of course.

[IndySam]

Originally Posted by Ryan Dognaux

I can't tell you how much I agree with this point. After attending BMR, I saw so many matches where balls were knocked off the overpass, but remained as counted on the overpass because they didn't hit the ground in time. This is lame. It confuses the crowd and even threw me for a loop at first. Little things like this make a big difference for the "wow factor" of an event. Many ends of matches that could've been exciting were quickly extinguished because of this flaw.

I didn't see that, what I saw were balls that were knocked off right at the buzzer and were rightfully counted because they were in contact with the robot knocking them off at the time of the buzzer.

[Ken Leung]

Originally Posted by Anne Shade

2. Provide for a method of evaluating the referees, specifically the head referees. Training is only half of the equation. Training provides the knowledge but is it being applied correctly? Referees in all professional sports are evaluated by their organization after all games. If too many mistakes are made, they are re-trained or released. Where is the evaluation for FIRST referees? How do we know that the training received was effective if their is no evaluation of performance? Some people are just not cut out for the job but we have no way to determine that until something goes grossly wrong.

Originally Posted by Brandon Holley

My suggestions:
1. There absolutely must be a way to train these referees better than just a test. Corey balint had suggested a video of possible scenarios played out via animation or even people robots like done in the kickoff examples, and do this pre regional events.

Perhaps we can take care of both of issues with one stone, maybe 3 issues with one stone. Ever thought about focus groups of FIRST Referees?

The difficulties of using video footages to train refs is an excellent idea, but I don't think animation is going to cut it. We literally need footages from the first week regionals. It's true that there will be no footages until the first week, but that is not that different from how things are now.

The Refs and the GDC go into week 1 without a clear vision of how things will played out, but their vision is refined after that, and refined again after another week, and so on. So, its a matter of how you can capture the first week to help you train the Refs better in the weeks that follow. Besides, there are certain elements that will continue to come back year after year, such as entanglement, tipping, and high speed ramping.

Anyway, back to the focus group. Here is the idea. You look at videos from week one. You look for instances where rule violations occur. And you look for them so much that you begin build up a spectrum of instances between clearly acceptable acts and clear violations. Say you get 5 video clips of ramming that give you an understanding of, on a scale of 1 to 5, what's acceptable ramming and what's unacceptable, and the gray area in between.

Here is how you apply these videos to training and evaluation. During the training, you show 1 clip as an example of clear violation of a certain rule. After the training is over, you show 5 clips of the same rule, and you ask the person in training which ones they feel are violation and which ones aren't. You give them a chance to make mistakes so they have a better understanding of the variation.

Then you show them another 5 clips for another evaluation. After you repeat that once or twice, you should have a good idea whether that person has caught on to what's acceptable and what isn't.

For training purposes, it is very important for the head Referee at the headquarter to decide, on scales of 1 to 5, what is acceptable and what isn't. Once a line is drawn, you drill that line into all referees' head in the weeks that follow. Consistency is what's important here. And you continue to use these clips and newer ones to train and evaluate refs before and after the events, and before and after the years.

Won't be for every rule of course, just the difficult ones.

Here are some side bonuses:

1. You release some of these clips to the teams so they too will get a better understanding of what's acceptable and what isn't.
2. You collect data as people evaluate various clips. The more data you collect, the better you are at understanding, in general, how people are calling certain rules. This understanding will help you decide whether a rule need to be better written, or if the training need to be improved.


Sounds complicated, right? You betcha! There is probably a simplier idea in here somewhere. On the other hand, there are many people watching many videos of many competition during the weeks. Won't be hard to find volunteers whose job is to identify these clips for the GDC and the head Ref at headquarter. Once a system is in place, Refs and refs in training just need to go to the training site and watch some videos.

[Anne Shade]

Ken, I think you have some valid points but the evaluation I was refering to is an evaluation of a head referees' performance at a regional. There needs to be a way to say that a head ref did a good or bad job and to determine who is not right for the job. Tests and questionaires only get you so far in the evaluation of a person's skills and performance.

Part of that evaluation could be team and key regional personel feedback but most really should be video review or in person evaluation. Why not have the committee of championship head referees review random matches from each regional and evaluate the play calling and the refereeing procedures utilized by the head referee? Head referees would receive feedback on how they can improve and those that are not performing up to par can be replaced.

This is the second part of training. Without the evaluation, the training has very little value.