FRC Blogged - 2013 Championship: Einstein Scoring Error

[Hallry]

Taken from the FRC Blog, 5/15/13: http://www.usfirst.org/roboticsprogr...-scoring-error

2013 Championship: Einstein Scoring Error

Blog Date: Wednesday, May 15, 2013 - 10:04

Many people are aware there was a scoring error on the Einstein field, but not everyone knows the details. I’d like to explain what happened and our plans to minimize the chance of this happening again.

We knew that the automated scoring system in use had a design limitation which could result in disc count errors, so a manual count verification process was put in place at the start of the season to double check counts before scores were finalized. The goals over the driver stations had hinged bottoms to allow easy removal of discs. On Einstein, a field reset person would use a pole to push up on the hinge, allowing the discs to drop in to two separate bins, one on the left side, one on the right. The number of discs in each bin would be counted, written on a scratch piece of paper, totaled, and then the number of discs scored in autonomous would be subtracted from that total count before being transferred to the official score sheet for the match in the ‘teleop’ section. In Semi-Final 2-2 on Einstein, the field reset crew added 20 and 26 on the scratch paper and wrote ‘64’ instead of ‘46’, accidentally reversing the numbers. 10 discs had been scored in autonomous, so the official score sheet showed 10/54 for auto/teleop discs, rather than the correct totals of 10/36.

The alliance captain pointed out the scoring error to the Chief Referee. We reviewed the information, spoke with the field reset crew, and found the scoring error. The field reset crew was very forthcoming about the mistake, and was obviously shaken that it had happened. We told the teams we were correcting the score, and would be going to a third match in that series. Dean came over to talk to the alliance that we erroneously believed had made it to the finals. I went over to the alliance as well, apologized, and showed them the sheets on which the error had been made.

The problem was not with the field reset team, but with the overall system we had in place, which exposed too many opportunities for error. In most cases, the manual disc verification system functioned well and resulted in accurate disc counts, but on Einstein and several other events, we know for certain it did not. At those events during which the errors were caught in time, they were corrected. However, I know there were events at which errors were not caught until it was too late to correct, and to those teams, I apologize. I’m certain this was a painful experience.

Everyone makes mistakes, particularly under high-pressure situations, and I think it’s unlikely that we will be able to fully eliminate human judgment from every possible scoring or refereeing situation. Much of our ability to automate scoring depends on the game, some of which are relatively easy to score automatically, some realistically can’t be scored automatically at all. In any case, we remain committed to making sure that the scoring system in each FRC game is as accurate as possible. We’ll be working to reduce the opportunities for errors by employing more robust system checks and testing the system earlier and even more thoroughly, taking in to account any potential human factors and unlikely ‘corner cases’ that still seem to crop up with the number of matches we run at FRC events - almost 7,000 in the 2013 season. The FRC staff, just like the FRC teams, want matches to be won or lost based on what actually happened on the field.



I’ll blog again soon.

Frank

Once more, Frank being frank. +1.

[akoscielski3]

Even though I was already told what happened by a few members of the Einstein Teams involved I am glad that FIRST has released an "Official" statement of what happened. Though I do kind of wish they said more at Championships, it is understandable that they may have not known what to say or how to say it.

I am happy that FIRST is taking some sort of action towards trying to prevent this in the future, like they have in the past. And hope that scoring takes less work for the Volunteers and the Score Keeper.

As a "Score Keeper in Training" at GTR East I can see how this could easily be mistaken by the volunteers and/or the score keeper.

To everyone at FIRST, on the GDC, and in anyway involved in FRC this year, thank you for a great year, this is one that will go down in my mind as one of the best years of my life. Not just because it is my final year of FRC but because it was an amazing game to play and design for (even though I had some mental break downs and frustrating moments while designing a robot for this game)
still! THANK YOU ALL!
I can't wait to see how next year turns out as a mentor!

[Tetraman]

The problem was not with the field reset team, but with the overall system we had in place, which exposed too many opportunities for error. In most cases, the manual disc verification system functioned well and resulted in accurate disc counts, but on Einstein and several other events, we know for certain it did not.

This is a very true statement. I was the scorekeeper for FLR and I can say that the manual disc counts had many opportunities for error. I do, however, stand by the scores of FLR; I'm confidant in those who were manual counters at the event and that their counts were accurate.

I believe the GDC should start paying more attention to more than just how the game is designed, but how it is also played and operated. This is apparent due to the GDC unaware of the "rain of discs" which caused the Week 0 rule change, and I hope the GDC is going to start more game playtesting than there already is. (if you need any help, give me a call )

The FIRST game design committee has the most dedicated of FIRST members and together have designed some of the most amazing games we have ever played these past couple of years. Please continue to make FIRST the most intense learning experience for everyone.