I have a pet peeve in FRC, and it’s about fouls that “didn’t affect the outcome of the match”.
The reality with real-time scoring is that every foul can affect the outcome of the match, and I think most of us know that.
In any match, looking at the score tells the drivers what they might need to do, whether that’s making a last minute Hail Mary High Goal attempt or playing some serious defense on whoever has the ball on the opposing alliance.
If 30 seconds into the match, when my alliance might otherwise be ahead, and due to a foul, my team is behind, my drive team is going to take some crazy risks to try to win the match, whereas if my alliance maintains a comfortable lead, we might play it safer and make the shot when it is a sure thing.
These split-decision moments affect match outcomes, and these are affected by what fouls are in play or aren’t.
So when I talk to a head referee in the question box or even another team after losing a match, and they tell me not to sweat the foul because it “didn’t affect the outcome of the match”, I clench up a little bit inside, because really, every foul affects the outcome of each match.
Maybe we wouldn’t have risked that crazy shot and missed if we were close or gone for the low goal instead of the high goal.
I’m not sure how to fix this- it requires the real-time scoring to be much more accurate more of the time and for fouls to be decided fairly and evenly- which is currently a major problem in Aerial Assist.
My biggest suggestion would be changing the rules so that bad, poorly judged fouls were less common, but that’s too little too late (except for off-seasons).
I hope in future years the game is more gentle toward referees, because this one is not, and that messes up matches by the dozens.
What do you think? Am I over-reacting or is this a real problem?
I completely agree. While a 5 yard penalty does not actually put the ball in the end zone, both teams must change their strategy on the next down. The same applies to FRC.
This bothers me too, and has happened to me multiple times. I believe that all calls should be made whether or not it would have an effect on the outcome. Had this been the case, I’m sure many matches would’ve had much different outcomes throughout the season.
With real-time scoring, penalties should not be assessed after the match has finished. (Unless it is a yellow or red card, etc.). In NFL games, you don’t see yellow penalty flags thrown during halftime or during commercial breaks. I know this game was tough and there are different circumstances for every situation, but changing the score after the match has ended is annoying forspectators and players. I don’t have any solutions to offer up, but it was something that happens a lot in FRC, and I don’t really see it happen in other spectator sports.
(I mean, imagine that you’re watching a hockey game, and after the final period ends, the refs gather together, then go back out onto the ice and announce that team A will get a penalty shot. How insane would it look?)
As a driver who was affected four time in the semifinals of two different regional competitions by fouls scored after the match (both times we lost the semis after these fouls were accredited), I can say that it does feel frustrating. You look at the score, think you won the match, then realize 5-10 minutes later that you lost it because of a penalty that hadn’t been scored.
That being said, I can completely understand why it happens. The referees this year, as has been previously discussed, had a tremendous amount of work to do in both counting possessions and judging fouls that are highly subjective calls. The simple truth for this season, and the lesson that I feel can be taken away from it is that, going forward, the best idea would be to try and always have separate foul and scoring referees. Especially in a game like this where the outcome of a match can be determined by a single referee determining split-second possession, having scorers freed up to watch for that and referees able to watch for fouls would improve both the scoring and assigned fouls.
While it is frustrating to have it happen to you, where you thought you won a match before losing it 15 minutes after it ended, it’s something that I can understand. I know that, if I were in the shoes of a referee, the same thing would end up occurring if a foul was committed in the last 30 seconds or so of a match that had a fast pace to pay attention to. Just putting myself in the shoes of a referee helps, and always remember to smile, congratulate, and thank the opposing team for a match close enough that a foul could sway the match. Those matches are always the most fun for me anyways, since it could go either way at the end.
While I understand your frustration, it makes me ask what are your Drive Team Coaches doing during the match?
It is impossible for anyone to see everything that is happening in a match. That is especially true of a student who needs to be focused on operating their robot or preparing to in bound a game piece.
The role of a Coach includes being aware of the situations during the match and that includes being assessed a foul. Fouls are signaled during the match and in some cases they are included in the real-time scoring (as I recall).
The Coach is also responsible for communicating with the other Coaches. Let me relate an example from a few years ago to illustrate my point.
The game was Breakaway (the game with the incentive to score points for your opponents). One of our partners was very good and routinely would score points for the opponents to improve their own seeding. I noticed that we threw a chain and it was temporarily outside the frame. I quickly instructed our driver to move and cover the chain so as not to incur penalties.
I thought that the referee had indicated a penalty. I immediately turned to our partners Coach and advised him. Based on my communication, he immediately instructed their drivers to stop scoring for the opponents and only score for our alliance.
You are absolutely correct Kevin. I don’t think this is an aspect the referees or GDC understands (or if they do its not widespread).
I have made calls mid-match for similar reasons. If we are up by a good amount we call off the defense or go light so as to not risk getting a penalty especially with most refs this year calling penalties for merely touching an opponents ball while driving around the field. Last year we wouldn’t risk the last cycle (4th or 5th) if we didn’t feel the additional 12 points was worth risking the climb points. Additionally if the scores were really tight (and we were down by 1-2 points) we were prepared to do our 5th cycle and settle for a 20 point climb as it would give us an extra 2 points. Yes, 2 points makes a massive difference see Finals match 3 at Battlecry last year where we won by 1 point.
2012 was one of my favorite years because the penalties were reasonable, called/scored during the match, and I don’t think we ever needed to question a call all year (because there were so few). I really hope FIRST brings back something similar because this year was ridiculous. If we knew certain penalties were called we would have spent our time doing more triple assists. We still would have lost the match but I would have more assist points for ranking.
How about when they call a penalty on the intercom against the wrong robot and don’t correct it. While it may seem insignificant from a score standpoint, that kind of stuff makes it into scounting notes, and it seems like itd be easy to say “We got that on the wrong team, it was XXXX not YYYY”
There have been multiple, exhaustive threads about what was wrong with the rules and the refereeing/scoring set up this year. And I agree that FIRST needs to do better. Starting with having dedicated scorekeepers and not relying on buggy hardware/software to keep game play flowing (pedestals).
I also read some peoples’ posts and analysis that this year fouls and scoring were particularly troublesome in terms of making for a good game. Before becoming a teacher modeling systems using a actuarial and game theoretic approaches was my job. So I wanted to investigate whether the fouls this year were indeed particularly troublesome. And one of the questions you always want to look at is “How often does a penalty affect the end result?” (It doesn’t have to be a penalty, it can be any action/situation.) It is not an easy question to answer.
Kevin is completely correct that the fact the match was decided by 60 points and the foul was “only 50 points” does not mean the foul decided the outcome. Of course, the reverse is also true. A 40 point match with a 50 point foul might have turned out the same way without the foul being called. And a match without fouls might have turned out differently if teams were not afraid of getting an “unfair” foul. One approach used to answer the question is to pick some (arbitrary) point, and say “net foul points more than twice margin of victory” and make any matches at least that extreme a 1. Matches with fouls not meeting this threshold are scaled. So we would say a match with one 50 point foul and a 50 point margin would be 0.5 of a “decided” match. It’s not perfect but it is a way to make a first approximation stab at an answer.
I have not been able to work my way through more than a few competitions worth of data, but it has been interesting so far. Aerial Assist looks pretty middle of the road to me at first glance. Part of that is the fact that in my FRC tenure (2000, 2003-2014) there have been a lot of years where many “fouls” were actually disqualifications that simply won a match for the other side. In 2004 and 2005 at our regional competitions there were matches in eliminations decided by such calls. In 2005 in particular I remember a controversial call deciding the finals. Even in 2012 there was controversy. I was at Queen City in the semis. Even though I think the referees made the correct call by the letter of the rules (which were later changed I believe) the call was very controversial and decided the outcome.
To be clear, there were a lot of foul points this year. Even once you factor for total scores. I think another factor to consider was that the real time scoring was better than it often is. It was certainly much better than 2013. 2011 is a special case because the tube scoring was good and easy to keep up with, but the glitch-prone minibot towers ended up deciding a bunch of matches. There have been seasons where it seemed like every other match had a substantially changed score due either to fouls or to recounts of the scoring objects. But this year fouls called early certainly caused some teams to alter their choices because you could actually see the adjusted point totals. There is no way to completely avoid having to “redo” scoring after matches sometimes. There is a big difference between FRC and most other team sports: We don’t stop the match for fouls. But I still think the real time scoring this year, in spite of easily correctable flaws, was pretty good. The problem was in the rules and in the fact that there was too much for the referees to do.
I will admit, as a cross country and track coach, where the only consequence of of a foul (which can be something as silly as a too large logo on a t-shirt worn under the uniform or forgetting to take off an earring) is a disqualification probably makes me more tolerant of scoring adjustments. Heck in cross country you often have to wait 15-20 (or more) minutes to find out who won a race. In any event I don’t know that the “the call didn’t affect the outcome of the match” issue was particularly bad this year compared to previous years. And in pretty much every team sport there are going to be calls that are controversial and people upset at being treated unfairly. What I hope for is for FIRST to keep working to minimize such occurrences.
This was already the case this year. Early weeks had 2 scoring refs and 2+1 foul refs (+head ref). By Worlds it was 2 scoring refs per ball and around 2+1 for fouls (at NYC we got up to 4+1). I still watched assists get missed at Worlds.
Certainly, but this depends on the foul being signaled, and being signaled correctly. I’ll keep score (goals and fouls) in my head as a coach regardless of year. The amount of attention I pay on top of that to the projector varies by game. For 2013-type games where scores can be post-checked, I only look for fouls that get entered without signalling (happened a lot, actually). When there’s basically no cross-check like 2014, you basically have to babysit the darn thing. Nonetheless, it’s not uncommon for signaled fouls to be removed post-match. Occasionally there are even unsignaled fouls added post-match. More often there’s a TFoul that becomes a Foul or vice versa, though I have some success predicting these with crews I’ve reffed with.
This is interesting. I’ve read some of the other fouls-deciding-matches data, and they seem to reflect that 2014 is higher than recent history (self-defined as a high school senior’s FRC career). Do we understand the discrepancy? Also, since you’re going by event, I wonder if there’s a way to identify event-over-event trends.
I’d like to delve into ‘real time scoring was better’, though. I think your definition is reasonable for the OP’s question, but now it sounds we’re conflating that with actual quality of the game for play. Yes, real time and post-match scores are closer. Barring fouls, auton/teleop changes, and a semi-common software bug, they were almost always identical. But this isn’t a sign of success–it’s a sign of lack of cross-checking. In 2013, we could correctly* recount post-match, and I could tally crashes above me and flashes in front me. 2012’s had a good ‘one ball in one basket’; 2011 was super easy; 2010 was super slow; 2008 was closer to 2013; 2007 was 2011 with a drunken calculator; 2006 was 2013 with too much pressure on auto scores; 2005 could post-check… Yes, I did that on purpose. Let’s not talk about Lunacy.
The problem with 2013 (and 2008 laps, and somewhat 2006 auto) is that when assists get missed, they’re gone. Yes, real-time reflected post-match more often, but as a reflection of what happened on the field it’s at least the worst in half a decade, and really it’s longer.
As I said, the data I have looked at so far is only from a few events. In general I find it much more profitable to look at smaller, more complete sets of data to try to find the right questions to ask and the right techniques to use. There is no question that as a percentage of total points scored, the foul points were very high this year. But that is not necessarily the same thing as saying that the fouls were more consequential than all other years. In earlier incarnations of FRC competition, fouls weren’t called like now. And generally (particularly in eliminations) meant a disqualification. I am still pondering different approaches to try in analyzing the impact of fouls on this game. I feel that most of the analyses I have read so far on ChiefDelphi have been accurate. But I also feel they have been analyses done to prove the point that this game was bad. That doesn’t make them wrong, but all my statistical training and practice has taught me to look for the possibility of different approaches when an analysis confirms what you want it to confirm.
As for the scores reflecting what happened on the field, I feel the worst scoring problem I have seen in 13 years of FRC was in 2011 with the inconsistency of the minibot towers. We won two matches in regional competition with the aid of opponents not getting credit for a win they should have earned in the minibot race. As for quality of game play, I felt 2013 would have been a much better game if real time scoring had worked better. I don’t think missing some assists is necessarily better or worse than not knowing whether you are ahead or behind as a match winds down.
None of this means that I think 2014 was a perfect game. It was particularly frustrating to me because there were flaws I felt were fairly easily corrected. But I also felt that in actual game play this was a fun game. For me, that is the most important aspect of any game. To be clear, I have never gone to a FIRST competitions and failed to have fun. Even when the robot is constantly breaking or failing to function properly. Even years when we went to one regional, didn’t make eliminations and that was it. But there have certainly been games I enjoyed less than this one.
As a team we talked about the entertainment value of this game a lot. If you read threads on ChiefDelphi there are a ton of people that really did not like this game. There are also people who did like it. In 2011 our robot worked very well. We were also fortunate with our draw in qualifications. It is the only time we have been a top 8 team at the Championships. We had a good robot this year, but not as good as 2011. And we had a murderous draw. We went 3-7 in qualifying at the Championships this year and 9-1 in 2011. Yet the opinion of every student and mentor who participated in both years was that this game was more fun to play.
I understand the point about potentially biased methodologies, but the analyses (at least the one I linked), aren’t saying that more foul points were scored this year–they’re at least attempting to speak to the consequentiality question.
Good catch on 2011; I’d blanked on the coin flip. Not just the towers malfunctioning, but the refs-as-garage-door-sensors. Another real- to post- scoring agreement without a view of reality. Albiet I personally saw a lot more problems in 2011 than 2014 (the later certainly non-negligible), which is pretty impressive considering 2011 was so much easier to spot unless someone really pushed the window. Fortunately, once I assumed it was an end-match coin flip, it didn’t much affect strategy. (Honestly, it was a coin flip by the end anyway, so the failures were mostly just annoying.)
I still can’t grasp the argument that 2013 was hard to keep track of, though. Even if the real-time was bad at the event, the math was easy and you only had to tally at the walls, as opposed to tallying assists everywhere for both balls.
Drive coaches can only react to fouls if they’ve been properly signaled by the referee crew. The Ref crew on Newton was especially bad at this. They consistently used a “post match huddle” as a safety net. We had one match where our alliance was given 2 technical fouls for contact inside the frame perimeter and a tipping violation. Once I saw the other robot go down, I scanned the referees and not a single ref raised a flag. Watching the video of the match confirmed that no flags were raised. However after the match there were 2 technical fouls called. Nothing drive coaches can do in that situation. Referees need to understand both their actions and in-actions have a direct impact on the match.
Kevin is right, it’s something that needs to be fixed in next years game design.
Let me ask a theoretical question. If a ref sees an action which could be a foul, but they need to check with the HR or another ref who had a better sight angle, would you, as a team member, prefer the ref to immediately enter the foul, only to delete it after the match if it was incorrect, or not to enter the foul, and add it after the match after checking?
Yes, in a perfect worlds all fouls are immediately caught and interpreted correctly, but we don’t live in a perfect world with professional referees.
Match type Q
Year Total Points Foul Points % Foul
2012 255330 19194 7.5%
2013 778520 41320 5.3%
2014 1576037 188080 11.9%
Match type E
Year Total Points Foul Points % Foul
2012 92904 3876 4.2%
2013 242568 9031 3.7%
2014 404948 33670 8.3%
Match types Q & E
Year Total Points Foul Points % Foul
2012 348234 23070 6.6%
2013 1021088 50351 4.9%
2014 1980985 221750 11.2%
This is from the same data referenced in the other post. The raw point totals don’t really matter due to different scoring rules and different number of matches counted, but percentages may be directly compared.
Every way I’ve broken it down, foul scoring was a significantly bigger factor in the 2014 data set than it was in 2012 and 2013 ones.
I think that the referee in question needs to make their best judgement call at that moment, and not wait until the end of the match to check with the head referee. If another ref had a better sight angle, then both refs should communicate at that moment (thumbs up/thumbs down) which I have often seen them do. After match corrective measures should be few and far between. Each referees need to take ownership of their own calls, not wait around until the end and force the head referee to make a call/no call on a situation that he didn’t even see.
On another note, if the ref is unsure of whether something is a foul, their default call should be no foul, just like in every other sport.
My point about isn’t that anyone’s methodology was bad or biased. Just that a question like “was this foul consequential?” is a hard question to answer statistically. It is pretty clear to me that the fact that fouls were a bigger percentage of the total points is evidence that points to fouls being more consequential this year than the last two. A comparison of fouls to margin of victory might be better, but then again I can think of some reasons why it might not. But there are probably some better ways to attack the question, but most of those would require knowing when fouls occurred. But I am still noodling around trying to find a good way to approach the issue.
As for 2013, I (and my team) found it frustrating that there were many matches where the final score was dramatically different than what was on the scoreboard at the end of the match. I don’t know that our strategy was necessarily changed all that much, but twice we thought we had wins and lost, and one thought we lost when we won.
I am not arguing that Aerial Assist was without flaws. In many ways I think the flaws were more irritating to me than they might have been because there were some simple fixes that could have greatly improved things. I took a team to Crossroads and the Championships, and I was a referee at Queen City, so I saw the game from two different perspectives. I think the single biggest issue was having the referees both calling fouls and scoring the match. Two dedicated scorekeepers would have greatly improved game play. If the referees didn’t have to toggle between screens and were only calling fouls their would have been far fewer missed assists and the like. A close second as far as issues was the pedestal. I think the game would have been better if there was a foul for entering a ball into play before the other was scored, rather than having to depend on a sometimes glitchy pad to pedestal system.
As for 2011, I think the perspective on how annoying the towers were depends on whether your team got short changed. I had two friends whose teams lost a match (or two) that would have left them top 8 in which their minibot clearly reached the top first but was not credited with a score. If your team didn’t make eliminations or were eliminated in that way, it would probably make you less favorably inclined to the game as a whole.
Personally, I prefer it signaled and not entered. Reffing, I typically end up half-hearting the signal (some kind of a flag point with coach eye contact, rather than an aggressive wave). It’s subconscious, but recently a few of coaches that know me have noted it’s a nice ‘this might be coming your way’ warning. I like the signal approach as a coach myself, but fortunately in my case a ‘signal’ from one of you guys can be a grimace or hitting the radio button. My druthers resolution? Have an official signal and/or a button (because we all so love buttons) that calls it up on the projector. I prefer the former, but I could see the latter for the crowd. Make it a driver’s meeting topic.
I can’t believe I’ve never talked about this before. This is why Carol rocks.
First, I look forward to any analysis you do decide on. You’ve clearly got a very strong handle on the complexity of the question, and you have my apologies as I feel like I’ve been unintentionally adversarial about it.
Queen City - Queen City was the same week a 2nd of 3 ref gigs, and while I would’ve loved more refs, we were never toggling screens if we were a scoring ref. (Actually, I never did this, thouh at NYC we had 8 refs.) If I was a foul ref, I’d sit on the possession screen to cross-check, and only flip to the foul screen after I’d signaled one and was otherwise clear–no robots in my zone of responsibility, and my cycle either had all 3 assists already or at least wasn’t about to end. Otherwise, we’d radio for entry. I had a truss ref come over and punch fouls for one or both of us a couple times. How was it done a QC? Different places I reffed or played under had better ideas for different tasks, but there didn’t seem to be a lot of cross-event consistency even in logistics.
Pedestal - yes, there were a lot of things that worried me about the rules on kickoff (and since), but they seemed to have trade-offs. The pedestal on Kickoff Sunday was my first ‘what the heck is the point of that?’ moment. The trashcan’s only purpose in this game was to make coaches mad at refs, make refs feel bad, upset the audience, raise the responsibility and failure rate of field reset, induce replays, slow down game play, and make dead balls suck more for all parties. So, in retrospect it actually had a pretty expansive purpose. If I never have to spend hundreds of dollars staring at an unlit trashcan again, it’ll be too soon.
Yes, I would presume that this is true of any game. (It’s why I keep scores in my head–not that that helped with the Towers, but it did for 2013.) Unfortunately I can’t think of any quantitative dataset that would allow us to check things like this, so I’ve tried to shift my anecdotal assessment at least from ‘did I hate this?’ to ‘how many did I see?’
I don’t think you have been particularly adversarial about it. I feel like I have been the adversarial one, though that isn’t my intention. In any event, I am still noodling around with modeling ideas, but all of them will likely need to wait until after June 21 when the Central Ohio Robotics Initiative is over. Between the end of the school year, track season, the FRC Ohio Championships and hosting the CORI event I just don’t have the energy to focus on much else.
I had the same thoughts about the pedestal on kick-off day. I told one of the mentors I know “I hope the pedestal lights don’t turn out to be 2014’s version of the minibot towers.” We are changing the rules for CORI. Not having to watch the pedestal should also make refereeing easier.
At QCR we tried as much as possible to (after autonomous) have the near side referees do all the scoring entry and the far side referees do all the foul entry. We had a bunch of post match huddles, but most of the time those did not result in scoring changes. When they did it was not always added fouls. I would say that the most common change was adding assists. A lot of the time we would hit a second or third possession followed by a score and submit it quickly and one possession would not register. Almost every time a team came up and asked us about a missed assist we either said “Yes we know, it is being corrected” or “Team xxx didn’t get into the white zone before they took the truss shot.” At our event I am trying to recruit four scorekeepers who will watch just for possessions, trusses, catches and goals as well as a complement of referees.
My druthers resolution? Have an official signal and/or a button (because we all so love buttons) that calls it up on the projector. I prefer the former, but I could see the latter for the crowd. Make it a driver’s meeting topic.