Some thoughts on rules, refs, and ranking

[zzzag]

Originally Posted by Racer26

Also, part of passing inspection is that all software is the correct version. How was this even a problem on the field?

Just hypothesizing here, but: it might have had something to do with the optional driver station update (it was strongly recommended but not mandatory).

[alex334]

Originally Posted by zzzag

Maybe it's just me coming from SBPLI but today had the highest number of replayed matches I've ever seen. As part of a team that has had connectivity issues in the past and never had anything replayed, it struck me as a bit odd.

Anyway... thank you for the clarification. From the video it was not apparent that there was a malfunction - of course, I look for lack of motion to determine a connectivity issue most of the time. To I would guess many onlookers, 334's robot seemed to be functioning similarly to how it had in previous (not replayed) matches, for all if not most of the match.

That said, if there was an issue with your system that was impacting your performance on the field and delaying the progression of gameplay by requiring replayed matches "during the entire regional", I would assume common sense would dictate installing those updates at first opportunity.
The combination of upset victory by the other team with what appeared to be a very delayed call for a replay did seem just a tad fishy.

To clarify, it was not our system that was malfunctioning. The problem stemmed from driver station updates. If one team's driver station was not fully updated, the entire alliance would lag out. We found this out during a practice match and immediately fixed it, however, other teams did not and continued using the old version. Moral: CHECK FOR UPDATES!

As to the malfunction, our robot had mechanical jams in a few of the matches. We installed guards to prevent this, so we knew this was not the case. Not to mention that our climber was not working.

[DominickC]

Originally Posted by alex334

The problem stemmed from driver station updates. If one team's driver station was not fully updated, the entire alliance would lag out.

Were the FTA's alerted to this?

[savage]

First of can we stop bashing the Refs. I do not want to have a MOD close this thread. Second the Refs are doing the best job that they can. Just to let you all know all most all of the refs have close to 7-10 years in FIRST EACH, is not like its their first time at an event. Like some one said we are only Human, we cant see everything at once and we do miss things but teams need to learn and move on. Yes blockading is a tough call to make on the field but when it is called there is a reason behind it. And As to the Ops video post of the match that would be a tough call to make but there is NO way that any video replay would or should over rule a head Refs Final call. And this was in the semis so there was no more of giving teams a break when it comes to fouls.

[zzzag]

Originally Posted by alex334

To clarify, it was not our system that was malfunctioning. The problem stemmed from driver station updates. If one team's driver station was not fully updated, the entire alliance would lag out. We found this out during a practice match and immediately fixed it, however, other teams did not and continued using the old version. Moral: CHECK FOR UPDATES!

As to the malfunction, our robot had mechanical jams in a few of the matches. We installed guards to prevent this, so we knew this was not the case. Not to mention that our climber was not working.

So what you're saying is that not you but somebody else on your alliance had the older version?

Having had issues with this previously, do you think this is something that should have been checked as soon as your alliance was formed? It's not like you didn't have a lunch break to confirm that and have your alliance partners install any necessary software updates.

Also, how was this avoided in your elimination matches before semifinal 3?

And is it normal for part of a control system to malfunction while the rest continues to work perfectly?

[alex334]

Originally Posted by IndySam

No, if you let teams start using video replay you open up a huge pandoras box of potential delays and bad angles and disputes. It might suck but it's the only way.



I think we were the poster child of questionable rules and calls last year, they even changed the rules at championships because of what happened to us.

After words I consoled my team and praised them on what a great job they did, then I found the head ref and complimented him on doing a good job and making a tough call. I didn't come on CD and complain about the refs and unfairness of it all.

BTW If you/we hadn't lost one of the other matches in the first place a questionable call couldn't have hurt us. Can't blame anyone else but you/us for that.

I definitely consoled my team as well. We found solace in the Industrial Design award. We're fine. I just wanted to hear the community's opinion on the matter. I'm sorry if I seemed to be whining.

The fact that teams should account for bad calls is unacceptable. That's comparable to NASA having to account for changes in the laws of physics.

I'm not saying that video proof would be the end all of a dispute. I just want it to be used. Period. The fact that it's being completely ignored bothers me tremendously.

[zzzag]

Originally Posted by alex334

The fact that teams should account for bad calls is unacceptable. That's comparable to NASA having to account for changes in the laws of physics.

The laws of physics don't change from day to day due to human error.

[alex334]

Originally Posted by zzzag

So what you're saying is that not you but somebody else on your alliance had the older version?

Having had issues with this previously, do you think this is something that should have been checked as soon as your alliance was formed? It's not like you didn't have a lunch break to confirm that and have your alliance partners install any necessary software updates.

Also, how was this avoided in your elimination matches before semifinal 3?

And is it normal for part of a control system to malfunction while the rest continues to work perfectly?

Yes, FTA was made aware very early on.
The update issue was not of concern during eliminations. We checked everything. The problems stemmed from something else. We don't know what.
I'm just putting out a warning to you guys. This was a problem for some qualification matches until we started verifying all of our alliance members.

[zzzag]

Originally Posted by alex334

Yes, FTA was made aware very early on.
The update issue was not of concern during eliminations. We checked everything. The problems stemmed from something else. We don't know what.
I'm just putting out a warning to you guys. This was a problem for some qualification matches until we started verifying all of our alliance members.

Essentially, all of a sudden with what was basically the deciding match of the regional on the line, you had a mysterious failure of unknown cause in your ability to score?

[alex334]

Originally Posted by zzzag

Essentially, all of a sudden with what was basically the deciding match of the regional on the line, you had a mysterious failure of unknown cause in your ability to score?

Haha. It sounds so fishy. But yes. We checked everything and were extremely distraught. The day was full of ups and downs.

[PayneTrain]

Originally Posted by savage

First of can we stop bashing the Refs. I do not want to have a MOD close this thread. Second the Refs are doing the best job that they can. Just to let you all know all most all of the refs have close to 7-10 years in FIRST EACH, is not like its their first time at an event. Like some one said we are only Human, we cant see everything at once and we do miss things but teams need to learn and move on. Yes blockading is a tough call to make on the field but when it is called there is a reason behind it. And As to the Ops video post of the match that would be a tough call to make but there is NO way that any video replay would or should over rule a head Refs Final call. And this was in the semis so there was no more of giving teams a break when it comes to fouls.

Mods will close the thread if we don't check our passion at the door of the thread that is as touchy as this, so let's step it down a notch. FIRST is a passionate event, but that does not always warrant fiery discussion that may come off poorly. I can assume you are a referee, and I would not say anyone is going out of your way to tear you or your volunteering kin down. I will say it is good to have a healthy discussion on rule interpretations in the hope that the game evolves properly over the 8 weekends it is in-season, be it rules, strategy, robot execution, and overall event quality. Some of the rules are left up to interpretations the Supreme Court could not easily decide on, but that's how it is.

Originally Posted by zzzag

Essentially, all of a sudden with what was basically the deciding match of the regional on the line, you had a mysterious failure of unknown cause in your ability to score?

I hope you weren't meaning to come off as mildly condescending here.

Let's all take a deep breath, read this, sleep on it, and try again tomorrow.

[AmoryG]

I do agree with the OP on some point, particularly with the occasional horrible penalty call. I empathize with your team because I remember a time when I felt the rules screwed my team out of a win that would of sent our alliance to the semifinals.

I also agree that the best teams aren't always on top. I occasionally see teams I did not expect at the top of the rankings, or even elite teams that somehow are in the bottom half of the rankings. Take 254 at the San Diego Regional. They finished with a 5-5 record, but were picked by the #1 seed even though they struggled throughout the event. And on the other side of the spectrum are teams that are ranked near the top, but are declined by many higher seeding teams. I agree that the rankings can be a little messed up, but generally they're pretty good and almost always the best teams form the best alliances and they will usually win the events.

Also, keep in mind that FIRST really is doing the best job it can do. And given that the competitions are almost exclusively run by volunteers, I think FIRST is doing a pretty good job. And if you don't think so, just look at other competitions such as in college and professional sports, and you'll know FIRST is doing something right. College sports are dominated by a few teams, and it's very rare other teams rise above the rest. Every regional is worth watching in FIRST because there's always a few outstanding teams that aren't recognized outside of the region. And while our volunteer referees and judges aren't perfect, is our system so much worse than the systems in professional sports? I'm always hearing how the greatest players are favored by referees, and the blatant cheating that participants get away because of how difficult it is to objectively call fouls and penalties. And in every sport I can think of, the perennial playoff contenders are almost universally hated. Great teams in FIRST are universally respected, and we actually cheer for winners as often as we cheer for the underdogs. And think of all the smack players talk about players from other teams. It really is amazing how a bunch of high school students act more maturely than these "professional" athletes.

So I agree FIRST has problems, but I can't agree that it hasn't been doing a good job for the most part.

[alex334]

Originally Posted by AmoryG

I do agree with the OP on some point, particularly with the occasional horrible penalty call. I empathize with your team because I remember a time when I felt the rules screwed my team out of a win that would of sent our alliance to the semifinals.

I also agree that the best teams aren't always on top. I occasionally see teams I did not expect at the top of the rankings, or even elite teams that somehow are in the bottom half of the rankings. Take 254 at the San Diego Regional. They finished with a 5-5 record, but were picked by the #1 seed even though they struggled throughout the event. And on the other side of the spectrum are teams that are ranked near the top, but are declined by many higher seeding teams. I agree that the rankings can be a little messed up, but generally they're pretty good and almost always the best teams form the best alliances and they will usually win the events.

Also, keep in mind that FIRST really is doing the best job it can do. And given that the competitions are almost exclusively run by volunteers, I think FIRST is doing a pretty good job. And if you don't think so, just look at other competitions such as in college and professional sports, and you'll know FIRST is doing something right. College sports are dominated by a few teams, and it's very rare other teams rise above the rest. Every regional is worth watching in FIRST because there's always a few outstanding teams that aren't recognized outside of the region. And while our volunteer referees and judges aren't perfect, is our system so much worse than the systems in professional sports? I'm always hearing how the greatest players are favored by referees, and the blatant cheating that participants get away because of how difficult it is to objectively call fouls and penalties. And in every sport I can think of, the perennial playoff contenders are almost universally hated. Great teams in FIRST are universally respected, and we actually cheer for winners as often as we cheer for the underdogs. And think of all the smack players talk about players from other teams. It really is amazing how a bunch of high school students act more maturely than these "professional" athletes.

So I agree FIRST has problems, but I can't agree that it hasn't been doing a good job for the most part.

Yes, FIRST does a very good job. It got me to where I am today. The volunteers are amazing and you can never thank them enough. Yes, the attitude of FIRST is pretty great. However, that doesn't mean it cannot greatly improve. You should always be seeking to improve. I wanted to call attention to a few things that I noticed that could/might be improved. I wanted to see some explanations for both sides of the argument in all cases. If something is not possible, that's fine, I'll accept that. I just wanted clarifications on a few thoughts.

The thing for me with powerhouse teams is that they inspire others. I always drew inspiration from 254. I always wanted to and still do want to become more like them. If they shine in competition and seed high, that gives teams a clear goal. But if teams like that are not in the top spots, what message does that send?

[EricH]

Originally Posted by alex334

But if teams like that are not in the top spots, what message does that send?

It sends a message that even the best have an off event now and again. They have their issues like the rest of us. And, there is hope that maybe we can be asking them to join our alliance instead of the other way around.

Incidentally, I'd like to point out some other, really annoying ranking systems.
2001--besides the 4v0, IIRC it was just the scores of the teams--if your colored ball was on the goal you got 10 extra points.
2003--details escape me, but I remember it being some function of scoring.
2010--When Coopertition Score was the biggest driving factor of the standings, things got "interesting", including a "planned" 6v0 match where the loser shot ahead of the winner in the standings.
2012--The bridge points affected an awful lot of rankings. 254 couldn't get a good ranking at one event because for one reason or another nobody went for the Coop Bridge with them.

This year, it's back to the good ol' win-loss-tie system, which like a republic is the worst system around except for all the others.

[slijin]

A long post justifies a long response.

Originally Posted by alex334

Firstly, the ranking system. This year's seemed to particularly wonky. At a regional as large as NYC, eight matches per team simply don't cut it. How can a robot's abilities be assessed fairly in comparison with the rest when it mathematically does not have the chance to play with/against every other team. A lot of luck seemed to be involved.

Luck will always be involved. Time will always be a constraint. Unless a better, viable alternative emerges, this is how things will work.

Originally Posted by alex334

I would propose a larger implementation of the district model. That seems to work a lot better and attract smaller amounts of teams.

The district model also requires a more expansive program, even more experienced and efficient volunteers, and stronger outreach than NYC offers. The unfortunate reality is that despite efforts to do so, we haven't been able to introduce NYC to the district model precisely because we don't have the resources to make the district model viable here.

Originally Posted by alex334

The large amount of penalties (I saw 120+ points scored on just penalties) also seemed to skew the rankings, especially when fouls were counted as goal/climb points. Let's face it, not everyone reads the rules. Why punish teams that do by dragging them down? Why not find a better way to rank based on individual performance.

Because teams shouldn't be ranked just by individual performance, but how they contribute to an alliance. That match you mention? We were on the receiving end of those penalty points; we won that match 10-165 because of those penalty points. Our very next match, we were partnered with the team that incurred those penalties, coached them so that they wouldn't do so, and went on to win that match as well.

Moreover, individual performance isn't exactly something you can judge quantitatively - e.g. how do you compare offense with defense? 3137 played fantastic defense throughout eliminations and were critical to our wins, but they didn't earn our alliance any actual points.

Originally Posted by alex334

Secondly, rules. In general, I liked the rules a lot. The ones that bothered me were those regarding penalties (this is going to be a common theme in this post). "Obstructing the flow of the game." What's that all about? It just sounds like defense to me, not to mention that it's extremely subjective. When does a good defense begin to obstruct the game?

IMO, its intent is to encourage scoring over blockading - that is, to encourage active gameplay and discourage anything that inhibits active gameplay. Maybe it's not a "fair" rule, persay; certainly, it is very subjective precisely because of its ambiguity, and makes defense harder, but it's part of the game. And when the game itself prevents you from executing a certain strategy, you work around it.

Originally Posted by alex334

Instead of creating rules like this, why not just create a game in which this is not possible? Instead of placing two obnoxiously large pyramids in the middle of the field, with either side being easily blocked, why not create a climbing wall and leave a wide open field? Maybe place a different, smaller element there instead.

It's not exactly a new rule:

Originally Posted by 2012 Competition Manual

[G23]
Robots on the same Alliance may not work together to blockade the Court in an attempt to stop the flow of the Match.
This rule has no effect on individual Robot-to-Robot defense.
Violation: Technical-Foul

Originally Posted by 2011 Competition Manual

<G48-C> ALLIANCE ROBOTS may not work together to blockade the FIELD in an attempt to stop the
flow of the MATCH. This rule has no effect on individual ROBOT-to-ROBOT defense.

Originally Posted by alex334

Which brings me to my final complaint. This exact bias in ruling occurred during the elimination matches of the NYC regional this year. (http://www.ustream.tv/recorded/29860861). Not only do I find this unfair, but it all of the refs seemed to disagree on what the large penalty was for.

This is an inevitable problem that will arise when making judgments with respect to ambiguous rules - that is why there's a head ref, so that there can be someone to bring a close to the discussion.

Originally Posted by alex334

I asked him to consider watching videos of previous matches in which I would gladly point out more extreme scenarios only a few matches prior which were not penalized.

You aren't the first, nor you will you be the last person to argue with this policy, which FIRST explicitly lays out in the manual:

Originally Posted by 2013 Competition Manual

5.5.3 Referee Interaction
The Head Referee has the ultimate authority in the ARENA during the event, but may receive input from additional sources, e.g. Game Designers, FIRST personnel, and technical staff. The Head Referee rulings are final. The Head Referee will not review recorded replays under any circumstances.

Originally Posted by alex334

If you could prove your point using the footage available, should that not count?

If the footage you use to prove your point offers a misleading perspective of the situation, should it count?

This isn't something our Head Referee decided on the spot. This is an enduring FRC-wide policy precisely, because as Sam explained:

Originally Posted by IndySam

There is no way that FIRST can allow video reviews of matches. The NFL spends millions of dollars on equipment and gives the head refs many different angles to review a call and they still don;t always get it right. There is just no way for FIRST to implement a fair replay system, period.

Originally Posted by alex334

He refused and told me to relax, because "it's just a game" and the refs are "just volunteers." Well, needless to say, the majority of us are just volunteers. I personally spent two weeks of my college vacation helping my team and countless hours on the bus to see the regional. He told me that we still accomplished the mission of FIRST, since we learned a lot. He then walked away.

I am completely disappointed with this attitude. It is unprofessional. It is disrespectful to everyone doing FIRST and their efforts. I'm still pondering what the lesson learned was for my team. That life isn't fair? Is that such a great message? I personally don't care about winning. I'm done with FIRST. I just want others to get the experience that I got without having such great hopes shattered.

Originally Posted by 2013 Competition Manual

T13
If a TEAM needs clarification on a ruling or score, one (1) pre-college student from that TEAM should address the Head Referee after the ARENA reset signal (i.e. PLAYER STATION LED strings turn green). A TEAM member signals their desire to speak with the Head Referee by standing in the Red or Blue Question Box, which are located on the floor at each end of the scoring table. Depending on timing, the Head Referee may postpone any requested discussion until the end of the subsequent MATCH.

That he even listened to you instead of turning you away, as he easily could have done, is gracious in itself.

And you're wondering what the lesson learned for your team was? The lesson was your build season. You guys pulled yourself together, harnessed your resources, and put together an amazing robot. During both a strategy meeting yesterday and a celebratory dinner tonight, the consensus was that you guys had the most competitive robot there. There wasn't a single person at New York that wasn't impressed by your performance. The way you got there - the little things you did, the changes you went through, the experience you gained - that's what you come away with.

We consider our Lunacy robot, Michael1, to be the best robot we've ever built, even though it didn't go to CMP (which was partly because of abstruse FMS issues specific to our programming that caused us to lose matches) - and the lessons we've gained from Michael's season have driven our success as a team. Virtually everything about how our team operates today - our build criteria and process, the importance we place on driver practice, the effort we put into scouting - is driven by the lessons we learned from Michael's year. And as the cliché goes, those lessons we learned that year - about the importance of simplicity, of implementation, of efficiency - extend not just to FRC but life in general.

That's the kind of thing you walk away with.