FIRST please fix....

[johnr]

First please fix auton- telop - auton bug before worlds. If you can't fix it at least get the old fog horn to see it and stop the match. Thank You

[Jared Russell]

We are in the process of gathering our driver station logs from SVR to hopefully help FIRST hunt down the bug. I recommend that you do the same.

The behavior we saw was that 1.5 seconds into autonomous mode, the field would transition into "teleop disabled" for 0.5 seconds, then resume auto mode for 8 seconds. This happened in two of our matches, and has apparently happened at other events as well (for example, we saw this issue in 118's logs from one of their events).

[mikemat]

Originally Posted by Jared Russell

The behavior we saw was that 1.5 seconds into autonomous mode, the field would transition into "teleop disabled" for 0.5 seconds, then resume auto mode for 8 seconds. This happened in two of our matches, and has apparently happened at other events as well (for example, we saw this issue in 118's logs from one of their events).

This is the same behavior we saw in MSC QF1-1 and it's first replay.

[tStano]

http://youtu.be/HH_CG-Jmpr8?t=1m

I think that might have happened in QF1-2 at Milwaukee. It looks like all the robots stop at once, and then we(1675; big yellow PVC forks pointing up) ram our alliance partner. Same code, stopped in match 1, not in 2. This behavior didn't make sense at all to me, or anyone on our team at the time, but this field issue might explain it. Perhaps it screwed up our Timer or something.

[JohnFogarty]

This happened to me not once or twice but every single time we played on the field but i also seem to have the bug within my practice mode of my DS. Ill be sending my data as well. I did some figuring with a dumb luck trick involving the delay command in java in order to prevent the bug from causing major issues. I can't exactly explain it very easiily because im not sure how or what I did to fix it. Ill post my logs and code in this thread to see if you all can benefit from it.

My guess is that any time the delay command is running it ignores anything its told from the fms.

[Jim Zondag]

Greg McKaskle from National Instruments was on site at MSC this weekend. He and the event FTAs were able to capture a log of this glitch event happening in QF1.1. Hopefully this will provide sufficient detail for FIRST to root cause and solve this issue.

[SkittlesCharge]

I think that might have happened in QF1-2 at Milwaukee. It looks like all the robots stop at once, and then we(1675; big yellow PVC forks pointing up) ram our alliance partner. Same code, stopped in match 1, not in 2. This behavior didn't make sense at all to me, or anyone on our team at the time, but this field issue might explain it. Perhaps it screwed up our Timer or something.

From the other alliance's point of view, I could see this being a possibility as well. One of our alliance partners, 2039, drove a bit farther than they usually did, so they missed their shot. Our team, 537, also experienced problems as our robot never fired its ball during autonomous and drove a little bit farther than normal. Maybe the encoders on 2039 and 537 were reset part way through. I could see that possibly causing these problems, as 2039 drove farther and we never reached the encoder value that we fire our ball at.

[Iaquinto.Joe]

Shouldn't the FTAs call a field fault and stop the match right after the glitch occurs? The second replay of QF1-1 at MSC should have been stopped much earlier.

[Bryan Herbst]

Originally Posted by Iaquinto.Joe

Shouldn't the FTAs call a field fault and stop the match right after the glitch occurs? The second replay of QF1-1 at MSC should have been stopped much earlier.

If they catch it, yes. However, we don't know if the FMS is reporting this hiccup (I didn't see this at all at 10,000 Lakes or Lake Superior). If the FMS doesn't show it and no one reports it, the field staff won't know that anything happened.

The other issue is that if it is an FMS glitch that happens in every match, they really have no choice but to keep playing the matches with the glitch (with reasonable warning to the teams of course).

[Iaquinto.Joe]

Originally Posted by Tanis

If they catch it, yes. However, we don't know if the FMS is reporting this hiccup (I didn't see this at all at 10,000 Lakes or Lake Superior). If the FMS doesn't show it and no one reports it, the field staff won't know that anything happened.

The other issue is that if it is an FMS glitch that happens in every match, they really have no choice but to keep playing the matches with the glitch (with reasonable warning to the teams of course).

They replayed QF1-1 twice due to the issue. Any observer could have predicted the possibility of the glitch reoccurring and seen the effects of it if it did. Instead of stopping the match after the red alliance missed once more, the FTAs decided to let the match run it's course.

[John]

I think this explains what happened to us in QF-2-1 at Northeastern. Auto appeared to reset a couple of seconds after the match started. Our robot ended up driving into the wall (the encoders reset at the beginning of auto) and never shooting (since it never got to the correct number of encoder ticks). I think 1761 missed their auto too. No replay, but luckily for us we won the second two matches. As far as I know it never happened to us again.

[Chris Hibner]

Originally Posted by John

(the encoders reset at the beginning of auto)

Our software protects against this kind of field fault so I guess it's possible this happened to us but we wouldn't know. We've been doing it this way for years for various reasons.

Here's how we do it (in case anyone is interested):

1) All of the sensor zero-point calculation and resetting is done in disabled.vi

2) We keep a global variable called "VisitedEnabled" which gets initialized to FALSE. As soon as the robot becomes enabled, this variable is set to TRUE and can never become FALSE again without a cRIO reboot (or a debug-mode override).

3) In disabled.vi (or your disabled mode code) sensors are rezero'd/cal'd ONLY if VisitedEnabled is FALSE.

[apalrd]

We always reset everything when we exit disabled mode (disabled bit goes from low->high). This is done as a safety features, so that the robot will never do anything when entering enabled mode. Autonomous is also reset when this happens. This is also a development feature, as we frequently run auto many times during calibration and we don't want to have to reset anything in the software realm to run again. If we are ever in a situation where the field disables us and robs us of a second of our 10 second autonomous mode, we will contest it as a field fault to the highest level required and demand a replay.

FMS stands for Field Management System. Its job is to Manage the Field, its FIRST job is to make sure the match runs properly (secondary tasks include saving scores, audience displays, ranking calculations, ...). This includes correctly forwarding communications between all teams playing (which it failed at the 2012 Championships), and correctly time the match (which it has failed at every event this year). There is absolutely no technical reason that the FMS should be more than 20ms off on any time event that happens during the match - The enabling transitions of the robots, the actuating of the hot goals, the lighting of the pedestal, etc. and yet the FMS still manages to screw it all up. We can get timing accuracy of 10ms on our robot controls, and we could achieve 5ms if we wanted to. I have systems at work and in FSAE that go down to 1us or less, on processor hardware that has been in production for around 15 years. Why can't the FMS maintain time to half a second?

FIRST is telling the teams (and the poor refs) that they are held accountable to every plane in space and many totally subjective calls, yet they can't maintain the match or hot goal timing to any reasonable accuracy by week 7. This is absolutely unacceptable. I am happy that the decision was made at MSC to replay the match as many times as necessary.

[Joe Ross]

Originally Posted by Chris Hibner

Our software protects against this kind of field fault so I guess it's possible this happened to us but we wouldn't know. We've been doing it this way for years for various reasons.
...
2) We keep a global variable called "VisitedEnabled" which gets initialized to FALSE. As soon as the robot becomes enabled, this variable is set to TRUE and can never become FALSE again without a cRIO reboot (or a debug-mode override).

When testing autonomous modes back to back, do you always reboot the robot or use the debug override?

[Adam Freeman]

Originally Posted by Iaquinto.Joe

Shouldn't the FTAs call a field fault and stop the match right after the glitch occurs? The second replay of QF1-1 at MSC should have been stopped much earlier.

We asked that question to the FTAs. They told us if they stop the match then no match log files are created. So they kept the match going to collect additional data to help determine the root cause of the problem.

They told us before the third match that they would stop it early if it happened again since they weren't going to learn anything new by continuing with the match.

The second time the issue showed itself slightly longer into auton (maybe 6-7s in), it almost seemed to be some sort of interaction between our CheezyVision timing and the field....since both times it seemed to occur right when we selected the HOT goal. We were going to stop using it, but the FTAs wanted us to keep everything exactly the same.

We ran the same auto for about 8 qual matches and every single elim match....so I don't know why the outcome would have changed.

It'd love to know when the true root cause is determined and see the data that proves it.