Our robot has experienced this issue at both the NYC and CT regionals, so I have a good amount of experience with the issue. Additionally, I managed to take note of some interesting patterns.

First, the patterns I observed at CT:

1. As the day went on, the odds of a dead robot seemed to increase greatly. On Thursday, Friday, AND Saturday we noted this trend. Basically, all the robots would always work until like the last 10 or 20 matches of the day, at which point we'd start to see dead robots. On Friday in particular I'd say maybe the last 10 matches in a row (and really, only the last 10 matches) featured a non-moving robot. I think most of the time it was the robot in Red-2, but I think in one or two cases it was coming from another station.

2. The problem isn't limited to just the red or blue alliance. We experienced the problem exactly once all weekend, and it was when we were in station Blue-1.

3. It seems to affect teams regardless of whether they're using a classmate or not. In other words, the problem doesn't seem to have to do with the driver station.

4. It is totally independent of the programming language being used. We (195) use Labview, 118 uses C++, and yet both of us had the issue.


Now, here is everything we know about what exactly has happened to us:

In New York City, we first experienced the issue on the first day of Quals. We would connect to the field normally (we would see the camera feeds from our robot and such), the driver station would be receiving input from the joysticks, and autonomous would run perfectly. When the 15 seconds was over the robot wouldn't move. I would move the joystick or press buttons and nothing would happen. Again, the driver station was receiving commands from the joysticks, but for whatever reasons the robot wasn't moving. The robot sat still the entire match.

Our next match, or possibly the second match after that one (still the same day) the issue re-appeared the exact same way as before. This time, about 30 seconds into the match, the FTA suggested rebooting the C-RIO. Our robot regained communication with about 15 seconds left in the match, and I had perfect control over it for those 15 seconds. Obviously we lost the match though. Yet another 0 QP match. However, now I knew at least that if the issue re-appeared, rebooting the C-RIO would "fix" the problem.

Now, a little after that we were talking about what could be causing the problem when we noticed something about both matches we didn't move:

1. Both times we ran an autonomous program where we backed up to the co-op bridge and tried to collect balls from it
2. We were on the red alliance both times (but all day there had been robots not moving on both sides of the field, so I was pretty sure the alliance station had nothing to do with it)

We stopped running that autonomous program (just in case that was the issue for some reason) and instead only ran the program where we shoot two balls then sit still for the rest of the 15 seconds (the 2-shot auto as we call it). For the rest of the regional we only ran the 2-shot auto and never had another problem, but at this point I'm convinced it was just a coincidence.

That first day of quals both during lunch and after the last match that night we were talking with the FTAs and trying to reproduce the problem. We were always able to connect to the field just fine, we always got our camera feed to the driver station and such, but I don't think they ever actually ran a "test match" or anything like that with us actually set up on the field the way we would be during a match.

We tried running the "practice" mode with the driver station tethered to the robot in the pits, but again we were unable to reproduce the problem.

At one point a Labview expert (whose name I cannot recall) came over to inspect our code to see if anything on our end was causing the problem. He said that everything in our code looked perfect, which ultimately isn't surprising given its consistency with my observations above (the problem being independent of the programming language).

So, for the rest of NYC we personally never had the issue again except for those two matches, although I know some other teams had either the same problem or a similar problem. I know that the Pasack Pioneers competed at an event (and won!) the week before and never had the issue, came to NYC running the exact same code, and experienced the exact same problem as us in a practice or qual match. I'm pretty sure another team struggled with the issue all weekend as well.


In between NYC and CT my programmer and I did some digging here on ChiefDelphi and found an entire thread devoted to this year's field issues. My programmer also got the idea to end our autonomous program ourselves as opposed to letting the FMS end it (I'm not sure which thread he got the idea from, but he said he saw it on CD somewhere). What this means is that at 14.5 seconds we tell our autonomous loop to stop executing.


Now fast forward to the CT regional. We run the 2-shot auto a few times and we also run the code where we back up to the co-op bridge. Neither causes a problem. After alliance selection we modified our autonomous code slightly so that our intake would accept the two extra balls from team 20. Again, we did not have a problem...

...until the second finals match. The autonomous worked fine as usual, but then when teleop started I moved the joystick and nothing happened. It was the horror from NYC all over again. Because of my experience there, I knew that the only way to solve the problem was to reboot the C-RIO and hope that I got comm back before the match was over, so I did it as fast as I probably could. I had the C-RIO rebooted like 2 seconds into the match, a bit before the FTA was even able to get to our driver station. He probably thought I was crazy, or maybe not. I just told him that I had no control, there was some issue, and that I knew I had to reboot the RIO ASAP.


So, what caused it? Honestly, I'm just completely at a loss. We ran an autonomous program that was working perfectly in every prior match. We never had an issue at all the entire time we were at the regional, along with the other top teams who encountered problems in the eliminations (118, 177, etc.). We were even on the blue alliance! All I know is that when that robot transitioned into Teleop I lost all control of it, and nobody seems to be able to explain why.


After the award ceremony I talked a bit with one of the mentors from 177 and 118's driver and both of them were at a complete loss as well. 118's driver said something about having to go to the field to meet with the FTA for some reason. A little while later, wondering what they called 118 to the field for, I went to their pit and asked one of their mentors what had happened. They said that they tried to reproduce the error with the FTA but that it was not done under real match conditions and that they were unable to reproduce the issue. It's remarkably similar to our experience in NYC, really. (Also, I should point out that I told them very much the same as what I posted here about my experience with the error, and that their team was extremely professional and polite even in spite of what had just happened to them.)


Now, about the issues team 20 had - in the second Quarterfinal match 20 didn't move for the majority of the match (might have been the whole match). I asked the FTA and he said that it had something to do with team 20's USB hub. I know that team 20 then went as far as to replace their entire driver station laptop.

I also know that in the timeout after the first finals match 20 actually replaced their entire C-RIO, thinking that their C-RIO had failed.

I think something happened to 20 in one or two other matches, but I am not totally aware of the specifics.


So now, my personal theories on what has been happening. Here's what we know:

1. It has nothing to do with the driver station. In the first finals match our robot worked fine. In the second it didn't move. In the third match it worked fine, and I changed absolutely nothing about our driver station between those three matches. (I'm the guy who always carries our control board around, the one that's shaped like a giant sword. It's my job to make sure it's set up correctly for each match, so I know that nothing changed.)

2. It has nothing to do with any code that's on the robot. Our code did not change one bit between those three matches, and our programming language is different than other teams who encountered the error.

3. It has nothing to do with wiring, because after you reboot the C-RIO everything works perfectly fine.

With those three possibilities eliminated, that only leaves three possible places the problem could be:

1. The firmware on the RIO has a glitch very, very deep inside it. This could explain why teams have absolutely no control over the issue and why the FTA is adamant that the problem isn't with the field itself. I actually like this explanation simply because it means that everyone at the event that day truly had no control over the issue, and it also means that everyone was right in saying "it's not our problem!"

2. Field system hardware. Imagine this - after a long day of processing and transmitting data some random electrical component somewhere between the driver station and the wireless routers is overheating and is causing something bizarre to happen every so often. I really like this explanation as well because it explains why the problem is so random yet so heavily biased towards one alliance station, and it also explains why it occurs mostly towards the end of the day.

3. Field system software. Some buggy code somewhere is causing robots not to receive the right signal for some reason.

It could also be a combination of the above.

Ultimately though, I can only present what I've heard and what I've encountered for myself. Hopefully team 20 can shed some light on what exactly happened to them in all the matches they had issues.