Connection issues at regionals

[Wes2443]

Aloha, this past weekend at the Hawaii Regional our team, as well as others, experienced communication issues. As far as we know, ours is unique (not sure, could be the static problem). When we turn on the robot during a match and it boots up, the COMMs light turns green but there is no connection to the Driver Station. We also suffered from this at San Diego in Week 1 and we learned there that doing a hard reset on the RoboRIO (holding RESET button, releasing then pressing it again) reboots it then it connects to DS, solving the problem. This happened occasionally at San Diego and every match at Hawaii (and the FTA person got annoyed for the hold up). In San Diego this happened in the pit too which rules out the FMS but it happened little in Hawaii's pits. We are pretty sure its not due to wires disconnecting because we don't lose connection after getting hit.

Our mentor said it could be due to the RIO starting up before the Radio can fully boot up. When we turned on the radio the four lights lit up but then shut off for a second, then started turning on again light it was rebooting. Other than the manual reset on the RoboRIO on the field, we had somewhat success turning on the robot before placing it on the field, depending on whether we plugged in the ethernet in the driver station laptop early or not. We already tried two separate RoboRIOs and Radios and the problem still persisted.

If anyone knows why this happens or a possible solution that would really help. Could this be a hardware problem on the RoboRIO or the Radio? We program in Java and we ran with all components (other than the driver station which we had to change to dynamic at Hawaii) on static IPs (i.e. Axis Camera and RoboRIO). Thanks.

P.S. Possibly a non-related problem, but during our Semi-finals match at Hawaii we intermittently lost connection when they were about to start which apparently reset our code/rio somehow, re-setting our autonomous selection though it still displayed on the SmartDashboard, and we were not able to re-set it. We don't think something in our code that ran killed the process. The FTA person said we had constant link to the robot the whole time but on the driver station there was a definite loss of communication for about a second right before they started the match. We aren't fully sure why this occurred also.

[Cecil]

As FTAA in Buckeye, we had this problem with another team, also using Java. It absolutely baffled everyone on the field, as we had them apply the firmware update again, re-image the rio, swap radios, etc. In the end, the most simple fix was this - put the ethernet cable into the other port in the radio. I'm guessing it is in the port furthest away from the power connector. Once we made the switch to the port closest the connector, it worked the first time every time.

It cost us several hours and most of our hair being pulled out, but the issue went away completely after the switch.

[ATannahill]

I assume that you mean the Comm light on the actual roboRIO, not on the DS. My first idea is that you have two instances of the driver station software running on your computer, possibly in different user accounts. I suggest you do a restart of your computer while in the queue to avoid this issue.

I don't think it is due to a difference in boot time on the radio and roboRIO, since every team is using the same roboRIO and radio, including the firmware.

There is a (small) chance that there is a problem with the image on your roboRIO, but I would rule it out because you replicated the problem with other roboRIOs.

[Alan Anderson]

Originally Posted by Cecil

In the end, the most simple fix was this - put the ethernet cable into the other port in the radio.

The vast majority of teams don't see this problem, but it's very frustrating for the few who do. It seems to be an interaction with specific roboRIOs. The team I helped with it couldn't/wouldn't move the Ethernet connector (they didn't want to change the way the radio was mounted). They eventually replaced the roboRIO and the problem went away.

[Cecil]

Originally Posted by Alan Anderson

The vast majority of teams don't see this problem, but it's very frustrating for the few who do. It seems to be an interaction with specific roboRIOs. The team I helped with it couldn't/wouldn't move the Ethernet connector (they didn't want to change the way the radio was mounted). They eventually replaced the roboRIO and the problem went away.

We thought the same thing on the field. Many robots had it plugged into the same port and worked, but this one absolutely refused. Seems weird that only a handful act up.

I actually misread the post - the OP is talking the comms light on the rio itself (I think). The one we had problems with wouldn't even have link lights on the ethernet connection light up - even if we unplugged the cable and reseated it. It seemed so much like a hardware problem, but the minor inconvenience of rebooting was better than trying to swap a rio. Then we tried the other radio port and the problem went away. Absolutely baffling that some work, some don't, even with the same firmwares all around.

[ayeckley]

Sorry to cast aspersions, but wasn't one of the major benefits of the new radio supposed to be that FIRST would be able to prevent these sort of issues since they controlled the firmware? We don't seem to have achieved that this year.

Edit: I'm referring specifically to the recurring issue of "plugging it into the other Ethernet port mysteriously fixed the problem" which I attribute to firmware/undocumented usage requirements. I appreciate that the majority of problems are related to things within the ability of teams to control.

[Jpatterson1710]

Originally Posted by ayeckley

Sorry to cast aspersions, but wasn't one of the major benefits of the new radio supposed to be that FIRST would be able to prevent these sort of issues since they controlled the firmware? We don't seem to have achieved that this year.

I was an FTAA at the Greater Kansas City, Oklahoma, and Smokey Mountain Regionals. Trust me, we did see improvements. You may not have personally seen them, but they exist.

The game is rough on robots, and that causes issues. Its hard for me to speak on any specific issue without seeing it myself, especially considering many issues have similar symptoms with vastly different resolutions. Overall, we should be praising the control system for working as well as we saw this year with such a rough game.

[Cecil]

Originally Posted by Jpatterson1710

I was an FTAA at the Greater Kansas City, Oklahoma, and Smokey Mountain Regionals. Trust me, we did see improvements. You may not have personally seen them, but they exist.

The game is rough on robots, and that causes issues. Its hard for me to speak on any specific issue without seeing it myself, especially considering many issues have similar symptoms with vastly different resolutions. Overall, we should be praising the control system for working as well as we saw this year with such a rough game.

Absolutely agree with this. We didn't have issues on the field like radios in the wrong mode, the dreaded "Christmas tree" (where someone was drawing too much bandwidth, effectively crippling the field), and they worked very well once connected. Some teams had radio reboots, just like every year, but we could usually pinpoint the cause to loose wiring. Just make sure your connections are tight this year.

[SuperBK]

Going with my standard response: Make sure the firewall is turned off on the drivers station. Before we turned it off, I would see driver's station Com light turn green after rebooting the Roborio.

What version of windows is the driver's station?

Also, we switched to static IP addresses.

Brian

[Jpatterson1710]

Originally Posted by SuperBK

Also, we switched to static IP addresses.

That's funny, because as an FTAA my first step to troubleshoot issues at the DS is to make sure its set to DHCP.

[Jpatterson1710]

Originally Posted by Cecil

the dreaded "Christmas tree"

Bingo... Not impossible now, just incredibly unlikely.

[Cecil]

Originally Posted by Jpatterson1710

That's funny, because as an FTAA my first step to troubleshoot issues at the DS is to make sure its set to DHCP.

Static IP addresses WILL work on the field - you just have to make sure the network is 10.TE.AM.(number less than 20) and the subnet mask is correct to talk to the FMS server.

Subnet masks on the robot have to be 255.255.255.0

Subnet mask on the DS has to be 255.0.0.0

More information can be found on Screensteps.

We used it a few times for teams that couldn't use their cameras in the pits. This cleared up their problems and worked without problems on the field.

[plnyyanks]

Originally Posted by SuperBK

Also, we switched to static IP addresses.

Originally Posted by Jpatterson1710

That's funny, because as an FTAA my first step to troubleshoot issues at the DS is to make sure its set to DHCP.

Me too, I'm pretty sure it's not recommended anymore to use static IPs with the field.

IIRC, the problem last year was going between the field and the pits - one environment had a DHCP server and the other didn't. So when you went from the field to the pits, the DS laptop realized it's old DHCP lease from the field was invalid and tried to obtain a new one, but since there was no DHCP server, it timed out (slow!) and fell back to a 169.whatever link-local address. This wasn't a connection problem, per se, since mDNS could handle it once everything resolved. But it could take some time for everything to work (and may have required a force DHCP refresh by disabling/reenabling the adapter).

One way around this problem is to take last year's dlink radio, turn the wifi off, and use it as a DHCP server in the pit that you connect your DS and robot to so you can tether.

I also think the DS update this year that enabled regular DNS in addition to mDNS when finding the robot may have had some effect.

[ATannahill]

The recommendation is to use DHCP unless you have a reason not to, most of the time being an additional ip based device. If that is the case than CSAs are told to use static IPs and they will work fine with the field if the last octet is less than 20 and not 1 or 4. As a rule of thumb I use 2 for the roboRIO and 5 for the DS.

The robot should not have a DHCP server when in the pits. Edit: An external router can act as a DHCP server in the pits if not wifi enabled.

[Wes2443]

Originally Posted by rtfgnow

I assume that you mean the Comm light on the actual roboRIO, not on the DS. My first idea is that you have two instances of the driver station software running on your computer, possibly in different user accounts. I suggest you do a restart of your computer while in the queue to avoid this issue.

Yes the Comms light on the roboRIO were on but not on the DS, and I doubt there were two instances because it happened across regionals/days/matches were we restarted the power many times.

Originally Posted by SuperBK

Going with my standard response: Make sure the firewall is turned off on the drivers station. Before we turned it off, I would see driver's station Com light turn green after rebooting the Roborio.

What version of windows is the driver's station?

Also, we switched to static IP addresses.

Brian

Originally Posted by Jpatterson1710

That's funny, because as an FTAA my first step to troubleshoot issues at the DS is to make sure its set to DHCP.

The firewall on the DS is off and runs Win7. The FTAA did in fact tell us to switch the DS to DHCP (not in SD though).

If it helps we have the ethernet to the roboRIO connected through the port furthest away from the power. Also we did try experimenting with using a ethernet switch to test our camera in the pit. In the pit we had to switch to static IPs (had last octet was 40) and switch to DHCP on the field.