Aloha, this past weekend at the Hawaii Regional our team, as well as others, experienced communication issues. As far as we know, ours is unique (not sure, could be the static problem). When we turn on the robot during a match and it boots up, the COMMs light turns green but there is no connection to the Driver Station. We also suffered from this at San Diego in Week 1 and we learned there that doing a hard reset on the RoboRIO (holding RESET button, releasing then pressing it again) reboots it then it connects to DS, solving the problem. This happened occasionally at San Diego and every match at Hawaii (and the FTA person got annoyed for the hold up). In San Diego this happened in the pit too which rules out the FMS but it happened little in Hawaii’s pits. We are pretty sure its not due to wires disconnecting because we don’t lose connection after getting hit.
Our mentor said it could be due to the RIO starting up before the Radio can fully boot up. When we turned on the radio the four lights lit up but then shut off for a second, then started turning on again light it was rebooting. Other than the manual reset on the RoboRIO on the field, we had somewhat success turning on the robot before placing it on the field, depending on whether we plugged in the ethernet in the driver station laptop early or not. We already tried two separate RoboRIOs and Radios and the problem still persisted.
If anyone knows why this happens or a possible solution that would really help. Could this be a hardware problem on the RoboRIO or the Radio? We program in Java and we ran with all components (other than the driver station which we had to change to dynamic at Hawaii) on static IPs (i.e. Axis Camera and RoboRIO). Thanks.
P.S. Possibly a non-related problem, but during our Semi-finals match at Hawaii we intermittently lost connection when they were about to start which apparently reset our code/rio somehow, re-setting our autonomous selection though it still displayed on the SmartDashboard, and we were not able to re-set it. We don’t think something in our code that ran killed the process. The FTA person said we had constant link to the robot the whole time but on the driver station there was a definite loss of communication for about a second right before they started the match. We aren’t fully sure why this occurred also.
As FTAA in Buckeye, we had this problem with another team, also using Java. It absolutely baffled everyone on the field, as we had them apply the firmware update again, re-image the rio, swap radios, etc. In the end, the most simple fix was this - put the ethernet cable into the other port in the radio. I’m guessing it is in the port furthest away from the power connector. Once we made the switch to the port closest the connector, it worked the first time every time.
It cost us several hours and most of our hair being pulled out, but the issue went away completely after the switch.
I assume that you mean the Comm light on the actual roboRIO, not on the DS. My first idea is that you have two instances of the driver station software running on your computer, possibly in different user accounts. I suggest you do a restart of your computer while in the queue to avoid this issue.
I don’t think it is due to a difference in boot time on the radio and roboRIO, since every team is using the same roboRIO and radio, including the firmware.
There is a (small) chance that there is a problem with the image on your roboRIO, but I would rule it out because you replicated the problem with other roboRIOs.
The vast majority of teams don’t see this problem, but it’s very frustrating for the few who do. It seems to be an interaction with specific roboRIOs. The team I helped with it couldn’t/wouldn’t move the Ethernet connector (they didn’t want to change the way the radio was mounted). They eventually replaced the roboRIO and the problem went away.
We thought the same thing on the field. Many robots had it plugged into the same port and worked, but this one absolutely refused. Seems weird that only a handful act up.
I actually misread the post - the OP is talking the comms light on the rio itself (I think). The one we had problems with wouldn’t even have link lights on the ethernet connection light up - even if we unplugged the cable and reseated it. It seemed so much like a hardware problem, but the minor inconvenience of rebooting was better than trying to swap a rio. Then we tried the other radio port and the problem went away. Absolutely baffling that some work, some don’t, even with the same firmwares all around.
Sorry to cast aspersions, but wasn’t one of the major benefits of the new radio supposed to be that FIRST would be able to prevent these sort of issues since they controlled the firmware? We don’t seem to have achieved that this year.
Edit: I’m referring specifically to the recurring issue of “plugging it into the other Ethernet port mysteriously fixed the problem” which I attribute to firmware/undocumented usage requirements. I appreciate that the majority of problems are related to things within the ability of teams to control.
I was an FTAA at the Greater Kansas City, Oklahoma, and Smokey Mountain Regionals. Trust me, we did see improvements. You may not have personally seen them, but they exist.
The game is rough on robots, and that causes issues. Its hard for me to speak on any specific issue without seeing it myself, especially considering many issues have similar symptoms with vastly different resolutions. Overall, we should be praising the control system for working as well as we saw this year with such a rough game.
Absolutely agree with this. We didn’t have issues on the field like radios in the wrong mode, the dreaded “Christmas tree” (where someone was drawing too much bandwidth, effectively crippling the field), and they worked very well once connected. Some teams had radio reboots, just like every year, but we could usually pinpoint the cause to loose wiring. Just make sure your connections are tight this year.
Going with my standard response: Make sure the firewall is turned off on the drivers station. Before we turned it off, I would see driver’s station Com light turn green after rebooting the Roborio.
Static IP addresses WILL work on the field - you just have to make sure the network is 10.TE.AM.(number less than 20) and the subnet mask is correct to talk to the FMS server.
Subnet masks on the robot have to be 255.255.255.0
Me too, I’m pretty sure it’s not recommended anymore to use static IPs with the field.
IIRC, the problem last year was going between the field and the pits - one environment had a DHCP server and the other didn’t. So when you went from the field to the pits, the DS laptop realized it’s old DHCP lease from the field was invalid and tried to obtain a new one, but since there was no DHCP server, it timed out (slow!) and fell back to a 169.whatever link-local address. This wasn’t a connection problem, per se, since mDNS could handle it once everything resolved. But it could take some time for everything to work (and may have required a force DHCP refresh by disabling/reenabling the adapter).
One way around this problem is to take last year’s dlink radio, turn the wifi off, and use it as a DHCP server in the pit that you connect your DS and robot to so you can tether.
I also think the DS update this year that enabled regular DNS in addition to mDNS when finding the robot may have had some effect.
The recommendation is to use DHCP unless you have a reason not to, most of the time being an additional ip based device. If that is the case than CSAs are told to use static IPs and they will work fine with the field if the last octet is less than 20 and not 1 or 4. As a rule of thumb I use 2 for the roboRIO and 5 for the DS.
The robot should not have a DHCP server when in the pits. Edit: An external router can act as a DHCP server in the pits if not wifi enabled.
Yes the Comms light on the roboRIO were on but not on the DS, and I doubt there were two instances because it happened across regionals/days/matches were we restarted the power many times.
The firewall on the DS is off and runs Win7. The FTAA did in fact tell us to switch the DS to DHCP (not in SD though).
If it helps we have the ethernet to the roboRIO connected through the port furthest away from the power. Also we did try experimenting with using a ethernet switch to test our camera in the pit. In the pit we had to switch to static IPs (had last octet was 40) and switch to DHCP on the field.
This is true. It’s much more simple to use DHCP. Additional IP devices aren’t usually an issue if the IP on that device is set correctly. (Should be under 20 on the last octet.) FMS begins assigning the DS at 20, reserving lower addresses for these cases.
Wes, if/when this happens, did you happen to notice if the link lights on the RoboRIO were on?
This is one of the indications that the radio and roboRIO link didn’t come up and one of the devices needs to be rebooted – we generally reboot the roboRIO because it is faster. This is likely caused by power saving features on one or both devices, and we are hoping to fix this for good, but it happens quite rarely, so didn’t get enough instances during beta. The general guideline is to swap to the other radio port. It appears the the one near the power does this less frequently, but there are only two, so it is a quick experiment.
FIRST does have much more control over the firmware, but things like this are rare, and there are many configuration options. Why would they fiddle with POE and power settings away from the factory settings unless they see an issue? I think there are already far fewer gremlins in the system than the end-of-life Dlink, but with a season and another beta period, we will really know how beneficial the open image is.
As for static and dynamic IPs. The majority of teams are configured for dynamic, or the CSAs and FTAs do it for them. Occasionally, a team wants to be static, which is compatible with the field DHCP, so we ensure that they stay out of the range of the dynamic server and have the recommended values. As long as you are consistent, both work fine. I prefer DHCP in almost all instances, but let the team decide after I explain why their current settings were not working.
Note that it was specifically noted in Update 8 that you should plug into the other port. I think that would be worth a try.
**Control System Note:**We have heard sporadic reports of link failure between the roboRIO and the radio after a cold boot, characterized by the amber and green link LEDs on the roboRIO Ethernet port remaining off (and a roboRIO reset resolving the issue). We have located a set of hardware that reproduces the issue and are currently investigating the root cause. In the meantime, teams experiencing this issue can connect the roboRIO Ethernet to the radio port labeled “18-24v POE” (next to the power connector) as a temporary workaround (the issue only affects the port labeled “802.3af POE”).
I believe the COMMS light was on on the roboRIO, didn’t notice the link light. Now I’m starting to think whether disconnecting the ethernet and reconnecting it on the DS would help. We only used static IPs on the rio and camera components because we were testing it with the ethernet switch which wouldnt work on DHCP.
We will try switching the ethernet port on the radio. We also noticed long startup times on the radio. On the typical startup, the radio would boot and light would come on normally, then the lights would go off then start acting like it is rebooting which confused many of us.