My team is having an extremely odd radio problem that so far none of the other 26 teams in our competition are having.
We started having some radio issues before bag n tag that had to do with schedulers on the code (this could be said a little oddily as I am electrical lead and no code lead, but I’ll translate as best as I can). That problem was fixed, and we switched out our radio after the normal one died. We updated the firmware, and it gave us a little trouble getting programmed by the volunteers, but it ended up working.
The Problem
We are having a recurring comms loss. This is both on the field and off the field. It appears as if the roboRIO is rebooting. Most matches, we regain connection to the robot after 10-15 seconds, and usually the roboRIO leds go dark and the radio leds remain lit. The COMM and the MODE leds on the roboRIO are unlit. We this problem occurs, all LEDs on the driver station go dark.
When we use a physical tether to the robot, FRC Driver Station shows a connection to the Radio but no connection to the roboRIO.
Some things to check off All anti-virus programs on our driver station laptop have been disabled
The first thing I would look at is the power connection between PDP and Rio. Look for loose strands on either end that could be shorting between the red and black wires. The next think would be to make sure that the fuse on the PDP that protects the Rio is seated properly. I think it is the red 10 amp automotive type fuse.
Does it occur when the robot is jostled in any way, or is it purely random even when sitting still?
You say all the lights on the Rio itself are going dark- that’s not a comms issue, that’s a system failure issue. Something is causing your Rio to go into a stage 3 brownout, or crash, or have some other complete failure. Get that fixed before you go chasing any other chickens around. :]
Inspect all power connections between your Rio and the PDP, especially where the wire goes into the screw-in Rio plug. Swap the fuse in the PDP as well- they can start to fail internally (rare, but it happens, and they’re literally a two-cent part anyway). It might also behoove you to disassemble the Rio and blow it out with compressed air in case there’s a loose metal shaving causing a short somewhere.
Nab a multimeter and stick the pins for it into the plug on the Rio where the power wires go in. Monitor that closely and see if the voltage decides to drop when your Rio goes dark, assuming you can get it to fail on command. Just start working your way back from there towards the battery, and you’ll find the problem eventually.
Have you talked to your FTA? They may be able to tell you what is going wrong- whether the field can see your radio or not. You can also switch to the diagnostics tab on your driver station to see status lights for radio, rio, and FMS, which tells you everything the FTA knows about your robot connection on the field. This information is also in the driver station logs.
To me, 10-15s sounds like a rio issue, but I’m just a person on the internet. Check all your wiring between the battery and the rio. rio wago connector, PDP weidmuller connectors, PDP main terminals, main breaker terminals, battery terminals. Also make sure that there’s no short on the rio- if you turn it on, the power light should be green. If it is red or becomes red, check for swarf in your PWM, DIO, SPI, etc ports.
Which radio are you using? I’ve had issues with both of our OM5P-AC (the flat-topped ones) radios, but no issues with the -AN (curvy topped model). Your last issue sounds a lot like what I tried to deal with during inspection at Heartland- We would have no connection when using a tether after about ten seconds or so. Switching to one of our spare -ANs solved it altogether.
Are you using the KOP PoEs? Not going on the record that they have had issues… but we don’t use them.
In 2017 4607 ordered a number of these and they have worked flawlessly.
I believe that the MN LRI’s and CSAs are now supplying these same PoE’s to all teams in MN that do not have them on their robot.
We (4607) also had comm issues in 2015 as we had the radio plugged into the wrong port (5v vs 12v). It cost us our first 8 matches of the 2015 season. At one point we had 3 CSA’s and no one could diagnose the problem. On a positive note, it forced us into attending 2 regionals each season to help alleviate this type of problem - it was an expensive solution, but yay regionals!
Try to pull out the red 10A minifuse on the PDP. If you can do it without tools, or if it wiggles at all, it wasn’t fully inserted, and very well could have been causing the roboRIO to lose its power briefly. Is this a new PDP, or are you reusing one from last year?
To push it in all the way with your finger requires enough force to hurt. Make sure both the 10A red fuse for the roboRIO and 20A yellow fuse for the VRM (and PCM, if you use one) are completely in.
By the way, 10-15 seconds is not a radio issue. It sounds like a roboRIO reboot, which is confirmed by its LEDs going out. That’s almost certainly a power problem. If it were a problem with the radio, you’d lose contact for at least 30 seconds (if physically tethered through the radio’s second Ethernet port) and usually more like 55-65 seconds (if using the wireless connection).
Another power check should be at your battery. Make sure the leads connecting your Anderson plug are tightly secured to the battery. Loose connections there for a few milliseconds can cause the symptoms you’re describing.
Use the Big Al tug test! Agressively pull up on the wires that are attached to the PDP, RoboRio, VRM, main breaker, and PDP power input. Replace the ones that come out in your hand.
We had the same issue, it ended up being the wire from the battery to the breaker. If you lose contact for even a moment, the RoboRIO will sense the voltage drop and reboot to protect itself (the most severe level of its brownout protection). Because the contact loss is very quick if you simply have a slightly loose bolt, the rio will reboot immediately, and by the time you/an FTA approaches the robot, the power light will be back online as it reboots. This causes a comms loss for <30 secs (shorter than radio power loss).
Eventually, our problem got so bad that we would lose comms even when the robot was hardly moving. Try jigging every connection while connected to the driver station, and see if a certain jiggle replicates the issue. We actually saw our breaker sparking while jiggling the black wire entering the PDB, but you might not see anything that dramatic so have someone watching the rio lights and another person watching the DS lights/RSL.
Good luck!! Be sure to post if you find the problem, or if you don’t so we can offer more help!
This. We had this problem at Arkansas last year and couldn’t figure it out until an FTA came by and wiggled the connections on our breaker. It’s not an easy one to spot - but we had the same symptoms.
I have heard that the VRM (and possibly PCM, if existent) can interfere with the radio. The radio’s signal can also be blocked by metal (I think), so your radio’s space should be fairly open.
However, like others have said, this appears to be more of a power issue than a radio issue. Checking every fuse/breaker and wiring connection is great advice. In addition, check all ports for metal swarf. Some other things to check:
If you haven’t already, use a fully charged battery. I’ve seen radio connection loss due to a low battery, even when it didn’t appear low
Wires themselves, they could have broken internally (probably rare, but it does happen)
Main breaker; test a different one if possible
If you have or can get access to another one, swap out the RoboRIO and see if anything changes
Test the control system with only the basic components (no motors, pneumatics, etc.)
If the last step corrects the issue, slowly add everything back and test with each new item; when it happens again, you’ll know where the problem is
Use a different battery if you haven’t already
See if any components seem to have a higher operating temperature than usual – this could be indicative of a short
Make sure motors aren’t stalling or working too hard, check that gearboxes spin properly
This might not be related to your issue specifically, but one of the things I was told by the FTAs was that the new (flat-top) radios have a small design flaw. There’s a capacitor inside that shorts the radio when crushed, so try to mount your radio so that nothing is crushing it (e.g. no zip-ties).
Verify that all connections (bolted, crimped and Weidmuller connectors) from the battery through the breaker and PDP to the RoboRio are tight. Do a pull-test on all connections. If a bolted connections can be rotated using finger pressure, it needs to be tightened. Do not stop if you have found a bad connections since there may be others.
Use a flashlight and examine the wires going into the Weidmuller connectors and the power connector on the RoboRio to see if there are any stray strands or exposed conductor. The spacing is very small and stray strands can short out the power. If there is exposed conductor, see if it can be pushed in further or trim off excess length. Refer to page 9 of the VRM Users Manual for recommendations on how to prepare wires for the Weidmuller connectors.
When you think you have found and fixed all the bad wiring, do a pull test on all the connections again. Then use a rubber mallet and strike the chassis or the supports for the electronics panel in a variety of locations to verify that comms is not lost. Also tap on various locations on the electronics panel using the handle of a screwdriver to verify that comms is not lost.
I had to help fix two different robots at Lone Star Central this past weekend using the above techniques.
Is there any more info on this? We had a comms failure during elims last weekend that sounds like it may be connected to this issue (elevator PID control failed causing it to slam down rapidly on the frame, and the radio is zip-tied to a plate mounted off the frame in semi-close proximity to the elevator)… Would be interested to learn any additional insights that we could use to modify the bot for our next regional this week.
The FTAs didn’t say much more than what I already posted. They came to our pit after a match where we disconnected for the last 30 seconds of a round after another robot hit us. We fixed it pretty easily using some velcro tape (the crazy stuff that can stick to itself). I’m sure double sided tape would also work, although it isn’t nearly as practical.
Flexing the wires would also be a worthwhile test. If there are any holes in the insulation, the wire should be replaced. If the wire cannot be replaced, use heat shrink tube to patch them.
This sort of thing happened to us last year because of a bad roboRIO because of metal shavings getting into the ports and gyro so first make sure to protect the roboRIO but something else that happened to us was a cable went bad at kettering kickoff, we replaced it and it worked fine.