During our regional last weekend, we found that our robot would sometimes disconnect for 10-15 seconds after collisions with either other robots or sometimes the wall. The lights on the robot remained on but the signal light would turn solid and the driver station light began blinking (this is confusing why this is the opposite btw). It was very inconsistent when it would happen. Obviously, this is a serious problem because in one match we were unable to drive for as many as 30 seconds after multiple collisions, resulting in lost gear and climbing potential.
Does anyone have any ideas why this is occurring and any suggestions for how to prevent these problems in the future? The CSA at the time told us we may have loose wires which we double checked and found them a few times, but not every time. And even after fixing the loose wires between matches, we would still run into problems in future matches.
If smacking makes it stop working, it’s a hardware issue. Check all your connections again, as well as part placement and the like. Could be an internal issue in the rio or something too as well.
10-15 seconds leads me to believe an electrical short situation and/or roborio reboot. If it was longer (45 seconds), that would indicate a radio reboot. Because it is happening after a collision, it definitely makes me think something loose. The FRC Driver Station has logging capability that might assist your search. It will also indicate any electrical shorts that might have occurred and which voltage rail it happened on.
We got rocked pretty hard in a match at CVR two weeks ago. Our REV More Board actually popped out of the roboRio!
We lost comm for about 30s. We determined that the more board hit the ground wire powering the roborio and caused it to momentarily disconnect. So the rio had to reboot.
The ground wire for the rio was resting in its hole, but was no longer screwed in. So it appeared connected.
Last year we (as did many people) had terrible times with our radio losing power for a split second and rebooting too.
Those are probably symptoms of your roboRIO losing power momentarily.
You can check your Driver Station log for the match to see if data from the robot disappears for those seconds.
Typically, such a power loss is due to:
the wires connecting the PDP->roboRIO having exposed copper at the connections on either end, and/or
the connectors at either end partially clamping the insulation, and/or
the wire not being securely tightened at the roboRIO power connector, and/or
the roboRIO power connector itself not being securely screwed into the roboRIO case, and/or
the PDP connectors not able to be tugged firmly, and/or
I would be willing to bet that your radio is being disconnected. The power cable for the radio doesn’t have good contact and can break the circuit briefly when the robot is hit. Unfortunately, this forces the radio to reboot, which can take a while. Let’s go through the symptoms:
Disconnects for 10-15 seconds - This jives nicely with the boot time of the radio, especially when you say it can take up to 30 seconds at times. It might be a little bit short, which suggests that it might be the rio (which tends to boot up faster), but this seems less likely because the rio has less vulnerable points of failure.
Lights on the robot remained on - The battery would continue to provide power, keeping the speed controllers and whatnot connected and illuminated
Signal light turned solid - This means that your robot has been disabled. The robot automatically disables itself when it becomes disconnected from the driver station.
Driver station light began blinking - See above. I can understand the confusion, but yes, if the light over your driver station is blinking, that means you are disconnected.
Inconsistency - The power connection is very spotty, sometimes it stays, and sometimes it doesn’t.
Solution: secure your radio power wire. This is, unfortunately, easier said than done. We’ve found (we had this issue a couple times last year) that electrical tape is the easiest and most straightforward solution to the problem. You can also use a zip-tie wrapped around the radio to hold it in.
In that vein, a bad battery cell or two can cause a sudden drop of 2-4 volts.
All of these would be quite evident in your Driver Station log (the rosette on the top right-center of the Driver Station window).
From what I’ve seen, the radio typically takes a lot longer than 10-15 seconds to reboot. I suspect it’s either a roboRIO power issue or an issue with the ethernet cable between the roboRIO and the radio (although the latter is probably less likely, since ethernet cable issues tend to be resolved quickly or not at all).
The driver station logs should definitely be helpful here. There should be a message shortly after reconnecting indicating how long the roboRIO has been up - if you see something like 5 seconds halfway through a match, that’s a good sign that your roboRIO lost power briefly. (I’ve seen this message not show up sometimes, though, so your luck here might vary.)
As for the blinking stack lights in the driver station: yes, they’re inconsistent with the RSL, but on the field, I’ve found that they make it easier to spot disconnected robots, regardless of the match phase.
Yes, I thought the same. I made an edit to my post:
Disconnects for 10-15 seconds - This jives nicely with the boot time of the radio, especially when you say it can take up to 30 seconds at times. **It might be a little bit short, which suggests that it might be the rio (which tends to boot up faster), but that seems less likely because the rio has less vulnerable points of failure.
**
roboRIO reboot is ~15 seconds (communications indicator of the Driver Station), plus user code startup time (Robot Code indicator on the Driver Station).
radio reboot is ~39 seconds, plus a second or two to reconnect to the field AP, so ~42 seconds overall.
All this is visible in the Driver Station log.
If both devices reboot, the data gap in the log will reflect the radio time to reboot.
Check for shorts in the roboRIO, when inspecting recently there was a team recently and one of their PWM wires in the roboRIO was causing a short which the roboRIO detected and disabled itself as a safety precaution.
Test this by taking out all the data wires to the roboRIOand plugging each one in individually to test for shorts
The CSA at the time told us we may have loose wires which we double checked and found them a few times, but not every time. And even after fixing the loose wires between matches, we would still run into problems in future matches.
You have more loose wires, probably nicked insulation, maybe bad connectors.
Try thinking about it this way: you’ve found loose wires more than once. Why have you found more than one? What’s the root cause of all these bad wires? What will you do to fix it?
(note this comes from a place of understanding and ‘tough love’, 841 had this exact symptom at SF Regional this weekend & we took 3 matches to find the RoboRio power screw issue described above. Our students did a bunch of electronics cleanup live at the regional, and now we’re exploring using most of our 30lbs to drop in a 100% new electronics board with 100% soldered or ferrule’d wires.)
We had a similar symptoms in our semifinal match at Lone Star Central. Ours was caused by a very hard hit with the tower in autonomous that dislodged our battery and caused it to hit our pd board earlier in the day. We did not notice that this cause the little fuse for the Rio to bend and pull out slightly. Then later we had a pretty good hit that left us motionless because the fuse was damaged and loose. It took a few min to diagnose but we found when pressure was put on the fuse lights started to flash.
You don’t actually have to go unplugging everything.
The Driver Station narrows it down for you.
Any shorts in any of the roboRIO power rails are documented on the bottom tab of the Driver Station. The tab labeled Power & CAN.
The tab itself shows a red bar if an error has been detected, so you can see there is a problem without having the tab active.
The roboRIO itself will show a red power light if there is a constant short anywhere.
The tab will tell you if its a short on the 6v rail (PWMs), the 5v rail
(DIOs), and/or the 3.3v rail (secondary communications and some other things).
Are all your electrical connections tight? Do a pull test on every crimped connection. Do a gentle pull test on all the wires going into the Wago and Weidmuller connectors on the roboRio, PDP, VRM, PCM. Verify that none of the crimp lugs that are held on by a screw or bolt cannot turn when you apply finger pressure. Start at your battery and work your way to the outputs of the motor controllers (if they use such connections). Get someone meticulous to repeat these checks.
If it still happens, put the robot up on blocks so the wheels don’t touch the ground (or table). Use a large rubber mallet or a piece of 2 x 4 and apply some sharp Technical Taps (hit the frame). Start on one side and go all the way around. Use horizontal as well as vertical strikes. Watch what moves or vibrates and correlates with your roboRio resetting. You can be running the drivetrain to make it more obvious when there is a reset. Hold a medium sized screwdriver by the end of the shaft and use the handle to tap on the case of various electronic components such as the roboRio, the PDP, the VRM, etc. and watch the behavior of the robot.
Something we noticed on our practice bot a few days ago when it took a slight tumble from a bad knot on our makeshift davit, the OM5P-AC (2017 radio) was rebooting. We were able to recreate the issue by tapping on the case of the radio. Connections were checked to see how secure they were. Connections were fine. Viciously wiggled wiring going into radio, no reboot, tap on the radio body, reboot. Very strange. Swapped with a 2016 radio and no issue since.
It might have just been a strange anomaly defect but something to consider, good luck finding the gremlin.
This may mean the fall jarred the power input plug and made it loose. It may also mean that the circuit board inside might have been cracked in the fall.
if you are seeing intermittent connections to the robot but power remains on I’d also be checking the data connection between the RoboRio & the radio. Make sure the data cable is not under any strain. You may want to check the cable with a cable tester to ensure no breaks in the wiring of the cable. Avoid putting sharp bends in data cables.
Also check the RJ-45 plugs on both ends to ensure the plastic lock tab that pokes up from the side of the plug is not broken. This should click into place when the plug is inserted into the socket. If it is broken the plug will most likely pop out. I’d also check the RJ-45 socket as well to ensure no debris is present - especially if your socket faces upwards (e.g. RoboRio is mounts flat on the bas of the robot). A puff of compressed air every now and then (e.g. each comp session) should clear anything in the way.
Just want to emphasize Lil’ Lavery’s second point. 4118 was experiencing similar symptoms you described at Orlando this year. It took hours of checking all the wires and connections before we were able to figure it was the main breaker. If you haven’t yet, try switching that out. It was hard to replicate the issue in the pits, so you really need to hit the robot hard. Try driving it into a wall before and after switching the breaker to see if it worked.
