We competed at Buckeye this past weekend and suffered comm issues in most of our later matches on Friday and Saturday.
We had lots of folks trying to help us figure it out from the FTA’s and other field personnel to mentors from other teams.
Here is what I know.
From looking at logs and reported observations from the FTA’s they see the radio rebooting. Immediately after they see the radio return they see the RIO so the belief is that the RADIO is the only think rebooting as the RIO takes longer than the radio and would not show up immediately after the radio if it was also booting.
Even so. The initial thought, before we found out the above, as that we had loos battery connections. We went over all our batteries and did find some that were marginal so we tightened ALL our battery connections regardless if they were loose or not. This did not solve the problem. The problem could not be duplicated in the pit no matter what we wiggled and banged on. We build a practice bot and had no issues with that all preseason. We had the radio from that along so we swapped it in and no joy with that either.
We pulled and re-seated all the wires in the weidmuller connectors on the PDP and VRM and tug tested. Again, no joy. By this time we were past our second match of elims and got subbed out. Not a problem, I would have done the same thing.
Before we bagged we replaced the radio power cable with the one we used on our practice bot and replaced the VRM with a brand new out of the box new one.
Have not had a chance to test since.
Does anybody have any idea what else we should try if this does not work?
Thanks
have you guys checked the mini automotive fuse. my team just unbagged our robot yesterday and it drove fine until we tried to go over any defenses. those fuses can come loose but still look like they are seated. hope this helps.
We had a similar problem and discovered that one of the fuses on the PDP was loose. (I believe one of the small fuses at the bottom of the PDP goes to the VRM?)
After making sure it was securely pushed in, we didn’t have further problems.
That might be something to look at
Did you check that the yellow and red fuses were pressed all the way into the PDP? When they are inserted fully, there should only about 1/4 inch of the plastic showing. In Kansas City, there were a lot of robots with similar issues. The CSAs at the event checked all of the robots Friday afternoon and a lot of the roboRIO and Radio issues went away.
we had several folks look at the fuses and they were deemed OK but I will pull them and insert new ones Thursday.
Thanks
When does the radio reboot occur? Can you post links to matches where the radio is rebooting?
-
Since you’ve replaced everything else, consider replacing the wire from the PDP to the VRM. Be sure you’re using stranded wire.
-
Make sure that all of the wire along the path is tacked down with zip ties (use zip tie bases if needed). You want to minimize wiggling.
-
How is your radio fastened to the robot?
I know I am a flame thrower on this topic so I should probably just sit on my hands and watch this thread, but I will say this much, as a fan in the stands at UMass Dartmouth over the weekend, it was galling to watch team after team (and probably more veteran teams than rookies) sit dead on the field for close to a minute while the radio reboots (or reconnects or whatever – their dead and that orange light isn’t flashing so you can be sure that they are not coming back alive for half the match if at all).
I know, I know, I KNOW, it is almost always a power issue for the radio. I hear you, FIRST, but seriously, you need to stop blaming the teams on this matter.
When you run a remote controlled robot tournament, the first and primary contract you make with the participants that you promise them a solid data link to their robot.
I believe that FIRST is failing in this promise.
I argue that we are at the point where we need to either
find an idiot proof solution (that a team can do whatever they want and the radio link stays alive)
or
inspect in the set of behaviors that gets to a solution (e.g. every robot going on the field gets inspected for the proper wiring connections, strain reliefs, etc.).
Blaming teams was never a great plan but this year with such long reconnection times, it is just unsupportable.
Dr. Joe J.
The problem is solved with hot glue.
I would LOVE to see FIRST legalize opening up the Radio and soldering a power cable directly to the PCB pins. Not having a solid, locking connector is the biggest issue with the radio, and if teams could solder to create their own locking connector, maybe with a powerpole that would be great.
1712 had too many simultaneous fixed to truly isolate which mattered and which didn’t (we didn’t have time to only change one variable at a time in the heat of competition). But I do believe that getting better strain relief on wires was a big part of us solving our roboRIO power issues, and may help you with your radio issues. Ensure your wiring has a sufficient bend radius near either end, and then get it tied down securely. Even momentary losses of power can cause reboots or brown outs. Check (and possibly replace) both your power and cat5 connectors. We also added an elastic band (hairtie) around our radio to help all the connectors stay plugged in. Further still, we cushioned our radio mount with foam to help absorb shock.
This sounds like a power quality issue. Bad wires, bad crimps, bad connections, bad barrels all can contribute to this.
I would replace all of the wires from the batter to the radio, especially the black wires that go directly into the radio. Then I would reprogram your firmware on the radio in particular. Inspect your fuses, many people don’t notice when they are bad. Drive the robot around at home, drive over bumps see if you can replicate these issues. Make sure all of your wires are anchored down appropriately and nothing can shake loose. Make a good effort to replicate the driving conditions on the field, move around fast and crash frequently.
With how difficult the field is on robots this year, i’m not surprised that we are seeing a rise in power quality related issues on the communications.
What is the battery voltage showing on the logs? How old are your batteries?
[Bad advice retracted].
Try driving the robot in a stall condition by putting it against a wall and driving forward. See if your radio restarts.
Please don’t do this. Even doing this once is enough to damage the terminals on the battery even if the crimps are lose. Its much better to take them off of the battery, and test the crimps off the battery.
Also teams, if you are crimping your own Andersen connectors, you can actually purchase a hydraulic crimper from Harbor Freight for not too much money. We bought it a few years ago, and love it. Haven’t had a lose crimp in years, and we even cut a few apart to inspect the test crimps and they are really solid.
http://www.harborfreight.com/hydraulic-wire-crimping-tool-66150.html
That is a valid point. I would recomend pull testing them a better way.
with velcro not the regular stuff that kind that sticks to itself whatever that is called.
Been doing that for years
hot glue where?
ya but the robot is in the bag
we have all kinds of batteries but we lost comms some times when we were just going under the low bar. No big current draws, no big bumps.
We had some issues with our radio and/or FMS at the Tech Valley Regional this weekend.
One of them was on us (tug test failed on our radio power connector), the other was indeterminate. (CSA/FTA was blaming robot power, as a whole, but we have video showing no power loss on the RoboRio.)
Regardless of what the problem is, even as a rookie team, the instrumentation provided by the RoboRIO/PDU/FMS/DS completely fails during problem events.
During the aftermath of our issues, we attempted to use the DS logs to trace voltage, power draw, comms, and log messages from the RoboRIO. The problem is, you can make “guesses” about your voltage state in the fraction of a second leading up to the event, but once it happens, you’re blind.
We’re already redoing our robot code library for next year to address some of these shortcomings. We’ll be logging voltage and PDU stats, as well as bridge/radio and FMS connectivity via ethernet directly on the RoboRIO. In addition we’ll be using a cached-logger, such that if DS communication is lost we can back-fill in our logs.
We are attempting to address the issue of the complete lack of FMS logging, and the logging dropout during any communication loss event. We’re also pushing the logging closer to the source of the data. Our goal is to be able to document the voltages, RoboRIO, PDU, and Radio/FMS/DS status in an effort to diagnose issues.
Do a pull test on all wiring connections. That means where a wire goes into a Weidmuller or Wago connector, where a wire goes into a crimp lug, where a wire goes into an Anderson connector and where the wires go into the plug for the RoboRio power. If wire slips out, re-do the connection.
Check all screw/lug connections such as the battery cables on the battery and on your main breaker. The wire and lug should not be able to rotate relative to the battery or breaker. Tighten the screws if they move.
Use a flashlight and examine closely where the wires go into the small Weidmuller connectors (i.e. RoboRio power on your PDP). Look for stray strands on any wire that is not in the connector. These stray strands can touch adjacent wires which is always the opposite polarity on the PDP, VRM and PCM. If necessary, cut off the “crinkly” stripped ends, strip to the length shown in the Users Manual for the PDP, VRM and PCM, twist the exposed strands so they lay neatly together then re-insert the wire in the Wiedmuller connector carefully and ensuring that ALL the strands go in.