Possible FMS problems.

[techhelpbb]

Let's start from the bottom with this:

Please locate a multimeter with a Min/Max function, put it on your robot setup to measure minimum voltage DC and attach it to your battery lugs.

Then drive your robot around for 3 minutes (longer than a match) and see what the lowest voltage recorded is. Please post that here.

Then drive your robot around gently bumping stuff and see what the lowest recorded voltage is (similar to a situation with a field robot collision) and post that lowest recorded voltage here.

Generally a multimeter records voltages in Min/Max about 1,000 times a second. That's not bad considering the most (not all) critical electronic loads on the robot have additional capacitance which will ride through short drops.

Now once you achieve this, let's keep in mind that at high current you might have wiring somewhere that's too high in resistance (poor connections or an intermittent short). To test this put that same multimeter on the power inputs to these critical electronic devices (cRIO, radio, radio DC/DC converter, digital side car) again set to minimum voltage DC and repeat the tests above. Post that data.

With this in hand you are not depending on any aspect of your FIRST robot to measure the power supply critical to your robot. In fairness usually the driver's station charts will show voltage drops - but just in case there's something not right there this process I've outlined will show such problems as well.

Also I may have missed something but which programming language did your team use for your robot? C++, Java or LabView? Does your robot use vision?

Keep in mind that the field system does have some very real timing differences from other private wireless and wired robot situations. It's possible to cause yourself some headaches. This year I've seen problems with putting the Java camera get instance in the robot init section (this is a perfect example it only was an issue on the real field). I've seen problems with LabView and one team's attempt to use the standard compressor and pressure switch cause their robot to reboot consistently because LabView kept saying that relay 1 was an invalid index (clearly that's not an invalid index and strangely if you edit the block and set that directly it's fine for a bit and then the problem returns). Also with C++ you can easily shoot yourself in the foot (not so much a problem with C++ but just the nature having that much power it can blow up).

So I advise you even if you provide the voltages I request above to take your code and features apart starting at the most simple and adding on till you reach the full compliment. In this manner if one thing is causing headaches you can isolate which one(s) require more evaluation.

[Foster]

As always techhelpbb has a good set of steps to follow.

He asked

Please locate a multimeter with a Min/Max function, put it on your robot setup to measure minimum voltage DC and attach it to your battery lugs.

Putting on my psychic problem hat (*) I'm wondering if there is a correlation between your battery falling off the robot issue and the power cable and attachment to the Power Distribution Board. I'm thinking that the match was not the first time the battery has come off the robot.

So I'd like you to Please locate a multimeter with a Min/Max function, put it on your robot setup to measure minimum voltage DC and attach it to your Power Distribution lugs. That way you will be able to see if there is a problem with your main power feed.

Report back with the stuff that's been asked in the prior threads, that should exactly pinpoint the problem.

Good luck!

(*)Yes, I said psychic hat. I know the difference since I have one of each.

[Bald & Bearded]

To echo above. Check your software. When on the field several tasks on the CRIO interact with the FMS that pretty much sit idle when on a practice field.
If you have some sort of memory/buffer overrun or bad pointer in C++ that can cause that type of behavior.

[RufflesRidge]

Originally Posted by Bald & Bearded

To echo above. Check your software. When on the field several tasks on the CRIO interact with the FMS that pretty much sit idle when on a practice field.
If you have some sort of memory/buffer overrun or bad pointer in C++ that can cause that type of behavior.

Software will not cause your radio to reboot. This definitely sounds like a power problem.

[Secretspy97]

We had a similar problem this year. During driver practice we would be able to run the robot for about 3-4 minutes. After that the cRIO would randomly reboot or we would pop the 120 amp main fuse. We checked if the frame was grounded or if any metal shavings were found in the PD board and found nothing. This problem continued multiple times afterwards but we still haven't diagnosed the problem because the next day was bag and tag day.

However after searching chief delphi for answers we found this: http://www.chiefdelphi.com/forums/sh...ad.php?t=91733

We use a Banebots 775 to tilt our shooter. It is directly mounted to our aluminum frame. Since the motor only shorts out as different spots. This can explains why you don't see any electrical problems when we test for conductivity to the frame. We currently think that this may be our problem.

If you use a Banebot I would disconnect power to it and see if you experience the same problem.

[Jarren Harkema]

Originally Posted by Bald & Bearded

To echo above. Check your software. When on the field several tasks on the CRIO interact with the FMS that pretty much sit idle when on a practice field.
If you have some sort of memory/buffer overrun or bad pointer in C++ that can cause that type of behavior.

Originally Posted by RufflesRidge

Software will not cause your radio to reboot. This definitely sounds like a power problem.

We were having similar problems with our radio/cRIO rebooting. We aren't exactly sure what happened, but usually right after auto or half way through teleop, we would loose comms for 15 or so seconds. The radio power light would go red, and then turn back blue. Upon looking at the driver station log, just before loosing comms, there were a bunch of errors relating to the camera. Simply unplugging the camera ethernet camera resolved the issue. Somehow we must have a memory leak in our vision code causing the cRIO to reboot. but its weird that the radio would reboot as well, as it would not have lost power.

[Canon reeves]

Thanks for the great advice guys, I think during one or two of the matches we did have a loose battery lug, and we also had a loose anderson, we found this out by shaking them while it was on and causing it to cut out. I will try this week to give yall the information. We use labview, I'm not sure what the code exactly looks like because I don't program, but I will get that info as well. The big question that has me stumped is how most of these problems that yall talked about wouldn't occur off the field, even when we ran into the wall vigorously a lot? We only verified a D-link reboot twice, and that was on matches that we had definite battery problems. By no means am I an expert on any of this so please explain why it makes sense. Thanks for the input!

[Alan Anderson]

Originally Posted by Canon reeves

The big question that has me stumped is how most of these problems that yall talked about wouldn't occur off the field, even when we ran into the wall vigorously a lot?

One of your mentors asked me that several times, and my answer is still the same: running into the wall is a jolt in one direction. Being bumped by another robot from the side, or from behind, is another direction. If there's a loose connection that causes problems related to being hit, it won't necessarily show up with a hard impact to only the front of the robot.

The one time I was close enough to see the speed controllers' LEDs during one of the on-field episodes, the robot was definitely completely without power immediately after being bumped from behind. Some ten seconds later, a second gentle bump on the corner coincided with the power coming back on. Nothing in software or the field communication can cause that kind of failure.

We only verified a D-link reboot twice, and that was on matches that we had definite battery problems. By no means am I an expert on any of this so please explain why it makes sense. Thanks for the input!

The D-Link rebooting was only verified visually twice (it was mounted on the robot such that the lights are only visible from a narrow angle of view). The real-time field data monitor showed a loss of radio communication every time I was able to watch, and the FTA says it happened every time your robot had a problem on the field, though the logs I was able to review were only clear about the cRIO reboot and were inconclusive about the D-Link (the Driver Station apparently doesn't ping the bridge during enabled mode, and the FMS stops recording when the cRIO is not responding).

The one "smoking gun" we identified was the intermittent SB50 connection. Right after that penultimate match, it was definitely easy to cause the robot to lose power by pushing on the red wire in the right direction. I understood that the connector was going to be replaced. Afterwards, when the robot had yet another verified radio reboot with associated logs suggesting a complete power dropout, I noted the same loose feel to the red wire's terminal. I didn't think to ask whether it was the entire connector and its wiring that had been replaced, or if it was just the housing, or if it was just the terminals and wires. It is possible that the red SB50 shell was reused, and that the underlying fault is that its "spring" isn't holding the terminal hard enough to the matching battery connector. Or it is possible that the terminal itself wasn't changed and has a bad surface that doesn't provide a secure contact.

Being unable to help 4490 correct the repeatedly-appearing problem is definitely something I feel bad about.

[juchong]

Try replacing the buck converter that's connected to your radio and make sure that the converter case is isolated from the rest of your robot. We saw quite a bit of that on the field at Hub City, as well. We verified that all the connections were done correctly, that the battery was properly secured, and replaced the radio (just in case).

[BBray_T1296]

Originally Posted by Canon reeves

During our first qualification match we ran great until the last 15 seconds we lost power to the C-Rio, and all Comm. and Code. During the next match our battery fell out. And in every other qualification match we would run, then our C-Rio would reboot.

The FMS cannot cause your battery to fall out.
There is a good chance your wiring was pulled unnoticeably loose and caused a slight loss of power during some particular hits.

Possible causes: loose lugs somewhere between battery and PD board, loose/improper crimps/WAGO connections

Just because it looks correct, doesn't mean it is.

[Greg McKaskle]

The FMS doesn't directly communicates with the robot.

The FMS will tell your DS computer when to be disabled, auto, and tele modes, but it is your DS computer that sends all packets to the robot via the wifi radio on the field. The Practice match sends the same info. The code on the cRIO runs the same whether on field or in pits.

If your battery is able to fall out once, it is likely that it was loose at other times. When not held firmly in place, the battery is a 12 lb. sledge hammer banging into your other components. It easily knocks wires out of Wago terminals, knocks ribbon cables off, knocks wires and terminals against the frame, etc. As Alan said, its direction of travel will also depend quite a lot on the type of hit.

Greg McKaskle

[Canon reeves]

That makes a lot more sense, so what we really need to check next is our mainbreaker crimps, and connections, and our converter for the d-link, it is also very possible for the battery to mess up connections because we had our battery board fall down on at an angle and there was no visual damage. We will also try a good shaking of the power wires to see if we can get the problem to re occur so we can pinpoint it. Thanks for all the feedback guys, and a special help to Alan for all of his hard work and helping us find the problems!

[Richard Wallace]

Did each of the SB50 terminals "snap click" into its proper place within the housing; i.e., toes over the end of the diving board? Sometimes if the terminal is bent slightly, the "snap click" does not happen and the terminal can back out a little, just enough to make intermittent contact. A good whack could open such a contact, and another good whack later could close it again. This can be hard to find until the terminal backs out far enough to cause the connection to open statically.

[Al Skierkiewicz]

Guys,
There are two power supplies on the PD, one for the cRio and the other for the radio. while similar in design these are separate supplies. They will stop operating when the battery voltage input to the PD falls below 4.5 volts. Since they are separate supplies they may stop at different times. Loose battery connections, poor crimps, loose hardware on the battery and a bad SB50 connector can all contribute to losses that will shut down these supplies. However, even these supplies cannot stay up with no power to the robot. So, as inspector and electrical guy, I work this list...
1. Are the terminals on the battery tight and non-moving? Typical of carrying the battery by the wiring. Cannot be repaired, battery is likely candidate for recycling.
2. Do the wire terminals rotate on the battery terminals? Typical for these battery terminal types. The simple addition of a star washer between the terminals will prevent this.
3. Is the hardware tight on the PD? Tighten using metric nut driver.
4. Is the hardware tight on the main circuit breaker? Tighten using english nut driver.
5. Is the wire secure in the terminals used? Tighten and strip to correct length if needed. SLA connector in particular have an optimum strip length that is needed for secure electrical connection.
6. Is everything correctly insulated? Electrical tape, not duct tape is your electrical friend.

[Canon reeves]

So we have been running tests with a multimeter and it doesn't seem to be a PD board problem, and I replaced all of the power connections to the CRIO and when I got ready to drive I connected, enabled, then drove forward slightly and rebooted. The CRIO didn't reboot until enabled and drove? We also had the whole power go out, and I am not certain what caused that, it was most likely the battery because the Anderson was at a weird angle on it. Aside from that, could the problem be in the Ethernet cord from the D-link to the CRIO? Because we did tether when we got the robot back and it drove fine, so could it possibly be a bad Ethernet cord causing it? We went ahead and replaced it but could it have been the problem?