powr dist. board/cRIO puzzle - please advise!

[Steve Compton]

In our second quarterfinal match yesterday, our robot went over the bump and upon hitting the ground on opposite side, the cRIO cut out and we were dead on the field. Back to the pit, tethered up and could not recreate the problem. Out to the field and the first bump to the robot and it went out again, then rebooted after time, the cut out again.

Troubleshooting afterward (haing won the first match, we lost the second two), we found that if we pulled one specific fuse to one of our window motors, the cRIO problem was gone - we could smack the robot into walls, shove it, bounce it up and down, etc and no problem. We put the fuse back in... and simply picking one end of the robot up a couple inches and dropping it caused the failure to occur.

I'd appreciate your good minds on this one, and thanks!

[Jack Jones]

I'll take a guess that either the motor or the wiring to it have an intermitant short to the chassis.

[Evert Timberg]

The internal contacts inside the window motor may have come loose and touch the frame of the motor every time your robot goes over the hump. I would try switching the motor and see if you can recreate the problem. If the problem still occurs then it may be that the wires feeding the motor are touching due to movement.

[RyanN]

This isn't going to help much, but it shouldn't matter if the motor is shorting to the frame. The cRio and the camera should be insulated from the frame.

Check the resistance from both the positive and negative terminals of power distribution board to the frame. The resistance should be really high or at infinity if everything is properly insulated, preferably infinity... but moisture can cause some the resistance to go lower.

This is exactly one of the reasons why everything is supposed to be insulated from the frame.

If you find the resistance to be near zero or at zero, then you definitely have a grounding problem. Start pulling fuses and ground wires until you find which one is causing it (Hint: Try the cRio and camera first).

[Nick Lawrence]

You could also be suffering from metal chip(s) inside your PDB, near the area where that fuse is located. It doesn't hurt to vacuum out your electronics too, and try not to drill over top of them.

-Nick

[Pausert]

If you are using the WET610N wireless bridge, check how your power plug fits into it. It's really loose on all the models I've played with, which is why its relegated to our practice bot. If you don't have the older bridge on hand, try some hot glue.

[Al Skierkiewicz]

Steve,
You need to check everything following the breaker that feeds the window motor. You could have a bad speed controller, intermittent connection at the input or output of the controller or the motor itself. In some cases, a short in any of the input wiring to the frame of the robot can find it's way back to the Crio especially during hard robot frame hits. In some cases this causes a Crio reboot. You know how long that can take. You also may lose handshake with the wireless adapter. Could you see the RSL and can you tell us what the flash rate was during the outage?
The most sensitive wiring that could cause these faults are power wiring to the Crio, loose hardware on a controller, power wiring on the window motor, and power wiring on the DSC. I am going to bet that the two input connections on a speed controller are very close together. These touch with robot movements.

[electron]

we too had the problem of everything shutting off while going over the bump. We determined it to be electrical interference from the wireless bridge or a bad main breaker. either way, after replacing the main breaker, and moving/replacing the wireless bridge, the problem was gone.

[Steve Compton]

Thanks to everyone who has written so far with ideas. Unfortunately, we had to pack the robot into a shipping crate and off to ATL so didn;t have time to do as much as we would have liked. Trying to put together a protocol for when we get there to diagnose and correct as quickly as possible. I will ask the mentor who knows the most about all of this to jump on in to this discussion as time permits, and thanks for the help - keep the ideas coming!

[Jon Stratis]

Steve,
You need to check everything following the breaker that feeds the window motor. You could have a bad speed controller, intermittent connection at the input or output of the controller or the motor itself. In some cases, a short in any of the input wiring to the frame of the robot can find it's way back to the Crio especially during hard robot frame hits. In some cases this causes a Crio reboot. You know how long that can take. You also may lose handshake with the wireless adapter. Could you see the RSL and can you tell us what the flash rate was during the outage?
The most sensitive wiring that could cause these faults are power wiring to the Crio, loose hardware on a controller, power wiring on the window motor, and power wiring on the DSC. I am going to bet that the two input connections on a speed controller are very close together. These touch with robot movements.

I second everything Al says here. You have an intermittent short on that specific branch circuit somewhere. If it shorts to the frame and finds its way back to the cRio, as Al said it could cause a reboot. if your battery is a little low and it shorts, it may drop the voltage of the battery down low enough to cut power to the cRio, even temporarily, and cause a reboot.

If the robot was in the shop and you were able to spend some time with it, i would probably provide a nice list of trouble shooting steps here to help. however, you aren't, and you probably don't want to have to spend a lot of time with this in Atlanta. So i would recommend one simple thing: Replace everything on that branch circuit (wiring, speed controller, motor). You can have it all prepared and ready to go before you even show up, and just swap it all out. If possible, i would also change which ports you're using on the power distribution board, just in case the problem is there. Two years ago we had to do something similar between our first and second competitions. getting everything ready ahead of time made swapping things out a lot faster. Cut all the wire longer than you need - you can just wrap it up inside the robot somewhere.

[Al Skierkiewicz]

Steve,
If you don't find something quickly, please come and get an inspector to help you check it over. We may see something obvious that you don't. Little problems get to be big problems by about 10 AM on Thursday.

[EricVanWyk]

I third Al.

You are shorting through the chassis, through the cRIO's ground return and blowing the internal breaker on the PD.

This means you have two problems:
1) Your cRIO is not isolated from the robot frame.
2) That window motor branch is not properly insulated / isolated.

Fix both!

[Steve Compton]

I'll let Chad Spackman, wjo is really the brains behind this add in when he can, but today he wired up a bench system - battery to distribution panel to victor. Shorting a 15 amp fuse on the power distribution panel immediately shuts down the cRIO,just like we saw on our robot. On this nech system, no possibility of CRIO being in contact with metal frame, no possibility of wiring gluitches with victors, etc. A simple short to the power distribution board and the cRIO folds.

[Alan Anderson]

A simple short to the power distribution board and the cRIO folds.

Since the power being distributed by the board is coming directly from the battery, this should not be a surprise. If battery voltage drops low enough, the 24v boost supply for the cRIO shuts down. A breaker does not respond instantly, so the battery will likely be loaded long enough for the cRIO to lose power and reset after the breaker trips and the voltage comes back.


Your description of the testbed has a couple of problems, though. There should be no "15 amp fuse" anywhere in the system. Shorting a fuse should have no effect unless the fuse is blown and there is another short elsewhere. I'm going to assume you're talking about taking the positive side of a circuit powered through a 20A breaker and connecting it to the negative side, and not worry about it.

[Steve Compton]

Thanks!. This is exactly why I said I will have Chad start talking on this forum... I am sort of shhoing in the dark here and he knows what he's talking about...so beofre I say anything else half-educated, I'll pass it over! Again to ALL of you, thanks for the ongoing conversation.

Steve

[spack]

Thanks to all for suggestions.

What I know through monitoring and through some tests before crating are:
- It's not a comms issue.
- It is an interruption or disturbance of power to the CRio due to a high upstream branch current, or some nasty EM kick. The CRio reboots.
- The internal resistance of the battery is far and away the largest feed resistance in the circuit and the (very slow acting) auto-reset breakers are not meant to protect the CRio from an IR drop due to hard short, so no surprise there. Having said that, I did like the alternate battery approach we used to have to help when a battery may have gotten mismanaged. Also having said that, there is no way to know if there was or is a hard short without the robot in front of us. We can only theorize at this point and do what follows figuring that it probably is such a thing or that the commutator is stuffed.

We will be replacing the motor, victor, and using an alternate breaker port out of hand. And we will arrive with new batteries. Isolation will be checked.

I did some bench testing with a surge load while monitoring the 24v supply with a scope. Frankly I don't like the approach. I'd rather the CRio stayed up while our robot was billowing smoke rings.

Thanks again,
Chad

[Al Skierkiewicz]

The Crio is intended to inhibit output when the battery voltage falls below 5.5 volts. The Crio power supply quits when the battery voltage falls to 4.5 volts. the sense lead for this is the jumper you insert in the analog module.
What did you see when scoping the 24 volt supply?