![]() |
[HELP] Digital Sidecar troubles
We've managed to isolate our problem to our sidecar. After that, our team is completely lost as to what the problem is. We've switched through some sidecars but none seem to work, so it may be something wrong with the set up. Nothing electrically has changed between when we last competed, CTTD 11/3/12, and now. Our sidecar is set up, but suddenly the 5v and battery light are spazzing out. As well, we only sporadically get power to the RSL. Any suggestions? The lights are fine when we're disabled, but once we enable then all the spastic problems start happening. We've gone through and checked for shorts, but didn't find any.
EDIT: Also, we're getting a high pitched whine from the PDB...any ideas on that, and if it's an issue? We do have a replacement PDB but we'd rather not have to replace it if we don't have to. |
Re: [HELP] Digital Sidecar troubles
Every sidecar problem we ever had was caused buy the cable and not the sidecar.
Check the cable. |
Re: [HELP] Digital Sidecar troubles
Sounds like you have power problems, check that all the fuses are good and if the wago connection is not loose (the cords tend to slip out over time). Also, I'd check to see if the cable from the cRIO to the Sidecar is fine, since the robot's game state seems to have an influence, although I don't think that's the problem since you say it worked in CTTD and hasn't changed since.
|
Re: [HELP] Digital Sidecar troubles
Quote:
Quote:
|
Re: [HELP] Digital Sidecar troubles
We've also found the ribbon cables to be finicky.
|
Re: [HELP] Digital Sidecar troubles
We've had this problem quite a bit. For us it's been caused by a digital I/O or PWM connector being shorted out. You should try running the robot with nothing connected to the sidecar, then add the connections until you find the bad one. We've found that it's quite easy for the DIO cables to get damaged and short together and cause the 5V supply on the sidecar to drop out. We've also seen a very similar problem to this where all of the outputs on the sidecar drop out. This is caused by the 9403 digital module losing power. If your cRIO power connector is loose, it can cause the digital module(the part that goes in the cRIO, not the sidecar) to brown out when the rest of the cRIO doesn't.
|
Re: [HELP] Digital Sidecar troubles
A few thoughts that might help diagnose the issue:
- unplug the DSC and hold a multimeter up to the plug. Read the voltage when the robot is both enabled and disabled. Please make sure you do so safely! This should mimic the battery voltage, so it won't hold steady at 12V. - Even if you don't have a short, you could have an open circuit. I saw a team at an off season competition a month or two ago that used 22 gauge from the PDB to the DSC, and managed to burn it out completely - broken wire, burned/melted insulation... it was ugly. With a broken wire, you could see intermittent operation. - Remove all the breakers from the PDB except those required for the DSC and the analog breakout board. Do you still hear that whine? Do you still see the incorrect behavior on the DSC? If not go through replacing the breakers one at a time - at some point you'll start encountering your issues again, but this may give you a different place to look. - Are there any red LED's on beneath any of ports on the DSC? Those are there to tell you if there is an issue (a short) with that port! |
Re: [HELP] Digital Sidecar troubles
Quote:
Quote:
Quote:
|
Re: [HELP] Digital Sidecar troubles
If you try the suggestions here, still can't find the problem, and you'd like some on-site help, PM me. My schedule isn't too horrible this month, so we can probably find a mutually agreeable time for me to visit your shop and take a look.
|
Re: [HELP] Digital Sidecar troubles
I remember seeing Joe Hershberger (jhersh) diagnosing a problem at the Alamo regional once that mystified all present. On your list of things to check please add the following:
The pins on the DSub connector on the cRIO may get bent if the module is not inserted correctly. It is unusual, but I've seen it happen twice now. Depending on which pin is bent, the module may power up and report fine, but may not function properly. |
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
Quote:
Now, for the whine. Is it high pitched or low? It is either the sound made by the constant tripping of a circuit breaker or an audible harmonic emitted by one of the switch-mode power supplies in the PDB (5, 12, or 24V). Also note that the sidecar and the analog breakout also have switch-mode supplies in them too, so be sure that the noise isn't coming from a sidecar next to the PDB and you're just assuming it is the PDB. The sound of a circuit breaker constantly tripping will (in my experience) makes a lower pitched noise. The sound of the switch-mode supplies will be higher pitched and may change frequencies based on the current being drawn. This may be caused by the most popular robot construction sin -- metal debris in the electronics. To isolate whether it is the switch-mode supplies rather than a circuit breaker, you can remove all circuit breakers to see if it is the PDB, and then start adding one back at a time as others have suggested. Since the failure didn't go away after changing sidecars, I doubt that the sidecar is the issue. And I doubt that it is a bent pin since I can't think how the loss of connectivity would generate a whine. |
Re: [HELP] Digital Sidecar troubles
I know the sound of a constantly tripping circuit breaker. It isn't that noise. It's a high pitched squeal (Or whine as my girlfriend called it). Our sidecar is fairly remote from our PDB (a good 18"-24"), so I don't think it's the sidecar.
As far as clarification as to why we believe it's somewhere along the line of the sidecar, or perhaps the DB37 cable, would be that it we are still able to enable the robot, get spotty control of the robot, and then we die out. We tried switching the port the sidecar gets power from and still have the same problem. |
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
Does the power board continue to whine when unloaded? Meaning if you pull all the connections and energize the board do you still hear it make noise? This would mean removing *all* the breakers, disconnecting the radio, cRio, and camera (basically having an empty board). After it is disconnected, power the PD board up from the battery and see if you still have the whine.
|
Re: [HELP] Digital Sidecar troubles
I wouldn't worry too much about the whine.
It's probably just the boost converters. |
Re: [HELP] Digital Sidecar troubles
Haven't tried unloading the PDB nor have I tried removing the fuses yet. I'll have to try tomorrow afternoon (~Noonish if people would like to check in.) We have spare PDBs, I would just rather not have to go through and rewire the entire PDB . If it comes down to that being the problem then I might just have to do that.
|
Re: [HELP] Digital Sidecar troubles
Scott,
Whine in these power supplies is normal although often very low in level. I have to ask what the LEDs on the DSC are doing. Are they all ON (qty 3) at the same brilliance? Does the RSL LED (the one on the DSC board next to the white RSL connector) flash when the robot is enabled? Is the 6 volt LED changing? This is usually metallic debris in the servo output connectors. Do you have any of the 6 volt jumpers installed on speed controller outputs? Do you have any relays connected? What do their LEDs look like? If all of these appear normal, try removing all inputs and outputs from the DSC and then try installing them one at a time if the DSC appears normal with everything removed. |
Re: [HELP] Digital Sidecar troubles
Quote:
Is that a correct understanding, or did you mean something else? |
Re: [HELP] Digital Sidecar troubles
Quote:
Quote:
Quote:
Quote:
Thank you, all of you, for your help. It is much appreciated. |
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
I'm beside the robot right now. All three, battery 6v and 5v LED, are strong when disabled, but when we enable then the 5v LED goes off or really really dim. RSL appears to have stopped responding.
|
Re: [HELP] Digital Sidecar troubles
Have you tried just removing the RSL completely, or replacing it? I remember at some event some team this year had damaged/defective RSLs that caused strange things to happen to their digital sidecar.
|
Re: [HELP] Digital Sidecar troubles
When disabled, the battery light flickers now.
|
Re: [HELP] Digital Sidecar troubles
The description still sounds like a short that only happens when Enabled.
The Relays and PWM signal lines are the only things that are enabled when you enable the robot, and disabled when the robot is Disabled. The DIOs & RSL keep doing what they are doing in either state, so no change there. The DSL itself remains a possibility, since some internal circuits could be shorting too. Have you turned it upside down over a clean cheet of paper and given it a shaking to see if metal dust falls out? I'd still look at pulling all the PWM and Relay connections. P.S. A flickering battery light in Disabled is going to mean a bad power connection. Use a meter on the 12v power input to the DSL to insure you've got battery voltage at that point when Enabled. |
Re: [HELP] Digital Sidecar troubles
I tried running with a different PWM removed each time. Problem not solved. We don't have any relay connections so that can't be it. The DSC still gets a solid 12.something voltage between being enabled and disabled.
EDIT: Also, every few (probably 3 or 4) times we enable, the robot will "jump." Ie. motors will pulse for a moment. |
Re: [HELP] Digital Sidecar troubles
You have a second DSL you can hookup just the power and the 37-pin cable (no other connectors) to see if it exhibits the same faults.
Do the upsidedown shake test on the DSL first. P.S. I can get servos to jump like that sometimes on Enable. I don't think it's symptomatic of anything in particular. |
Re: [HELP] Digital Sidecar troubles
Hooked up a second DSL with just power and DB37, LEDs strong. No fault so far. 30 seconds full operation. Or, at least, as much operation we could get.
|
Re: [HELP] Digital Sidecar troubles
When you're happy that the DSL alone is not going to fault, keep it Enabled and move your PWN connections over a single one at a time.
wiggle each one to see if you can get it to short of cause a fault. P.S. How many PWM connections do you have? More than 4? |
Re: [HELP] Digital Sidecar troubles
Alright I'll try that tonight when I'm by the robot again. When you say move them one at a time, do you mean just plug one at a time in or move one, check it works, add another?
|
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
You may have to do it both ways, but I'd start with having only a single connect on at a time.
One alone connected to the DSL will show a short on individual lines. Good if there is more than one connector at fault. Don't assume any of the connectors are good if you find a bad one. Test each and every one. If the fault is related to the total power draw (why I asked how many), then you'll have to repeat and work your way up to having all of them connected. Or finding out how many connections until it causes a fault. Just be systematic about it. |
Re: [HELP] Digital Sidecar troubles
Check that it works after each individual change.
The idea here is to further isolate the problem. If you plug in one PWM cable, and it still works fine, you know that one cable (and the attached speed controller and motor) aren't causing your issue. However, when you plug one in and start having problems, then you've narrowed it down to that specific branch. (This is assuming it's not a broader problem related to the number of controllers being used) Once you've narrowed it down to a specific PWM branch off the Digital Sidecar, you can then figure out the problem pretty quickly. For example, if the PWM is controlling a speed controller, disconnect the motor outputs from the speed controller and see if you still have the problem - if you don't, the issue is likely with the motor or the wiring between the motor and the speed controller. If you do, still have the problem, try replacing the PWM cable. If that fixes it, you know there was an issue with the cable. If it still isn't working, try replacing the speed controller. Generally speaking, when debugging issues like this, you aren't trying to find the problem directly - you're trying to eliminate as many possible locations for the problem as you can. This is something I do all that time at work. Narrow down the scope of what you're looking at, and you'll find the problem easier. |
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
At this moment, we have a new PDB on the robot. All we have plugged into the PDB is cRio, Sidecar, and one Jaguar. Attached to the Jaguar is one CIM, going through a CIMple box, to a 6in KOP wheel. The Sidecar is attached to the cRio using a DB37 connector. The lone Jag is in port 2. The sidecar, when both disabled and enabled, shows all three lights strong. Yet when we enable it the Jag itself doesn't enable. (It's not the Jag that's the problem, none of the motor controllers [Jags or Victors] would enable.)
|
Re: [HELP] Digital Sidecar troubles
Of course, make sure your software really is talking to port 2.
OK, with only one PWM, power is solid. Good Are you seeing PWM to the motor controller? Use an AC voltmeter on the white wire, robot enabled. Verify a good ground path, about 0 ohms from the black PWM wire at the motor controller to battery negative (all power off) Let us know. The next step is to check the data signal coming out of the digital bumper on the cRio, robot enabled. Got a spare digital module? |
Re: [HELP] Digital Sidecar troubles
Quote:
Sidecar lights went out again. When we're plugged into the cRio via DB37, we have no 6v light, and 5v light goes out as soon as we enable. This problem showed up recently. For a half hour tonight we had all three lights strong. When you said check PWM, can you elaborate on how to check? EDIT: If unplugged from the cRio and we still don't have all three lights, does this mean the sidecar is fried? If yes then does anybody know what typically fries a sidecar? This new sidecar has been hooked up to nothing but our simplistic setup, and must have been fried sometime in the last half hour. |
Re: [HELP] Digital Sidecar troubles
Cal,
The LED conditions you are describing are typical for no power to the DSC. The DB37 connector can back feed enough current into the DSC to light the LEDs with no current demand. When you enable and start drawing current in the DSC, the LEDs go out or significantly dim because there simply is not enough current available through the DB37. Please double check the polarity of the power supply wiring to the DSC and inspect carefully the power connector, pull on the wires. There is a method of reverse polarity protection in the DSC but I have seen it fail with abuse in the past. You said this DSC worked in another robot, move it back to that robot to check it. |
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
I can elaborate, but read Al's response above first, then help us by answering Alan's question.
To check a PWM signal, put an AC voltmeter between the white signal wire and the black ground wire. It should read about 5 volts AC, or thereabouts, while the PWM is actively being commanded (used). This will vary slightly from full reverse to full forward, but only a tiny bit. |
Re: [HELP] Digital Sidecar troubles
Quote:
Quote:
Quote:
Thank you for your help and hopefully we can get this resolved soon. |
Re: [HELP] Digital Sidecar troubles
One more thing that I don't think anybody has mentioned: you're using a full-charged battery, right? Weird things happen when battery voltage drops too far, including PDB whine.
|
Re: [HELP] Digital Sidecar troubles
Yupp, full batteries. We've switched batteries twice now.
|
Re: [HELP] Digital Sidecar troubles
Cal,
Have you moved this DSC back to the other robot? Does it still work over there? If the wiring is all the same, there is a remote chance you have a defective WAGO connector at the DSC. It may have a bent/broken pin inside the connector, or the pin in the circuit board is broken or improperly soldered. You won't be able to to check the power supply voltage on the DSC board without taking the cover off but that would be my next step if I was working on this problem. For a long shot, I would open the case anyway for inspection. I have had on rare occasions, found loose material that floats around inside, causing problems in one orientation but not in others. Or a crack in the board, allows it to work in one mounting but not in others. |
Re: [HELP] Digital Sidecar troubles
Quote:
If you're using an inexpensive meter (like mine) that doesn't do true rms, it will read 0 volts on the AC setting when trying to measure the PWM signal if you have the test leads reversed. That's because in AC mode it rectifies and filters the signal and assumes it's zero-offset sinusoidal in order to get an AC reading. If the test leads are reversed, the meter's diode will block the PWM signal voltage. If the test lead polarity is correct, it will give a reading. As you vary the command from full forward (2.0ms pulse width) to full reverse (1.0ms pulse width), the voltage at full reverse will be approx 1/2 of the voltage at full forward (because the duty cycle at full reverse is 50% of the duty cycle at full forward: 1.0ms/2.0ms=1/2). Also, you will get different voltage readings depending on the PWM period you are using (e.g. Jag vs Vic driver). |
Re: [HELP] Digital Sidecar troubles
We have really nice meters; advantages of our head coach being a s
Digital electronics teacher :D I'll have to do the checks at our meeting tonight |
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
3 Attachment(s)
So I took apart two of the sidecars that stopped working. The insides were not pretty. Anyone have any idea what could be causing them to burn out so fast? We've checked over everything short of splitting the wires and inspecting individual strands, but came up short. Apart from the fact that our robot is frying sidecars like eggs, she's in perfect health.
|
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
Double chassis fault? The think I can't understand about this happening is we checked over our wiring twice, and it was all in order. Plus, the first sidecar that was on there was literally the first sidecar on the robot. It survived 3 competitions and an off season and then just decided to burn out and left an apparent curse on the rest of the sidecars plugged in. Can't really plug a Wago in backwards so that's out.
|
Re: [HELP] Digital Sidecar troubles
It's likely a short somewhere past the sidecar. Remember, shorts don't *just* happen when you first build a robot... any repairs or changes made to the robot could result in someone accidentally drilling through a wire without realizing it! It could be anything plugged into the sidecar... You have to diagnose everything plugged into it to see what isn't working properly.
Also, while a wago connector can't be plugged in backwards, it can be wired backwards... tripple check it. Then check it again :p |
Re: [HELP] Digital Sidecar troubles
1 Attachment(s)
The part that fried is used to connect the ground signals within the DSC.
Attachment 13255 We had this happen to us and was caused by either +5V or +12V connecting to one of the ground pins [PWM or digital] on the DSC. We were troubleshooting and had 12V wire touch the PWM ground connection. Our board only had the top of the IC blow off, which looked like the blue PCB you showed. Did you notice any smoke or electrical burnt smell when things stopped working. -pete |
Re: [HELP] Digital Sidecar troubles
Cal,
Eric has pointed out that Q1 is the reverse polarity protection device. So if you are sure that someone hasn't rewired the WAGO connector, I would start looking for a PWM cable that is laying across or next to a power lead. In this kind of fault, the FET is not going to be the only thing damaged. If you feel along PWM cables, one will likely have a textured feel indicating it has gotten hot during the fault. Often these occur where multiple wires run together across a rotating joint or inside a moving arm. As Jon pointed out, don't rule out the drilled fault either. I have seen stranger things, so it is possible that the fault has existed for a long time and is just now raising it's ugly head. The best way to trouble shoot at this time is an ohm meter measuring continuity between parts on the robot. I would start by checking each of the PWM cables first to chassis and then to battery terminals on the PD with the power off and all PWM cables disconnected from the DSC. All should read very high resistance, greater than 100K. If your probes are sharp enough, you can touch the contacts in the PWM block through the holes in the side. Alternately, a resistor lead wrapped around the probe and inserted into the end of the PWM connector also works. |
Re: [HELP] Digital Sidecar troubles
Well, guess I know what I'm doing later today. I've got a date with an ohm meter and our robot. Should be fun.
|
Re: [HELP] Digital Sidecar troubles
Checked everything over and everything is isolated. Battery lugs to frame is open, each strand of the pwms to the chassis are open. Wago is wired correctly, also not shorting internally. Visually inspected the PWMs and they're not broken nor "bubbled."
|
Re: [HELP] Digital Sidecar troubles
Then it's probably an intermittent short that only occurs when a mechanism is in or passes through a particular position. Those are fun to trace down.
|
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
You've got two faults: 1) a ground PWM connected to the DSL, 2) a 12v power feed that's somehow intermittently finding a path to the PWM ground line (this can be direct cable-to-cable or through some intermediary such as the robot frame or even yet another cable).
You've already narrowed down the list of possible culprits to one of the ground lines on the PWMs or DIOs plugged into the DSL. It's going to be a short that shows up only intermittently on one of the limited number of ground lines connected back to the DSL via one of the PWM cable ground lines. The origin of the 12v can be anything on the robot connected to a live 12v power feed, so everything carrying 12v from the PDB to the motors are suspects. Some motor types can ground through their bolts to the frame, and may only conduct power when a certain winding with a worn spot is energized which means rotating the motor slowly through it's rotary motion. And remember the cRIO and camera chassis are both hot and need to be electrically isolated. If your memory is good and you were systematic in your testing, then try to narrow your investigation down to the parts of the robot you were exercising just before you noticed the second DSL power LEDs went south. You don't want to burn out another DSL, so testing for the shorting ground wire involves putting your multi-meter on each PWM cable ground line & the 12v power lug on the PDB one-at-a-time and moving the robot mechanisms by hand if possible. It also involves shaking and moving the wires to stress them a little. Don't be gentle with your wire runs, put them under the same movements, vibrating, rubbing together that they undergo normally while the robot is racing around the field, bouncing off other robots and walls. Searching for the shorting 12v line involves the same sort of continuity test for each of the 12v power pathways. A little tricky since you won't be able to power the motors, and continuity of the power pathway will be broken at the motor controllers. |
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
Cal,
Measure from each PWM pin to PD and then check the Crio frame to PD and robot frame. Then check the output of each speed controller. We have had CIMs short to case after they have been stressed in continual demo mode. You know that something on the robot is taking out the DSC, you just have to find it. |
Re: [HELP] Digital Sidecar troubles
Quote:
Quote:
Tracking down errors like this is much harder than avoiding them in the first place. Use this as a lesson in enforcing clean wiring. |
Re: [HELP] Digital Sidecar troubles
I remember in the first year or so, our cRIO was killing the D-I/O modules. We went through a few. Check the module in another port if possible, or see if another area team could lend you a D-I/O module to check with.
|
Re: [HELP] Digital Sidecar troubles
Quote:
|
Re: [HELP] Digital Sidecar troubles
Quote:
We were having some problems with a motor at CowTown. I'll have to add that to my check list Monday. There wouldn't happen to be any way to fix a sidecar once it's burned, is there? |
Re: [HELP] Digital Sidecar troubles
Cal,
It depends...At least one of the photos above shows significant board charring. I would say that one is not repairable. If you have experience or tools to remove surface mount components, then yes you can replace the damaged FET and other parts. This is not for the faint of heart. Since you have gone through so much already without finding anything significant, there is an off chance that one of the controllers has shorted power to the PWM. This is very unlikely but still possible. Another possibility, though highly unlikely, is a damaged Spike. Suspect everything on the robot, I have seen compressors and all types of motors short to case. If you had trouble with a motor earlier, than start there first. Here's a thought, remove only the power from the DSC, and power the robot. See if you measure any voltage at the power input to the DSC. If you do, start removing PWM cables from the DSC and see if the voltage goes away. You are not trying to power LEDs from a digital output are you? Are you using digital inputs/outputs for other purposes or devices? |
Re: [HELP] Digital Sidecar troubles
Quote:
Quote:
[/quote] Here's a thought, remove only the power from the DSC, and power the robot. See if you measure any voltage at the power input to the DSC. If you do, start removing PWM cables from the DSC and see if the voltage goes away. You are not trying to power LEDs from a digital output are you? Are you using digital inputs/outputs for other purposes or devices?[/quote] We don't have any DIOs on the robot. I'll take some pictures of the entire setup on Monday, but all we had were: Drive Motors: 2 CIMs -> 1 Black Jaguar and 1 Victor-884 -> PWM-Y -> PWM port 1 2 CIMs ->1 Black Jaguar and 1 Victor-884 -> PWM-Y -> PWM port 2 Ball Acquisition: 2 Window Motors -> 2 Victor-884s -> PWM-Y -> PWM port 4 Shooter: 1 Banebot 550 (20:1) -> 1 Victor-884 -> PWM (straight) -> PWM port 6 2 Banebot 550 (5:1) -> 2 Victor-884 -> PWM-Y -> PWM port 8 The Sidecar, as stated, is hooked up to an 8 slot cRio using a DB37 cable. Checked all the PWM cables today, they're all continuous, no breakage, and no shorting across strands. Not sure how I would check a motor controller for a short. |
| All times are GMT -5. The time now is 05:51. |
Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi