View Full Version : Spontaneous motor skipping?
Damaku250
29-03-2010, 17:15
We appear to be having an odd problem with our drive motors.
Following an odd voltage drop resulting in the loss of control of our bot in the quarter-finals at WAT, we had discovered that one CIM and its accompanying Jag were noticeably hotter than the others.
After removing the electronics and offending gearbox, getting home and testing them on benchtop, we've noticed two things:
1) after removing our CIMs from the gearbox the one that was overheating is also making an odd clicking and is noticeably louder than the other. [Resolved]
2) that our CIMs "jump" and will randomly stop for a random duration between 1/4 and 1/2 a second. When this happens, all the motors jump simultaneously, and the lights on all the Jaguars flash yellow/amber for a brief period while the motors stall.
We HAVE tried changing the DSC and 9403 cRIO modules, to no avail.
Al Skierkiewicz
29-03-2010, 17:23
Craig,
Sounds like you have a shorted winding in the CIM motor. No way to fix that, just replace.
Damaku250
29-03-2010, 17:24
Craig,
Sounds like you have a shorted winding in the CIM motor. No way to fix that, just replace.
duly noted, but would this cause the second part of the problem?
Al Skierkiewicz
29-03-2010, 17:26
Yes, you are drawing down the 12 volt supply which draws down the DSC power that in turn plays havoc with the PWM outputs.
Damaku250
29-03-2010, 17:28
Yes, you are drawing down the 12 volt supply which draws down the DSC power that in turn plays havoc with the PWM outputs.
We have now removed the borked motor and are testing with the good one. We are still experiencing "skipping". What could be causing this?
Al Skierkiewicz
29-03-2010, 17:30
Craig,
Is the robot signal light also blinking out at the same time?
Damaku250
29-03-2010, 17:34
Craig,
Is the robot signal light also blinking out at the same time?
No, the RSL is staying on and we do not appear to be experiencing power interruptions on any other components on the board.
Al Skierkiewicz
29-03-2010, 17:46
You are using the same Jaguar? Suspect that as well.
Are any of the lights on the PD winking out at the same time?
Damaku250
29-03-2010, 17:49
You are using the same Jaguar? Suspect that as well.
Are any of the lights on the PD winking out at the same time?
We have tried the motor with all 4 different jaguars mounted on our board. All jaguars experience an interruption at the same time.
The lights on the PDB are staying solid.
We have tried 5 different versions of code, all of which are not affecting the problem.
Al Skierkiewicz
29-03-2010, 17:54
Craig,
Is this a new problem? Have you had this all season or just recently? It is possible that you have the dreaded watchdog issue. Someone posted last night that there is an issue with Classmate power settings as well. I believe the Classmate issue occurs at 15-20 second intervals while the watchdog is something longer.
eugenebrooks
29-03-2010, 17:55
Watch the little screen on the lower left of the
driver station display. Do you see a momentary
"watchdog not fed" message correlated with
the motor glitches? If you see it, you have the
dreaded watchdog issue Al mentions.
Eugene
It is definitely not a 15-20 second cycle, we've had the CIM running for up to 1:30 before it skips before. It may be a Watchdog issue - I'll go run another test and keep an eye out for the Watchdog issue.
This is, as far as we know, a completely new issue however. We hadn't noticed this motor skipping problem for all of build season or all of the practice and seeding matches at Waterloo.
It is definitely not a 15-20 second cycle, we've had the CIM running for up to 1:30 before it skips before. It may be a Watchdog issue - I'll go run another test and keep an eye out for the Watchdog issue.
This is, as far as we know, a completely new issue however. We hadn't noticed this motor skipping problem for all of build season or all of the practice and seeding matches at Waterloo.
Just ran the test. Watchdog is indeed cutting out...
TubaMorg
29-03-2010, 18:29
Yes we had exactly the same symptoms early in our build season. At first it was manifested in a compressor that wouldn't go up to 120psi, then we noticed the intermittent Watchdog Not Fed error. It also caused the motors to hitch and jerk when they should just be running. We widened the setpoint for the watch dog to .4 seconds and made sure every bracketed piece of code feeds the dog and haven't had an issue since.
Widened the setpoint? I'm not sure what you mean by this. A delay before first feeding it?
360skier
29-03-2010, 19:37
The watchdog works like a timer. If it doesn't receive a feed command from your code every so often, it shuts down your output. This is to prevent you from setting your motors to full forward, and then lagging the classmate so you can't shut them off.
If you look at the watchdog methods (or blocks if you're using labview), there should be one something like SetPeriod which takes a float. This will allow you to specify the watchdog timeout, in seconds I believe. Making this slightly longer will hopefully stop causing the intermittent motor shutdown you've been seeing.
The watchdog works like a timer. If it doesn't receive a feed command from your code every so often, it shuts down your output. This is to prevent you from setting your motors to full forward, and then lagging the classmate so you can't shut them off.
If you look at the watchdog methods (or blocks if you're using labview), there should be one something like SetPeriod which takes a float. This will allow you to specify the watchdog timeout, in seconds I believe. Making this slightly longer will hopefully stop causing the intermittent motor shutdown you've been seeing.
NO! :eek:
I'm not an expert but from what the NI-Guy(tm) at our regional (Go Andrew!) told me, you are not supposed to change the watchdog timeout from it's default (0.5sec?).
That would be like putting a bigger bucket under a leak, instead of fixing the leak.
what you need to do is have a good look at your code, and find out what routine is bogging up the processor. I'm sure many people at the programming forum would have the capability to help you look over the code if you can't find the problem...
Good luck!
-Leav
TubaMorg
29-03-2010, 19:50
The watchdog works like a timer. If it doesn't receive a feed command from your code every so often, it shuts down your output. This is to prevent you from setting your motors to full forward, and then lagging the classmate so you can't shut them off.
If you look at the watchdog methods (or blocks if you're using labview), there should be one something like SetPeriod which takes a float. This will allow you to specify the watchdog timeout, in seconds I believe. Making this slightly longer will hopefully stop causing the intermittent motor shutdown you've been seeing.
Yes this is what I meant by widen the setpoint. I apologize for the vague terminology! Avoid the temptation to set it too long, it really is an important safety feature. As leav pointed out, more than 0.5 seconds is too long. Our Java had a default of 0.2 seconds that we changed to 0.4. If your code is getting stuck somewhere for longer than this, it definitely needs some optimization work
I was able to resolve the issue (it seems) by changing the power management settings on the Classmate as well as going into its BIOS and turning off SpeedStep and C-State or something like that (they just seem to reduce CPU power to lengthen battery life). We have a UPS to power the Classmate so battery life still shouldn't be an issue :p. And if it comes down to it, we have an inverter that works fine as well. I've gone over my code several times and had the other programmers on the team - although I'm the only one who wrote the robot code, we have others who are capable but were too busy with building the rest of the robot - look over it for errors and possible resource reduction possibilities and they couldn't come up with anything either.
Greg McKaskle
29-03-2010, 20:18
Do you know what the power settings were? Also, did you try only changing the power settings, or asked another way, was it necessary to change the BIOS?
Greg McKaskle
The power settings were already set pretty well by whoever did it before - while plugged it, everything was set to "Never." They had the battery settings as turning off the monitor and HDD's after 5 minutes of inactivity and everything else to "Never." I changed monitor shut down to 10 minutes and HDD to 15. So nothing really with the power management. I think it was the BIOS settings really that fixed the issue.
TubaMorg
29-03-2010, 20:48
So changing the power settings solved the Watchdog issues? Or is the Watchdog still cutting out?
eugenebrooks
29-03-2010, 21:32
Some instructions on how to get into the bios and change these
settings would be appreciated. Intermittent watchdog errors
are driving many teams nuts.
Edit: See http://www.chiefdelphi.com/forums/showthread.php?p=945073
Eugene
I was able to resolve the issue (it seems) by changing the power management settings on the Classmate as well as going into its BIOS and turning off SpeedStep and C-State or something like that (they just seem to reduce CPU power to lengthen battery life). We have a UPS to power the Classmate so battery life still shouldn't be an issue :p. And if it comes down to it, we have an inverter that works fine as well. I've gone over my code several times and had the other programmers on the team - although I'm the only one who wrote the robot code, we have others who are capable but were too busy with building the rest of the robot - look over it for errors and possible resource reduction possibilities and they couldn't come up with anything either.
Yes, changing the power management and SpeedStep/C-State settings (I did both at once before checking for Watchdog again) seems to have solved the issue. Like Eugene's linked thread says, reboot the Classmate and tap delete until you get into BIOS setup. I think it was under "CPU Settings" or something like that. Just disable the SpeedStep and first C-State option, then hit F10 to save and exit. Worked for us, hope it's so simple for you too :).
Greg McKaskle
30-03-2010, 08:54
I'm asking because I've been investigating the WD errors over the last four of five days. The problem is that even with Max Battery power settings, I can't get repeatable failures. I can see timing glitch differences, but not big enough to cause WD errors. I can repeatably cause other problems by setting power settings to that extreme -- this will be detailed soon.
If you have the time, can you stepwise back out the changes until the problem comes back? Clearly asking teams to make numerous BIOS changes has its risks, so I'm looking for the minimal changes, and ideally, the series of steps one can take to improve the timings. The tradeoff here is battery life.
Let me know your observations. Thanks.
Greg McKaskle
Yep, next chance I have I'll go back and set the BIOS to default and see if the WD errors come back.
Some instructions on how to get into the bios and change these
settings would be appreciated.
While the Classmate is booting up press the DEL key to get into the BIOS screen. When in the BIOS screen click on ADVANCED then CPU CONFIGURATION. I turned both the SpeedStep option and the C-State option to disabled (both were enabled).
I have not tried any variations of these two options (example, one enabled and the other disabled). From the websites I've read and the symptoms I've seen in my testing, my suspicion is the SpeedStep is the one that needs to be disabled, I just took the shotgun approach and disabled the C-State option as well as our competition starts this week. Its working for us so far, more testing with the bits of our bot that we kept tonight.
Merle Yoder
The GRUNTS Team #3146
vBulletin® v3.6.4, Copyright ©2000-2017, Jelsoft Enterprises Ltd.