OCCRA
Go to Post Titanium is almost twice as heavy as aluminum. I can have the bragging aspect of making a bullet resistant robot, but if someone is coming into a competition and shooting my robot, I have more important things to worry about! - Andrew Blair [more]
Home
Go Back   Chief Delphi > Technical > Electrical > CAN
CD-Events   CD-Media   CD-Spy   FRC-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
 
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 01-13-2012, 12:00 PM
dyanoshak dyanoshak is offline
Registered User
AKA: David Yanoshak
FRC #2158 (ausTIN CANs)
Team Role: Mentor
 
Join Date: Dec 2007
Rookie Year: 2007
Location: Austin, TX
Posts: 163
dyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant future
New Jaguar firmware version 101

There is an update to the Jaguar firwmare now available on www.ti.com/jaguar.

It is highly recommended that any teams using CAN update their Jaguar firmware to version 101. The new firmware will improve the Jaguar's ability to recover from errors caused by loose or faulty CAN cables.

Before, these errors could cause the Jaguar to stop responding to commands until it was power-cycled. Now, the Jaguar can recover without a power-cycle.

The detailed explanation:

The updated firmware allows Jaguar to gracefully recover from states where it can no longer communicate on the CAN bus. The Jaguar's CAN controller, by design, will take itself off the CAN bus when it detects too many transmit errors. The idea behind this is the following: If the CAN controller knows it is sending out lots of bad packets, it intelligently removes itself from the bus to prevent the bus from being flooded with bad traffic.

Before the update, Jaguar would stay in this "bus-off" state until the entire controller is reset (after a power-cycle). This would be good if the cause of the error never went away... but since we have found that the biggest cause for bad traffic has been loose or improperly crimped (therefore unreliable) CAN cables, it is better for the overall system that the Jaguar gets back on the bus.

With the new update, the Jaguar will see CAN controller go bus-off, it will report a COMM Fault, it will clear the state, and then it will return to the bus.

Note: Even though the Jaguar will recover and go back on the bus, there is still could be a large amount of bad traffic. Theoretically, if the bad traffic is frequent enough, you will not be communicating effectively with the Jaguar anyway. Be sure to read the Jaguar documentation found on www.ti.com/jaguar for CAN cable techniques or purchase already made cables from some sources listed on these forums.

-David
Reply With Quote
  #2   Spotlight this post!  
Unread 01-14-2012, 11:21 PM
slibert slibert is offline
Software Mentor
AKA: Scott Libert
FRC #2465 (Kauaibots)
Team Role: Mentor
 
Join Date: Oct 2011
Rookie Year: 2005
Location: Kauai, Hawaii
Posts: 63
slibert is on a distinguished road
Re: New Jaguar firmware version 101

Hi David,

We are using black Jaguar's over CAN, in Speed Mode, and have been experiencing cases where some of the motors shut down and have to be rebooted to work again.

After a lot of investigation, we are confident that the CAN wiring is OK, I guess you'll have to trust me on that one....

We updated to Firmware v. 101 in hopes that this behavior would change, but are still experiencing shutdowns requiring reboot.

To reproduce the case, we've narrowed it down to driving full speed (250 rpms) in one direction, and then rapidly shifting to driving the other direction at full speed. We have enabled the automatic ramp mode in hopes that would help, but it does not eliminate the errors.

W/the 2CAN web page, we monitor the current, we see it spike around 43A on one or two of the motors during the direction change.

In the diagnostics page of the driver station, we see getTransaction failures (indicative of CAN bus communication issues) reported right when the motor stops responding. Specifically, the error messages are:

<Code>-44087 ERROR: status == -44087 (0xFFFF53C9) in getTransaction() in C:/windriver/workspace/WPILib/CANJaguar.cpp at line 425

One or more (but not always the same one) motor will stop responding to drive commands. The jaguar light is solid yellow on after they stop responding to drive commands, and the 2CAN web page continues to be able to monitor the bus voltage/current, so it's clear that CAN communication is still working.

Using BDCComm, we view the status of the jaguar that is in the non-responsive state, and Comm errors are reported but no others.

We're discussing working on a software-based ramp mode that hopefully avoids the rapid changes in speed, but we are puzzled at this one thing:

- Why do some of the Jaguars no longer respond to drive commands until we reboot them?

Last edited by slibert : 01-14-2012 at 11:42 PM.
Reply With Quote
  #3   Spotlight this post!  
Unread 01-27-2012, 03:43 PM
gpetilli gpetilli is offline
Registered User
FRC #1559
 
Join Date: Jan 2009
Location: Victor, NY
Posts: 176
gpetilli is just really nicegpetilli is just really nicegpetilli is just really nicegpetilli is just really nicegpetilli is just really nice
Re: New Jaguar firmware version 101

Quote:
Originally Posted by slibert View Post
Hi David,

We are using black Jaguar's over CAN, in Speed Mode, and have been experiencing cases where some of the motors shut down and have to be rebooted to work again.

After a lot of investigation, we are confident that the CAN wiring is OK, I guess you'll have to trust me on that one....

We updated to Firmware v. 101 in hopes that this behavior would change, but are still experiencing shutdowns requiring reboot.

To reproduce the case, we've narrowed it down to driving full speed (250 rpms) in one direction, and then rapidly shifting to driving the other direction at full speed. We have enabled the automatic ramp mode in hopes that would help, but it does not eliminate the errors.

W/the 2CAN web page, we monitor the current, we see it spike around 43A on one or two of the motors during the direction change.

In the diagnostics page of the driver station, we see getTransaction failures (indicative of CAN bus communication issues) reported right when the motor stops responding. Specifically, the error messages are:

<Code>-44087 ERROR: status == -44087 (0xFFFF53C9) in getTransaction() in C:/windriver/workspace/WPILib/CANJaguar.cpp at line 425

One or more (but not always the same one) motor will stop responding to drive commands. The jaguar light is solid yellow on after they stop responding to drive commands, and the 2CAN web page continues to be able to monitor the bus voltage/current, so it's clear that CAN communication is still working.

Using BDCComm, we view the status of the jaguar that is in the non-responsive state, and Comm errors are reported but no others.

We're discussing working on a software-based ramp mode that hopefully avoids the rapid changes in speed, but we are puzzled at this one thing:

- Why do some of the Jaguars no longer respond to drive commands until we reboot them?
We had similar problems last year trying to run speed mode. We saw instantaneous currents reported by the jaguar of over 90A!! The fact that you see the VBUS voltage drop to 7V at only 43A indicates that you definatly have a supply problem. The 2012 KOP gives us 10gauge wire instead of 12gauge in previous year - probably for this reason. Last year we added periodic polling of the jaguars to verify they were in speed mode and re-initiallized them if the were in vbus mode (did not know at the time that they logged faults).

You said in your post that you enabled ramp mode. I too was hoping this would improve things this year but have not tried it yet. My read of the release notes has me thinking that ramp mode only applies to voltage control modes (PWM, percent VBUS, VCMP). Did you do this with software commands or with the limit switch jumpers? Can you confirm that it had any impact?

General question for Jaguar experts - I remember seeing a post last year and can not find it now that said the Jaguars would re-enumerate on reboot and you could trap that message to detect fault instead of polling. Anyone have details on how this might be done in Java?
Reply With Quote
  #4   Spotlight this post!  
Unread 02-26-2012, 12:59 AM
Levansic's Avatar
Levansic Levansic is offline
Registered User
AKA: Len Evansic
FRC #0585 (Cyber Penguins)
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2008
Location: Tehachapi, CA
Posts: 146
Levansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud of
Re: New Jaguar firmware version 101

Quote:
Originally Posted by gpetilli View Post
General question for Jaguar experts - I remember seeing a post last year and can not find it now that said the Jaguars would re-enumerate on reboot and you could trap that message to detect fault instead of polling. Anyone have details on how this might be done in Java?
I just read about the enumerate on reset, today, and had the same thought with respect to using LabView. Our team was working on a babysitter.vi, to query the mode settings once a second, but we'd much rather catch the enumerate and wait 100ms to reconfigure.

The only downside is the fact that if the voltage drops enough for one Jaguar to reboot, then most or all of them will be rebooting. Not sure if they will join the enumeration in progress, or if each one will re-issue the command.
Reply With Quote
  #5   Spotlight this post!  
Unread 01-15-2012, 09:24 AM
dyanoshak dyanoshak is offline
Registered User
AKA: David Yanoshak
FRC #2158 (ausTIN CANs)
Team Role: Mentor
 
Join Date: Dec 2007
Rookie Year: 2007
Location: Austin, TX
Posts: 163
dyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant future
What you are describing sounds like the Jaguar might be browning out. You didn't mention seeing a voltage fault, but if the supply voltage drops fast enough, the Jaguar can brown out before it can sense and report a voltage fault.

First I would make sure the battery is fully charged. Then I would check the power wiring for each Jag. A loose terminal or a poorly crimped cable can increase the resistance in your power path. A higher resistance means a higher voltage drop on the wire, especially noticeable at higher current levels.

If you have access to an oscilloscope you can monitor the input voltage to the Jag to catch any dips in the supply. I would set it to trigger at a falling edge at about 6 or 7 volts.

Hope this helps!

-David
Reply With Quote
  #6   Spotlight this post!  
Unread 01-15-2012, 12:33 PM
slibert slibert is offline
Software Mentor
AKA: Scott Libert
FRC #2465 (Kauaibots)
Team Role: Mentor
 
Join Date: Oct 2011
Rookie Year: 2005
Location: Kauai, Hawaii
Posts: 63
slibert is on a distinguished road
Re: New Jaguar firmware version 101

Thanks, David, that's a very interesting idea.

We do see (via 2Can diagnostics page) the voltage dropping to around 7 amps at the same time the Amps jump to ~40 when the shutdowns occur - so this hypothesis is very plausible. I'll see if we can get an onboard voltage history made to monitor the minimum voltage on the VBus.

After reading your mail, I found elsewhere on ChiefDelphi a post saying that post-brownout, the Jaguar would need to be re-initialized for speed-mode to work again. That makes sense!

To seal the deal, some follow-up questions come to mind:

- Via BDC-Comm, after one of these events, we didn't see any other faults than the "comm" faults in the Jaguar. No indication of voltage faults (too bad, because this is the only indicator that makes me question the under-voltage brown-out hypothesis). In the case of a brown-out, what indication should we see via BDC-Comm? Or put another way, is there a deterministic way via on-board Jaguar logs to prove this is what's happening?

- How to best deal with this case, which is surely going to happen in tele-op mode during competition? After reading further on ChiefDelphi, I'm thinking of detecting the "possibility" we're in that state by exception-handling on commands to the motors. Thereafter, to poll the Jaguar's GetPowerCycled() state (which I assume should transition to TRUE after a reset completes), and if it does indicate the Jaguar was power cycled, to re-initialize it so speed mode works again. Does that sound like a good approach to you?

- Are you aware of any support for a "persistent configuration" (configurable startup defaults) in the Jaguar so that after reboot, in our case, it could enter speed mode automatically?

Thanks again for your valuable insight. I think this debugging process is inspiring to at least one of our senior students who is working on the software team.

- scott
Reply With Quote
  #7   Spotlight this post!  
Unread 01-16-2012, 10:12 AM
dyanoshak dyanoshak is offline
Registered User
AKA: David Yanoshak
FRC #2158 (ausTIN CANs)
Team Role: Mentor
 
Join Date: Dec 2007
Rookie Year: 2007
Location: Austin, TX
Posts: 163
dyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant future
Re: New Jaguar firmware version 101

Quote:
Originally Posted by slibert View Post
We do see (via 2Can diagnostics page) the voltage dropping to around 7 amps at the same time the Amps jump to ~40 when the shutdowns occur - so this hypothesis is very plausible. I'll see if we can get an onboard voltage history made to monitor the minimum voltage on the VBus.

- Via BDC-Comm, after one of these events, we didn't see any other faults than the "comm" faults in the Jaguar. No indication of voltage faults (too bad, because this is the only indicator that makes me question the under-voltage brown-out hypothesis). In the case of a brown-out, what indication should we see via BDC-Comm? Or put another way, is there a deterministic way via on-board Jaguar logs to prove this is what's happening?
In order to see a fault on the BDC-COMM or 2CAN interface, 3 things need to happen:
  1. The fault condition occurs
  2. The Jaguar senses and records the fault
  3. Either BDC-COMM or the 2CAN polls the Jaguar for the fault status
Faults can happen very fast. Sometimes faster than the information can be polled for and sometimes faster than the Jaguar can sense/record the fault.

Voltage faults are especially tricky because a dropping bus voltage will trigger a fault and then shortly after cause the Jaguar to brown out. If the bus voltage is dropping fast enough, the Jaguar will not have time to sense and record the fault. A Jaguar will brown out just below 6V, so the fact that you're seeing 7V right before the issue most likely indicates a brown-out.

Unfortunately, the rate at which BDC-COMM or the 2CAN can poll for the bus voltage is probably too slow to see the voltage drops that are related to spikes in motor current (when changing speeds rapidly). An oscilloscope would be able to see those short dips, but I wouldn't want to put an o-scope on a robot that's moving around .

Quote:
Originally Posted by slibert View Post
- How to best deal with this case, which is surely going to happen in tele-op mode during competition? After reading further on ChiefDelphi, I'm thinking of detecting the "possibility" we're in that state by exception-handling on commands to the motors. Thereafter, to poll the Jaguar's GetPowerCycled() state (which I assume should transition to TRUE after a reset completes), and if it does indicate the Jaguar was power cycled, to re-initialize it so speed mode works again. Does that sound like a good approach to you?
You are correct, checking for the PowerCycled state is what you should do. That way you can reset your configuration parameters and regain control of the Jaguar.

Hopefully you can prevent further brownouts by following these tips:
  • Inspect crimped contacts and make sure they are making good electrical connections.
  • Make sure electrical connections are secure and not vibrating.
  • Reduce overall power wire length.
  • Make sure your battery is fully charged.
  • Make sure your battery is "healthy" (old batteries may look charged, but their voltage drops rapidly under use).
  • Implement a ramp in software or use the Jaguars ramp settings to prevent rapid (sub-second) changes in speed.
  • Make sure your gearing isn't too low. A low gear ratio can make it easier to stall your motors, therefore dropping the battery voltage lower and more often.

Quote:
Originally Posted by slibert View Post
- Are you aware of any support for a "persistent configuration" (configurable startup defaults) in the Jaguar so that after reboot, in our case, it could enter speed mode automatically?
There currently isn't a way to do this in the Jaguar firmware.

In addition to all of this, I would encourage you to look at the 2012 Jaguar FAQ and Jaguar Getting Started Guide. Both of these documents (and more) can be found at www.ti.com/jaguar. They provide lots of useful information.

-David
Reply With Quote
  #8   Spotlight this post!  
Unread 01-16-2012, 11:39 AM
slibert slibert is offline
Software Mentor
AKA: Scott Libert
FRC #2465 (Kauaibots)
Team Role: Mentor
 
Join Date: Oct 2011
Rookie Year: 2005
Location: Kauai, Hawaii
Posts: 63
slibert is on a distinguished road
Re: New Jaguar firmware version 101

Many thanks again, David. We now have an action plan.

One final question: What is the default state of the Jaguar upon power up?

We ask this because we don't notice Jaguars shutting down when we control the Jaguar in Percent-voltage mode.

We are wondering if the Jaguar defaults to Percent-voltage mode, and when a brown-out occurs we don't notice the problems when use utilize Percent-voltage mode - because the Jaguar resumes operation in this mode after it reboots.
Reply With Quote
  #9   Spotlight this post!  
Unread 01-16-2012, 12:56 PM
dyanoshak dyanoshak is offline
Registered User
AKA: David Yanoshak
FRC #2158 (ausTIN CANs)
Team Role: Mentor
 
Join Date: Dec 2007
Rookie Year: 2007
Location: Austin, TX
Posts: 163
dyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant futuredyanoshak has a brilliant future
Bingo, the Jag defaults to Percent-Voltage mode.
Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 02:36 AM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi