Unexplained intermittent CAN / 2CAN Jaguar problems at GSR

[John Heden]

We had a couple of interesting and still unexplained catastrophic CAN problems at the Granite State Regional this weekend that we are still puzzling over. We use the 2CAN interface to connect 5 ‘tan’ Jaguars and everything seemed to work fine while we were tethered and also well during most of our matches.

We had two matches, however, where we failed to move at all with an endless stream of CAN transaction time outs being reported to the driver station diagnostic tab. Our drivers reported that they were able to recover well into one of these matches by rebooting the Robot and did successfully drive a bit in the little time remaining. Everything was always fine once we were tethered back at the pits and we were only able to reproduce this stream of CAN errors by actually unplugging one of the CAN cables. Interestingly, we were also able to reproduce the same sequence by disconnecting the Ethernet cable from the 2CAN interface. I was hoping to see some type of different error to help differentiate between a problem on the actual CAN bus vs. a problem with the 2CAN adapter. While the error messages were consistent with a cable disconnect of some type, it seems unlikely that a robot reboot would have resolved this type of problem and we were unable to locate any intermittent connections despite our best efforts.

Our robot is now bagged and tagged and headed to the Hartford regional so we are unable to do any further experimentation and troubleshooting. There is now some pressure to abandon the CAN interface and rewire everything to our historical PWM interfaces if this problem can be understood and solved.

Here are a few questions for anybody familiar with the Jaguar – 2CAN interface. My recollections are by memory but hopefully accurate. Any thoughts would be most appreciated.

1) How do folks recommend powering the actual 2CAN device ? There are references to ensuring the Dlink radio is powered by the boosted 12V (white Wago connector) to help avoid excessive voltage drops and possible problems. The 2CAN itself has a wide voltage range (6.5 to 28V) but should it be connected on the raw 12V fused power bus? Our autonomous start is probably the worst case power consumption where we simultaneously power up 3 big CIM motors to near full power.
2) Simultaneously accessing the 2CAN webpage while our robot code is simultaneously executing does result in a number of our robot CAN transactions failing (~ 1/1000). This is more of an annoyance for us than a problem but is suggestive of some type of system vulnerability when the 2CAN is being accessed from multiple sources. The WPI libraries seem to have access protection but the 2CAN itself may have some issue when it is being accessed by both the CRIO robot code and a browser on the2CAN page.
3) There is some mention that the FRC system does send additional CAN related messages to CAN based robots, can anybody confirm this possibility ? Could there be some type of negative interaction with the FMS system?
4) I did speak to a few CAN Jaguar users who spoke of possible overloading the CAN bus and needing to drop their access intervals down. The current WPI library interfaces, however, look like simple blocking calls where all access is nicely serialized and completes before the next sequence is initiated. While all of these calls may eventually slow down our control loops, it seems unlikely that we could overload the bus.
5) We do see some type of “too much error data!” message once in a while we were playing but could not locate this error message anywhere in our code or in the WPI libraries. Any idea as to the source of this unique message?
6) There is a rev 66 1/29/2011 for FRC plug-in update for FRC posted on the Cross the Road website. We bagged our robot before I could confirm what version of 2CAN plug-in is incorporated in the 2/11/2011 V28 CRIO update? Is there a need for manually installing this file or is the latest included with the V28 update ?

Thanks in advance for any CAN thoughts…

John