Re: Jaguars intermittently disconnecting
 

His name is Andy Watchorn. A "watchdog error" as people are familiar with will make all motor controllers turn off at the same time. Not some of them on the bus. On the other hand, if noise is occasionally making the safety protocol fail to get through to a motor controller, then that controller will drop out until the next renegotiation. There is a 2 second renegotiation period.

-Joe

Re: Jaguars intermittently disconnecting
 

Well, I just confirmed with our student programmer that our watchdog has been disabled for all of these incidents. Also, we received no warnings about the watchdog not being fed, so I'm inclined to believe this is something else, unless there is a lower level watchdog potentially malfunctioning. We also experienced the same problem with BDC-COMM but an otherwise identical set-up, so I'm still leaning towards EMI.

Matt

Re: Jaguars intermittently disconnecting
 

That sounds like the symptoms I'm seeing. It doesn't explain though why this problem didn't appear until after I upgraded to v25 and v92.

Is there any code on the cRIO that could possibly be causing the set packets to be dropped? Also is the code supposed to send out a global heartbeat signal to the bus?

Re: Jaguars intermittently disconnecting
 

Joe,
Check with Andy, I never did fully understand the issue but when he explained it to a C+ programmer, he understood the problem and corrected it. (I am a hardware guy) That team had no further problems through the season and up to the Champ finals.
It manifested itself in a very similar manner to what has been described here. It did not show any watchdog error but it seemed to be less than a two second delay. If the robot was running, it would stop for a moment and then run normal. Since many teams used pneumatics last year, most manifestations occurred while observing the compressor.

Re: Jaguars intermittently disconnecting
 

It seems like a bug with communications, like I said before, we exhibited identical symptoms with the jaguar controlled by the desktop application with the cRio completely out of the loop. Unless the problem also exists in NIs application, that would seem to preclude it being a software issue.

My theory on EMI works if you think about the compressor kicking on causing a sudden surge of inrush current, or maybe IFI cheaped out and there's no snubbing diodes in those Spikes ;)

I don't know, I don't mean to seem stubborn, but I can't find any consistency with a software glitch.

Unfortunately I don't usually make it to the shop during the week (new job, don't want to start skipping out just yet!) so I won't be able to look in to this further until next weekend.

Re: Jaguars intermittently disconnecting
 

That's a key difference in our problems then. Driving all 4 drive motors at full speed via BDC-COMM makes no difference, so unless there is a interference problem with idle motors that only shows in the small gap between the cRIO and the black jag, there's something going on in software.

I'm fairly sure there is something weird that makes our setup different from NI's since I can find no evidence of a heartbeat signal being sent by the cRIO over the network even though Wind River shows FRC_JaguarHeartbeatTask (or whatever it's called) on the cRIO (though it's always shows as [delay])

and Al, this is a different problem. All the jaguars seem to fail independently of eachother, something a watchdog would not do (the new per-motor watchdogs have been disabled in my test program)

Re: Jaguars intermittently disconnecting
 

There's still a heartbeat (in fact, that's what keeps the motors enabled). However, I believe that's handled by the BlackJagPlugin.

Re: Jaguars intermittently disconnecting
 

are you sure the cables are good? i have had finicky cables before. usually a stripping error dmaging a wire or bad soldering on the termination.

also make sure they are firmly put in.

did you use 100 ohm resistors for termination? or 120? CAN standard is 120 but for robotics use you need to use 100.

Re: Jaguars intermittently disconnecting
 

This sounds very similar to a communication problem that I was having with my Jaguars. I built my serial cable/CAN terminator to the manual's specs which included putting the terminating resister in the shell of the DB9 connector on the cRIO side. This meant that the serial traffic and CAN traffic were running on parallel wire pairs for the length of the cable between the cRIO and the first Jaguar.

I learned from Joe H. that the BlackJag plugin forces the serial connection at 115200bps, which is pretty fast and is more susceptible to noise than slower bitrates. I suspect that CAN traffic was crosstalking with the serial traffic, since I'm using flat wire, and causing the serial connection to fail. My solution was to shorten the serial/CAN cable from 6ft to 2ft, and I haven't had any connection problems since then.

Joe also suggested moving the terminating resistor from the DB9 end of the cable over to the RJ-12 end. This would remove all CAN traffic from that cable and further decrease the potential for crosstalk or CAN-induced noise on the serial bus.

Re: Jaguars intermittently disconnecting
 

As far as I can tell that's not the case. Set commands are considered trusted and therefore can keep a jaguar alive. I've also never seen anything in the plugin to run the heartbeat.

In terms of my cables I haven't found anything wrong with them. Again it works fine in BDC-COMM all the way up to pulling the serial connector from the cRIO and plugging it into the laptop, leaving all the cables in the same configuration. I've checked that all connectors are seated properly, and there is no correlation between failure and physical location on the bus.

While the idea of interference occurring between the CAN and serial cables is reasonable, again BDC-COMM has shown no issues in communication in an identical setup (minus the cRIO), and under the (non-FRC version) source code runs under the same settings as the cRIO.

Re: Jaguars intermittently disconnecting
 

That is true. The Jaguar CAN Driver handles the heartbeats, but it also monitors the trusted commands. If you have sent a trusted command recently enough, then the heartbeat isn't sent. However, if you just tell the Jag to go to position X one time, the heartbeat will be sent every 75 - 90ms to keep the Jag running in the absence of trusted commands.

Are you running much other code on the robot at the time? Can you use the System Manager in LabVIEW under the Tools menu >> Real-Time Module >> System Manager to look at the processor utilization while the Jags are having this problem? I'm interested if this could be the cRIO being too busy with other things to keep up with all the Jags.

Thanks,
-Joe

Re: Jaguars intermittently disconnecting
 

Remember that the BDC-COMM does not put as much traffic on the CAN bus as the cRIO does, because the BDC-COMM usually sends messages to one Jaguar at a time, and only when you update a value. In contrast, your code is probably sending messages to four Jaguars every 20ms. (In my testing this took about 10-12ms for 4 Jaguars, but it may take longer if there are errors. I don't know what CAN does for error prevention, but I do know it has good error detection.)

Joe, must EACH Jaguar receive a trusted message to stay enabled, or will a trusted message to one Jaguar keep all of them alive?

Re: Jaguars intermittently disconnecting
 

Ah, that makes more sense. For some reason though that wasn't happening on last year's autonomous code. I'll check if there's a similar problem going on w/ v25. I assume the heartbeat is sent on a per-jaguar basis in this system?



I'll check when I get to the robot today, but this is happening under minimal code. The teleop loop is literally "leftA.Set(stickA.GetY());". I've also tried killing processes that seemed less important via Wind River (FTP task, RSL control, netconsole, etc) with no change.

Re: Jaguars intermittently disconnecting
 

Each Jaguar is independent. That's why one may drop out while others are fine.

-Joe

Re: Jaguars intermittently disconnecting
 

I was also able to successfully connect through BDC-COMM but not through the cRIO. I suspect that BDC-COMM connects with the Jaguars at a slower bitrate and is thus more tolerant of noise/crosstalk.

Here's the discussion thread where I outlined my problem and eventually found resolution via a shortened cable. Do my symptoms seem similar to your issue?

http://www.chiefdelphi.com/forums/sh...ad.php?t=89733