Talon SRX Missing Encoder Counts in Labview

[Chris_Elston]

Was hoping to bounce some ideas off of others on CD since these Talon SRX motor controllers are kinda of new and we are just learning about them.

So like most, our application is a PID position based application. Our current feedback sensor is a US Digital S5 shaft encoder that is 1000 PPR. It's coupled to the output shaft of a gear box.

Working with my lead programmer student today, we was able to setup a fully closed loop control method around our mechanical linear elevator. We spent some time tuning the PID and the result is fantastic. The rate is aggressive with a nice accel and decel right at the end of the setpoint.

That part works wonderfully well....kudos to the Cross Road Guys, we like how user friendly the controller is so far.

What we are having problems with is accuracy or what looks like we are missing encoder counts. That's where I wanted to bounce some ideas of some people to see what methods we can use for troubleshooting our problem.

Here are some thing we are sure are OK.

1. Wiring for sure is ok. The closed loop method works great and the position response is wonderful. We don't have an reverse or out-of-phase condition because we are able to set up setpoint commands and everything works great there.

2. The encoder is not slipping on the output shaft of the gearbox. The follower gear is tight and everything seems like it's mechanically coupled ok.

As a test, we used velocity sampling to see if we are missing some counts. In looking at the waveform, it does appear that some skips are happening because as we accelerate a "wave form" is made and there are horizontal flat spots then a rise in the step when we look at the wave form, small ones. I wished we would have grabbed a screen shot....

So here are my troubleshooting thoughts to try on Monday.

1. The obvious, switch out the encoder. Making sure we don't have a bad one. We don't have brand new one, but I have three more to try and see if we can repeat the problem.

2. I am wondering since the encoder is wired directly to the Talon SRX, what methods of debug do we have to see about missing encoder counts? Right now we have a 1000 PPR encoder. I was thinking about trying a 250 or 360 PPR encoder. Is our 1000 PPR encoder too much for the Talon SRX interrupts?

3. The while loop that my lead programmer has is a 50 millisec loop in it's own VI. So every 50 msec, it's samping the count information via CAN BUS of course to his VI. I don't think there would be an software issues, because if we can't get a repeatable count from the get count, then the problem is not software, it gotta be something up with the two above conditions...

Any other suggestions for troubleshooting to understanding here our encoder counts are going?

I am also going to stop by AndyMark in the morning and pick up a couple of MA3 analog sensors along with the analog breakout board and see how that works. Anyone tried the MA3 analog sensor yet with the Talon SRXs?

[Ether]

What exact model number S5 are you using?

How fast are you spinning it?

[ozrien]

My experience with S4s is if the shaft isn't perfectly concentric the encoder might miss ticks. In other words if the encoder shaft is torqued at all, the encoder might miss a few ticks.

I found one bad encoder this year by just spinning the shaft with a drill and monitoring the output on an oscilloscope. With a fixed velocity you'd expect to see a pretty uniform waveform. On this bad one I saw gaps.

[Ether]

Originally Posted by Ether

What exact model number S5 are you using?

If it's an S5-1000, the "1000" refers to CPR (cycles per rev). So I believe it would have 4000 edges per rev that need to be detected (TalonSRX decodes 4X).

[Chris_Elston]

Originally Posted by Ether

What exact model number S5 are you using?

How fast are you spinning it?

We have:

S5-1000-250-I-S-N
Is the model number.

I know the output shaft ratio, it's 4.71:1 soo...lets see according to VEX the encoder shaft is 3 times the output shaft... Maybe approx 3375 RPM? Does that sound right?

[Chris_Elston]

Originally Posted by ozrien

My experience with S4s is if the shaft isn't perfectly concentric the encoder might miss ticks. In other words if the encoder shaft is torqued at all, the encoder might miss a few ticks.

I found one bad encoder this year by just spinning the shaft with a drill and monitoring the output on an oscilloscope. With a fixed velocity you'd expect to see a pretty uniform waveform. On this bad one I saw gaps.

I think we've seen that problem too, hence why I am going to try and swap some out tomorrow. I think also, maybe try those MA3 analog rotary sensors. Those seem like they might be better. I noticed that firmware 1.4 is required to run an analog MA3 sensor, are the results pretty good with the upgrade?

[Ether]

Originally Posted by Chris_Elston

We have:

S5-1000-250-I-S-N
Is the model number.

That's a 1000 CPR encoder. It's 4000 PPR at 4X.

Maybe approx 3375 RPM?

That's 4.4 microseconds per edge, assuming zero tolerance on symmetry and quadrature delay.

I could not find symmetry or quadrature delay specs in the S5 datasheet.

The S4 datasheet has max quadrature delay tolerance of 60 electrical degrees. I doubt the S5 is that bad (because the disk is assembled at the factory, not by the user), but just for comparison: with a quadrature delay tolerance of 60 electrical degrees, the microseconds per edge could be as fast as 1.5.

[Alan Anderson]

You might try connecting the encoder simultaneously to the roboRIO and tracking the encoder counts there as well, to see if there is a discrepancy between the counts reported by the FPGA and Talon SRX. That could tell you whether the problem is with the encoder itself.

[Chris_Elston]

So tonight we swapped out another encoder. Just to try it. Same problem. If not worst. I forgot my O-Scope today. I walked right out of the office and it's sitting by office door...oh well...I will remember to grab it tomorrow so we can see what's up the encoders.

So we tried plan "B" which was we installed brand new MA3 Analog Encoders from Andy Mark: http://www.andymark.com/product-p/am-2899.htm

Wired them to an Analog Breakout board instead of the Encoder Breakout Board of course...Seemed to get a reading back. The Analog In and Sensor values is like 8.3 to the E6 power. It's some large numbers we are seeing.

I assume it's normal that we can not RESET the ZERO when using "Analog Encoder"? because we were not able to reset our zero. I assume because these are used as absolute positioning. <---NEVER MIND, I downloaded the new manual from the website...section 21.17 addresses this issue... 21.17. Firmware 1.4: When setting the “Sensor Position” of an analog encoder, multiple set commands are required.

But I think what was the fork in the process of using the MA3s, was when we noticed that the SLAVE Talon SRX was not following the MASTER Talon SRX any longer. We noticed that the master's led's was RED while the SLAVE led's would be GREEN. The amperage on the master would spike up to 40-55 amps. Like the slave motor was fighting the master motor. It would not do this all the time, only spikes or momentary.

We NEVER had that problem when the input sensor was configured for Digital Encoder verses Analog Encoder. I am not sure what was causing the master and slave to fight like that. All we did was change the sensor type.

Tomorrow we will go back to the encoders and we will Oscope the encoders we have and check and make sure they work. If we have time, we can feed them into the RobotRio like Mr. Anderson suggests. I do have a 128 cycle count to try verses the 1000 cycle count to see if that helps the Talon's SRX polling.

[Ether]

Originally Posted by Chris_Elston

I do have a 128 cycle count to try verses the 1000 cycle count to see if that helps the Talon's SRX polling.

It wouldn't hurt to try. But I discussed SRX quadrature decoding with Omar. SRX decodes quadrature encoder in hardware, and shouldn't have any trouble with your S5-1000.

[Chris_Elston]

Originally Posted by Ether

It wouldn't hurt to try. But I discussed SRX quadrature decoding with Omar. SRX decodes quadrature encoder in hardware, and shouldn't have any trouble with your S5-1000.

Ether, do you think we might have a bad sensor then? I do have some brand new S5 coming. I selected 360 CPR and 250 CPR this round.

We have QTY (4) S5-1000, so far two of them are missing counts...the second one we installed today seemed to be worst than the first one. An O-Scope tomorrow might be able to tell alittle bit about what the encoders are doing.

ANOTHER THOUGHT...

Would it help to change the Encoder Status Frame from 100 ms to 20 ms? What is the intent of the Status Frames in the Talon's?

section: 20.3. Quadrature Encoder Status
The Quadrature Encoder Status frame has a default period of 100ms.

Looks like it can be adjusted to 20 ms. Would that improve refresh rates?

[ozrien]

Your master and slave driving in different directions sounds like a robot-code issue.

Changing the feedback sensor will not affect the master's applied throttle unless the master is closed-looping. In which case maybe turn off the slaves until you work out a stable closed-loop response on your master before throwing more power into the equation.

The elevated current draw sounds like your master and slave are driving against each other, not a cause, just a symptom.

The status frame rates (section 20 of Talon SRX software ref manual) affect how often the signals are reported on CAN. They have nothing to do with quadrature decoding. It's not like your pressing the Self-Test button every 20ms, or print to dashboard every 20ms.

Changing the status frame rates is possible in case you want to process encoders yourself, like you would if encoders were plugged into the RIO (with a PIDContorller object or your own custom PID code).

[Chris_Elston]

Originally Posted by ozrien

Your master and slave driving in different directions sounds like a robot-code issue.

Maybe, however the code and the Talon worked great when it was selected for Digital Encoder Feedback as the sensor type. No code changes except switching to Analog Encoder. That's when we noticed the slave unit not having the same LED sequence as the master.

Originally Posted by ozrien

Changing the feedback sensor will not affect the master's applied throttle unless the master is closed-looping. In which case maybe turn off the slaves until you work out a stable closed-loop response on your master before throwing more power into the equation.

This might be the case, we did not have a stable PID loop after changing to Analog sensor. We did have a beautiful aggressive PID loop when the encoder was connected, the S5-1000 CPR, except losing counts...If not for that, this application would be just incredible in our opinion and impressive to see.

Originally Posted by ozrien

The elevated current draw sounds like your master and slave are driving against each other, not a cause, just a symptom.

Agreed. They are doing that, because when you see the master LED "red" and the slave LED "green", you know we've got some issues there...

Originally Posted by ozrien

The status frame rates (section 20 of Talon SRX software ref manual) affect how often the signals are reported on CAN. They have nothing to do with quadrature decoding. It's not like your pressing the Self-Test button every 20ms, or print to dashboard every 20ms.

Changing the status frame rates is possible in case you want to process encoders yourself, like you would if encoders were plugged into the RIO (with a PIDContorller object or your own custom PID code).

Good to know. Thanks for the explanation of what that does. I see, so this is a software "tunnel" to get the data from the Talon SRX > to Can > to Rio project without hooking up a wire directly to the Rio inputs. Pretty cool.

[Chris_Elston]

Just an update today (tuesday). We still have not resolved the problem. I went ahead and sent an email into support at cross the roads electronic with more specific details. However I wanted to update everyone that was interested in what we have done for troubleshooting.

Tonight we:

1. Removed the MA3 Analog Encoders, because there was something happening to the Slave Talon SRX to become out of sync with the master that we did not understand. So we put back in the US Digital S5-1000 encoder, and change the sensor type from Analog Encoder back to Digital Encoder, and the Slave problem went away. We went back to having a super SMOOTH PID closed loop control, except the sensor counts are not repeatable.

2. Removed the US Digital S5-1000 encoder and replaced them with some older GrayHill 61K-128 CPR encoders. We re-tuned our PID and got the same super SMOOTH closed loop control. However still losing counts.

3. I brought the FLUKE O-Scope home and wanted to prove there was nothing wrong with the encoders we have. Can someone else review a couple of these videos? I don't see anything that pops out here at me that is abnormal? We have QTY (4) US-Digital S5-1000 and QTY (2) Grayhill 61K-128. All six videos are here:
https://www.dropbox.com/sh/yha32m2df...kUVZG16Aa?dl=0

4. Finally what we decided to was disable the robot and only watch the "Sensor value" from the Get Status VI block. We manually moved our carriage by hand after marking a spot on the elevator. The sensor position would not repeat in this case even when in disabled state. We emailed this video and our code to Cross Roads.

Tomorrow what we plan on doing are these things.

1. Removing the sensor from the gear box, and manually turning the shaft and seeing what the values are to the Talon SRX. We will document this via video as well.

2. Re-wiring the sensor direct to the Robo-rio just as Mr. Anderson suggested.

The closed loop control works amazing in the Talon SRX, if we can just figure out this "repeatability" problem....ugh....so frustrating that we are so close with the Talons SRX...

Is there anyone else out there that has a working closed loop controls with the Talon SRX with an encoder with no position issues?

[Chris_Elston]

Update today: (2-11-2015)

As I thought, the US Digital S5 encoders are all "ok", but I wanted a professional opinion so I asked US Digital to review the videos. Here is their response. Which is positive. Thank you US Digital.

"Based on the videos of the scope I would say that the US Digital encoders are working properly. The small blip that you see on the A channel is some minor cross-talk between the A and B channels. In the video it does not appear significant enough to interfere with the motor controller. The size of the blip is dependent on the length of the cable, so if the cable is longer in your application then the blip might be larger in practice. A pull-up resistor on the A/B channels should eliminate this, typically anything between 2.2k-10k ohms is acceptable."