Testing and Cause of Failure for Encoders and Hall Effect sensors
 

I have a question about the use of optical encoders and hall effect sensors in our robots. When they fail, how do you test to determine that they have failed and what was the failure mode?

For example, if we are using an encoder to control the position of our arm, and the arm stops behaving properly (not moving on command, not stopping on command, or oscillating when before it was holding still). How would we determine whether the failure was in software or in hardware? If it in hardware (like a fried encoder) how would we determine whether the source of failure was a mechanical overload or some electrical problem?

Please keep the replies simple. I'm just a mechanical, but when something breaks I'm the one who has to correct the problem and prevent recurrence. I hate to use stuff where I don't even know how to check whether or not it is working in the first place.

Do we need to add an oscilloscope to our pit equipment? If so how do I hook it up to get the answer I need? I know what to do for limit switches and pots, but encoders seem to take a little more than my trusty VOM.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

I'm working through an encoder problem now. The support guy at US Digital suggested that lacking a scope, you can power the encoder and use a digital voltmeter to check the output between the output pins and ground. For their model he said it should vary between 0 and >=2.5 volts as you turn it. Obviously, you can't turn the shaft too quickly or the DVM will not have time to settle. (Makes you wonder if an analog meter might react better) I tried this and couldn't see anything on the B channel which was easiest for me to tap. In my case, I'm absolutely sure it's the encoder because I have two, one on each transmission, and simply moved the cable from the known good one to the suspect one and got bupkus.

As an aside, I was trying to test just the encoder unit out of the robot since I thought it would be easier. I slid a sheet of paper into the slot and saw nothing. However, the support guy said the HEDS detector they use will only register the lines on the disk and not show the paper. I'm also a mechanical also so I'm not up on how this works. You'd think breaking the beam is breaking the beam.

When I get around to it, I'm going to have to build a test connector to make some of these tests easy.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

take a camera and just point it at the IR LED to test that if it works you should see the light on the lcd, even a simple camera fone works

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Chris,
We use a calibration program that our crack software team writes up. It allows us to look at the raw data as the sensor is manipulated and when needed allows a calibration routine to set mechanical center or a soft stop. Of course using a pot for arm position or steering position is easily checked with power on for 0- 5 volts or with power off with an ohmmeter and the output disconnected. Most of our pots and sensors are connectorized so they are easy to check and change.
As to using a scope, connect the scope lead common to the sensor common and the scope probe to the output lead. It should be obviously be in one state or the other. (set vertical sensitivity to 2v/division.) Move the part that is being sensed and the output should change states. If the output is always low, check that the sensor is getting power. If the output is always high, suspect that the sensor output is open.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

You could build a test kit with the EDU-RC. Wire up green, yellow, and red LEDs in a line, and another special red "warning" LED. When the encoder is moved clockwise (forward) have the green light up. When at rest, have yellow light up. When revolving CCW, have red for reverse come up. If there is no signal, or an erratic signal have the warning LED come on. This may be an easy plug and play way to check quickly for dead encoders.

If you like this idea, or something similar, but are short staffed in the programming department, send me a PM and I could probably whip something up for you.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

We had a "diagnostic" mode in our software this year. With the proper switch activated, some of the OI LEDs show the direction of rotation of the three encoders we used on the robot. That sort of simple diagnostic feedback is something your programmer(s) will have to provide if they want to quickly eliminate hardware as a potential problem.
White paper doesn't completely block the light getting to the sensor. It gets illuminated, and the sensor detects the diffuse light coming from it. You'd have to put something totally opaque there, like a bit of electrical tape or foil, before the detector will stop seeing anything.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

During what became a very long troubleshooting session, I discovered that electrical tape isn't opaque to bright IR LEDs. It turns out that a lot of plastics aren't either. I try to use scraps of metal now.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Our team installed switches on our control board to select which motion we wanted to monitor. The selected measurement was displayed on the OI LED display. Data such as input states, measured position, and commanded position relating to the selected motion are also sent out the serial port for more thorough diagnostics.

I've used IR sensors in machinery for measuring lumber and it is amazing how much material a IR light source can go through. Block it with metal to be sure it's blocked.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Chris,

1) Usually I'll setup a timer in code that will periodically printf my encoder counter to the programming port - like a constantly running diagnostic mode. It prints about once a second or so - slow enough that it doesn't hamper performance during competition, but fast enough to get meaningful information for troubleshooting in the pits. The first check is to power on the robot while disabled, read the encoder counts, rotate the encoder a known amount, check encoder counts again. Repeat in opposite direction.

2) After doing 1), in general if the encoder counts agree, you have a software problem elsewhere. If counts don't agree you have a hardware problem. If counts only decrement, or increment no matter which way you turn your encoder, your non-interrupt phase is disconnected or blown. If you get no counts in either direction, the phase connected to your interrupt is disconnected or blown - or both phases are disconnected or blown.

3) With the encoders we use, the only clue to mechanical failure we've found is to compare the rolling resistance to a brand new encoder. With optical encoders with no detents, too much resistance means you've destroyed the bearing which we've done in the past with excessive side-loads. We don't have a scope either, so our electrical diagnostic is just swapping it out with a new one and seeing if it flies.

If you're reluctant to drop money on a scope, this might interest you. Some isolation circuitry is recommended though... your mileage may vary =).

-SlimBoJones...

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Actually we already have a scope, but it is rather large and heavy to be dragging to away competitions. Especially if we are flying. The software looks interestng as we have plenty of laptops around. I'll have to check that one out. What kind of isolation circuitry would I need?

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Yes seconded, what kind of isolation circuitry would be needed? Would attaching anything besides a mic to my line in on my new laptop void its warrantee and finally I'm assuming I'd have to split one end of a minijack cable in to two probes?

Thanks, this looks interesting...

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Seeing as this is a software only device, one must assume that the bandwidth is only what the sound card is capable of handling. Most sound cards are designed to handle an A/D conversion for CD quality audio which means, best case bandwidth of 22kHz with acceptable distortion of an audio signal. Add to that the aliasing of the conversion (dither, noise, approximation, etc.) you may be only able to see real bandwidth in the 15kHz range. One must assume that using the line level inputs, a maximum voltage of just over one volt RMS is the upper limit, which would require an input stage with attenuators capable of analyzing the 12 volt plus signal at the output of the speed controller.
On the plus side, the PWM output is around 2kHz and the limited bandwidth would certainly hide brush noise and other switching by products present when looking at motors. You may not be able to see the raw output of a sensor, particularly one that has an output frequency above the A/D. Additionally, the inputs are single ended, where one side of the signal path is computer common. As such anything that is common mode will raise the chassis and ground of the computer to a point that might interfere with normal computing operations. A transformer would be a big help in preventing these currents but needs to be shielded for obvious reasons. There are other scope accessories that can attach to laptop and desktop computers that may be easier to carry and less expensive than a full up scope.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

A logic analyzer is useful in these situations. You can actually see the steps and phase from a run. Professional grade logic analyzers are expensive. A pic however can be turned into a acceptable analyzer with the proper hardware and PC program. I believe Parallax sells one and there is this link.
http://kronosrobotics.com/pj_analyzer/analyzer.shtml
Great for testing coproc communications and I2C - SPI.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Al,

Let me see if I understand you correctly and maybe you can translate the parts I don't understand into something a lowly mechanical can get his head around.

Ok I think you just said the max sample rate would be 15kHz. Since I don't plan measuring the speed of any motors directly, this is probably not an issue. If the signal is cycling that fast I need to do something different.


I think you said we have to knock the robot's 12V signal down to 1V max. I assume our electricals can handle that. But a hint or so wouldn't hurt


One of the things we might want to look at is the PWM output and happily it falls within the likely range of the sound card. However certain sensors might output at a frequency about the range and so not read correctly.


If we don't do this right, the computer might fry, or at least get confused.


And these obvious reasons might be? and it prevents this how? and just how is the shielding accomplished?


There are other ways that might be cheaper and lighter than a full up scope. And these might be?


As I mentioned before, we already have a scope, we just never thought it would be useful to drag it with us to a competition. Of course our sensors have been pretty limited as well, so we haven't really needed it.

I think we are in sort of a chicken and egg situation. We don't use sensors because we can't test them when things go wrong, and therefore don't trust them. But since we don't use them we don't develop the knowledge and capabilty to test them. Why test something we don't use? So I guess you could say the goal of this thread is to lay an egg and render the question moot.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Chris,

I would suggest this.

On another note, I feel that the single most important function which the mechanical team can provide the electrical team is to integrate the sensor(s) and wiring into your design from the beginning. In other words, protect it so that it will not break...

Walk around any competition and you will see wonderfully designed mechanical mechanisms with limit switches or potentiometers mounted with rubber bands, tie wraps and hot glue. These devices and their wires are invariably exposed to the interface between 130 lb robots colliding at 5 to 10 ft/sec.

Just as invariably, you will hear the mechanical team complaining about the lack of robustness of the sensors. In my view, it is the shortsightedness of the mechanical team which allowed such a fragile item to be exposed to such an environment.

Don't leave the mechanical design of electrical components to the electrical team. They do not have your experience.

As an example: When you CAD a gearbox, add in an extra gear and mount for an encoder within the gearbox at design time. Without side loads and properly protected, these devices should be as robust as anything else in your geartrain.

Designing for failure tends to encourage failure. Worrying about failure analysis during the competition can become a self fulfilling prophesy. Break your pots in the fall and learn how to keep them from breaking.

Driver practice, software verification and strategy aside, the winning robot is not the fastest, the most powerful or the most clever design. At the end of the day, it is often the last robot standing which wins the competition.

JMHO,

Mike

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Chris,
I went to the site Mike linked to and wow, you get a lot for little money. It looks like it would solve the problems with using a sound card. Cables look a little short but you can probably get around that.
When sensors just are not working right, a scope with better that 15kHz bandwidth can tell you a lot. Some failure modes produce a lot of noise in the output and a low bandwidth may attenuate those spurious signals to a point where you would think they are insignificant. It is also possible that the sensors can't keep up with the stimulus, and a scope will show you the effect on the sensor output as speed increases. From and electrical standpoint, I will grab my Fluke first and a scope second.
As to using transformers...the device will couple signals without having to tie chassis to the circuit ground but it does so with magnetic fields. A shield is required (transformers are available with shields) to prevent magnetic fields from coupling into the transformer. Spinning motors do have some nasty magnetic coupling that would affect the display and accuracy. One other thing that came to me is that sound cards are capacitively coupled so looking at a DC component is out of the question with a sound card solution.
Many teams do not carry scopes with them to competition. We usually carry a beat up one and actually take it out from time to time to check on something. Since it is not new or in good condition, a bunch of bubble wrap will usually do OK to put it in the shipping container with the robot.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Al, and Mike,

Thanks for the help. The item Mike linked to does look pretty neat and I will propose getting one to our Fearless Leader. We have a plethora of laptops that were surplused by one of our sponsors (every kid on the team gets one as a loner, they remain team property) and they all have USB and meet the specs. Maybe they can be useful for something besides scouting

And yes Mike we do design the sensors into our design.

Last year we had an encoder. The encoder measured the rotation of a jack shaft. The shaft had a gear mounted on it and the encoder had it's own gear. It was mounted fairly rigidly and there was at least 0.015" slop between the teeth of the gears (approx 1/4 tooth). The encoder was buried in the robot where it would have taken major structural damage to be impacted by part of another robot. Shielding destroyed, major bent aluminum, nothing subtle here, 6" of deflection of the outer shell for contact.

We failed three encoders from two different manufacturers before we gave up. Fortunately our programmers followed my suggestion and made it so the encoder was not essential to operation. (always provide a manual overide for all automated functions) It was never determined just what caused the encoders to fail. One of them is still on the robot, but it has been disconnected. I'm not sure the fate of the others or if they could still be evaluated. We looked at using a pot but it didn't have the resolution we needed to be effective.

I have made it a project for the off-season to better understand how to use these neat little toys. As long as they worked, they worked pretty well, but losing 3 in one and a half competitions was a little too risky for us. I'm thinking about going to Vex chain and sprockets for mechanical isolation to make sure the motion transmission mechanism can't impact the encoder shaft. But that won't help if the real problem is power spikes floating around the electrical system, or the shock of being hit by another robot.

ChrisH

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Chris can you fill us in on the sensor that was failing and what the input frequency may have been?

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

I'd have to look at the robot to be sure, but as I recall it was a 256 step optical encoder with quadrature. The arm was rotating around 4 rpm and the jack shaft was about 5 times that due to a 5:1 reduction between it and the arm. The arm position was what we were trying to control.

So say we had 20 rpm and 256 steps/rev, that works out to about 17Hz. Well within the capability of the sound card scope anyway ;)

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Chris,

This thread caught my eye, and I have a few observations:

1) You mentioned that you don't use a pot because of resolution
issues, and I am wondering if you have tried a 10 turn pot on the
drive shaft that turns 5 times faster to resolve that problem. By
hooking the feedback closer to the motor, so to speak, you can also
remove instability arising from the slop in the gearing between the
motor and the arm.

2) With regard to sensor testing, we have used led indicators as
part of the design, so that a quick look can determine if it
is "blinking," software running in the RC that prints a message
each time a sensor value changes, and finally, a scope. There also
relatively cheap multi-meters that have a "frequency display" for
a TTL input level, and you can use these to see if a high pulse
rate sensor is working correctly.

3) I am a little worried about triggering an input to the RC
at 17khz and expecting the RC to track it, although I have not
tested the limits of the interrupt based inputs on the current RC.

3) The electrical environment in a robot is very noisy, with lots of
spikes on the power lines. If a sensor requires power from the 12
volt line, we bring that power in through a substantial inductor, a
capacitor bypass, and then on board voltage regulation. We usually
ground a sensor at the RC, instead of elsewhere, to avoid
any ground loops that might introduce spikes into the sensor output.

Although we have had some fun with PID control systems running amok
when a collision knocked the analog connector to the RC loose, we have
never had a sensor fail electrically, and we tend to salvage the more
expensive ones by desoldering from a prior years custom circuit board
and putting them into a new custom circuit board made for each build
season.

I hope that this helps...

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Doctor B,

It was 17 Hz that he reported, not 17 KHz.

Mike

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Sorry Chris, I tend to disappear from CD for days at a time, so I apologize for the late response.

http://www.virtins.com/ shows a little diode circuit to clip the signal at about 1V, which Al alluded to as roughly the limit of the sound card. Additionally, an amplifier set up as a unity gain buffer (doesn't actually amplify) provides some isolation, and impedance matching. I don't have any schematics for you handy, but I'm sure they can be found on the net, or someone here can provide them. You could probably get away with just the diode circuit in the link above, if you're using it on the 5V signals from your sensors. I can't make any promises though =). I haven't used the sound card oscilloscope myself yet.

Lastly, we used 256 ppr quadrature encoders on our drivetrain last year (Grayhill 63R series). We initially had a lot of problems when we bumped into things, or our drive chains backlashed. We worked out our max RPMS, and pulse rates, and they were all well below our maxes...

What we didn't account for were "jolts." Momentary hits that moved the encoder just far enough to record a count or two (at 256 ppr, this isn't much movement), and fast enough that the RC would read the quadrature signal incorrectly. The end result was a count going in one direction was being misread in the other direction - this is what happens when you read a quadrature signal too slowly. This caused havoc with our PID loops, which functioned great until you smacked the robot, and things would go haywire until you settled everything down again.

We actually worked around this with a quadrature stretching circuit, the schematics of which you can find here, courtesy of Kevin Watson.

-SlimBoJones...

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

I like the LED idea. Especially if we can implement it using the LEDs on the RC. I'm not sure how we were grounded, but I'm pretty sure we did it at the RC. At least there were no extra ground wires running around. We might have had an unintentional ground, but we're pretty good about following good electrical practice (two of our mentors do this stuff for a living and they make sure we do it right) The problem with pots was that they only give 255 "clicks" and we needed 540 to get the resolution we wanted.

I'll have to look more closely into just what was happening at failure. We spent a lot of time getting smacked while we were minding our own business stacking tetras. While we are reasonably sure there was not direct contact with the sensor, we haven't evaluated the possibility of shock inducted counts at a high rate.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

The RC analog input is ten bits, giving 1024 steps. The simple definitions in the default code throw away two bits of resolution in order to emulate the pre-2004 system, but the full range of values is available just as easily.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

The resolution, on the old PBASIC RC was 255. The resolution on the
new RC programmed in C is 1024 (only on the RC, not the OI), but I will
agree that your desired resolution is too close to the limit for comfort.

If you are using busy polling in the "fast loop" to detect state transitions,
you can be missing changes that happen while the RC is not looking.
This happens while the RC is busy with its periodic processing of inputs
and outputs, between the calls to get and put a packet.

One of the things that we do here is track the minimum number
of consecutive on, and off, measurements and report that as
a diagnostic to indicate the risk that we might have missed any.
You can indicate that the minimum has been violated with an
LED output, and print the number when the computer console is
being used. We start to get worried if the minimum number
gets lower than 5 or so.

Another strategy is to use an interrupt scheme to track the encoder,
but this method has plenty of its own potential gremlins. You have
to be very careful to avoid all the pitfalls, but an interrupt based
scheme is capable of tracking state transitions at a much higher rate
because only higher priority interrupts will interfere with it.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

firstly, let me say that i have no actual experience with the first gear, (i dropped out of high school way too long ago for that), but i did buy myself a vex kit to play with and the hardware seems to be pretty much the same. you guys just have a little more of it.

anyway, the problem with polling an encoder on my controller would be that for 2ms out of 18.5ms it's busy in a timing loop generating its pwm outputs. that seems like an excellent opportunity to miss a signal.

conversely, the problem with using an interrupt to catch the signals from the encoder is that if it occurs during the 2ms that the processor is counting cycles to produce the pwm it will cause glitches in the signals to the motors.

overall i'd say, that if your issue is positioning, go with a pot. you do have ten bits of resolution. on the other hand at 17Hz you're likely not going to miss anything.

oh, well, guess i've contributed approximately nothing.

later.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

Somewhere along the line this info seems to have gotten garbled or missed entirely by our team. Either that or I just didn't get the info from the right guy. While I'm a mechanical sort of guy, I have helped design and control process equipment. Due to some very bad experiences I absolutely hate input sources that give me back only 1 and 0 for non-binary input. For a temperature I don't want to hear from the sensor "The temperature has increased 3.7 degrees in the last 2 minutes." That may be an important secondary factor, but what I really want is "The temperature is 253F." If the rate of increase is important I can track that by comparing subsequent readings easily enough.

Encoders can tell me how fast something is moving, and how far it has moved from the start point (assuming the integration routine is working and nothing has been skipped) but they will not tell me absolute position. (unless I use a very special encoder which is out of our price range as I recall) Since position is what I'm trying to control, I'd rather work with a directly correlating signal than deal with trying to integrate or differentiate to get what I really need.

Do I just have to change the variable definition to increase the bit size?

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

have a look at the data sheet for the 18f8520 at the microchip website. it will explain how to configure the adc. eight bits of the output go into one eight bit register and the other two bits have to be extracted from another. whether you get the high order or low order eight bits in the register is configurable. you'll probably want the low order eight bits so you can move them to the low order byte an integer, then mask the high order bits into the two low order bits of the high order byte. wow, what a sentence.

i'll try to formulate a clearer response tomorrow.

g'nite.

Re: Testing and Cause of Failure for Encoders and Hall Effect sensors
 

The Get_Analog_Value() function returns the full 10-bit result from the analog-to-digital converter. For example, will give you a number from 0-1023 representing the voltage (from 0 to 5 volts) on the first RC analog input.

Typically, one will #define a more friendly name for the signal: And that's all it takes.