Sensor Failure Detection

In the automation world, back-checking sensors (did it go off when we expected) is common to detect failure modes. Not so much in FRC. I’m looking for simple and accurate ways to check common FRC sensors for failures during operation.

The pigeon gyro already has some handy feedback for that. If you’re writing your own calibration code for a gyro, you can use a standard deviation calculation on your array of calibration values to see if anything is out of whack. In actual competition time after you’ve enabled, I suppose you could cross-check it during turns against your encoders.

Drive encoders are more difficult. The only way I can see is to check when you expect both to be moving. If one isn’t, make the assumption that it has failed and only use the other.

Infrared or other presence sensors can be back checked. If it’s on, and we ejected a game piece, did it go off in time?

What other common FRC sensors or error checking schemes have you seen and used?

This year, we did encoder failure detection on our shooter encoder. We did this by running a simulation of the shooter model on the roborio, and checking if the the modelled shooter velocity was above a certain minimal velocity, but there was no change in the number of encoder ticks in since the last update. If it detected an encoder fault, it automatically switched to an (empirically tuned) open loop voltage.

This worked well for a velocity control system, since there wasn’t much drift between the estimate and the actual system, but applying this to a position controlled system would be much, much harder.

IHMO, it’s probably not worth it to do a lot of fancy logic like this in FRC. It’s hard to do right, and your efforts would likely be better spent making sure that your sensors don’t fail in the first place (I’ve found that hot-gluing encoder wires does wonders). That being said, it is fun to do, and I’m happy that we ran the logic that we did this year, since I’ve had the idea to do something like this since I first learned about model-based control and observers, and it was a lot of fun to finally see it working.

We’re not yet at real-time back checking of sensors, but after some interesting failure modes seen at our events, we are seriously looking at sensor check-out procedures to do whenever we unbag the robot and prep it for competition.

This includes physical once-overs like checking the wiring and connectors, to software that could exercise the sensors and make sure things are working as expected. For example, we could have a test mode autonomous that we could run in the pit (while the robot is on the cart) or in a practice field. One example would be to run the drive wheels forward for one second and confirm that both encoders moved in the correct direction.

We had talked about putting wrappers around the wpilib Ultrasonic class to provide some interpolation between physical sensor measurements; this would also be the opportunity to add some kind of built-in error checking.

The really professional way to do this would be to have an expected state of the robot (based on past states, sensor observations, etc.) and to determine whether a sensor reading is reasonable before using it. It gets really easy if you’re also keeping track of error in your state estimation (like if you’re running a kalman filter like 971).