cRIO troubleshooting

[Sconrad]

Originally Posted by Greg McKaskle

If the Communications and Robot Code light are on, that means that the cRIO has a startup app and image that is self consistent, and that the code is running. If the PWMs aren't being updated, that could be due to a number of reasons.

The analog breakout in slow 1 needs to have power and a breakout in order to return something other than 0.0. I don't know of anything a cRIO can do alone to melt a breakout board, but I could see how a faulty breakout, one that had power and ground pins shorted together might build up enough heat to melt, and clearly wouldn't measure anything useful while that was going on.

If you did something to your cRIO so that it is no longer valid for competition, I'm not sure that sharing that approach with other teams is the most helpful thing to share. Swapping cRIOs actually swaps lots of stuff including FPGA images and other libraries. Did you check what error messages were on the Diagnostics screen? Did you verify that the versions of the cRIO image and FPGA image were as expected?

Finally, would you please explain what was done to the cRIO to "fix" it but make it illegal?

Greg McKaskle

Thanks for the reply! This is meant to help teams that would use the robot in demonstrations, as we will be using ours, where FIRST rules don't necessarily have to be followed to the letter. The only thing that makes this cRIO ineligible for competition is the fact that Slot 1 is busted (the cRIO took quite a hit in competition in 2010 and so several things were bent, but worked until now). Otherwise, the cRIO works. The point was not to share how we busted the cRIO (we drove up a ramp and the circuitboard flipped over and landed on the cRIO ), but to share how we made it operational so that we would not have to spend several thousand dollars on a new cRIO (we already bought one at the FIRST price, so we would have had to buy one full price). This was not a software issue (I can verify that, the code we were using was a simple tank drive and I ran through it at least 8 times during the debugging process), but a hardware issue. The software was only modified to adjust to the adapted hardware configurations we used. cRIO image and FPGA version were as expected. The only errors on the diagnostics screen had to do with the analog inputs, which was to be expected, seeing as both the cRIO slot that they were attached to and the analog breakout chip were busted. After examining the cRIO, we think the issue is in the board in the cRIO, which is probably cracked or corroded due to the bangs it has taken over the years. I would imagine this is a rare issue based on the difficulty I had finding documentation on a lot of the problems we had. I hoped to offer some documentation through this thread.

[Sconrad]

Also, this is a temporary fix. We are looking in to fixing the broken cRIO slots so that the cRIO is once again competition worthy, but we needed a temporary placeholder solution until we can go about accomplishing this.

[Greg McKaskle]

What I got out of that was ...

Our robot wasn't working, and in debugging it, we discovered that our cRIO had some slots that didn't work. Could've been that those hard hits from last year really hurt something. Luckily we have a spare. We need to get the other one fixed.

------------
Since I actually posted after the 9:21 post, but before the others showed up, I couldn't tell what was going on, and envisioned team members with soldering irons or hack saws. It is indeed possible to use the alternate slots for digital and analog with a few changes to Begin.vi or your startup code, but as you note, the blinking robot light is expected to be connected to slot four, and the battery is expected to be read from slot 1. Without those, most FTAs will not let you on the field.

I'd encourage you to contact NI and send the cRIO back get it repaired or replaced. You may even get it back in time to use as a spare for the competition. Also, look at the bus connectors along the bottom. Look to see if there are bent pins or debris that is shorting it out. If so, consider what operations may have led it. You may also try wiggling the connector and see if slot 1 or slot 4 seem looser than the others. It may be that the hit busted the solder joint. You may want to consider a more protected mount of the cRIO for the upcoming season.

Greg McKaskle

[jhersh]

Originally Posted by Sconrad

The only errors on the diagnostics screen had to do with the analog inputs, which was to be expected, seeing as both the cRIO slot that they were attached to and the analog breakout chip were busted.

The Analog errors do not check the status of the hardware... they only come from software bugs. What specifically were the errors you were seeing?

-Joe

[Sconrad]

We hope to get the cRIO back in full working order at some point, but for now, all we need it for is practice, and we have a new cRIO incoming, so getting it totally fixed is not a 100 percent priority. Thanks for the feedback though! @GregMcKaskle: I can see how I might have miscommunicated there. Thanks for pointing it out so I could clarify it. @jhersh I can't honestly say I remember the exact diagnostics message. I was pretty sure that they had to do with the analog inputs, but if they have to do with the code then that seems unlikely, as we did not even refer to analog inputs. Maybe the errors were from the driver station code trying to get the battery voltage? Anyways, foolish me did not record the errors and I am basing this information on memory, so it could be inaccurate. However, I can say that we did look at them and they were not anything foreign or totally unexpected. Like I said, we resolved the problem to the point that we are satisfied with the results. Thanks for the help though guys!

[Al Skierkiewicz]

Conner,
I am guessing from your description that you fried a chip on the analog breakout board that plugs into the 25 pin connector on the top of the analog module in slot 1. From that description I believe you had reversed the battery leads to slot 1. In the process you damaged the five volt regulator on the board. That is the only chip on that board. The board is not reverse polarity protected. There is a single LED on that board that lights when correct voltage is present and the regulator is functioning. Unfortunately, simply replacing the chip may have not repaired everything on the board.
It is also possible that a slot on the Crio is defective. Often this occurs when foreign material drops into the connector or a pin on the connector is bent. While a bent pin is sometimes recoverable, generally this is not the case.
Please remember that the battery voltage on this board is used (through the jumper) to sense battery voltage for the Crio. Should it fall to less than 5.5 volts, the Crio disables output for motor control. The intent is to allow the battery voltage to rise and prevent the Crio from rebooting. The power supplied by the PD may fail when the battery falls below 4.5 volts. This voltage (as sensed) is also passed to the dashboard and is read during the match by the FMS.

[Mark McLeod]

Originally Posted by Al Skierkiewicz

The board is not reverse polarity protected.

One small update...

The Blue PCB Analog Breakout (circa 2010/2011) is reverse polarity protected.

The Red PCB Analog Breakout (circa 2009) is not.

But if they are dropping heavy objects onto their Breakout boards and modules then all bets are off.

[Sconrad]

Originally Posted by Al Skierkiewicz

It is also possible that a slot on the Crio is defective. Often this occurs when foreign material drops into the connector or a pin on the connector is bent. While a bent pin is sometimes recoverable, generally this is not the case.

Based on our attempts to isolate the problem, it seems that at least two of the slots are defective. We thoroughly checked both and there don't seem to be any pins bent or foreign matter. Therefore, we think that the board within the cRIO is damaged. Like I said earlier, this cRIO had the misfortune of being underneath a flipped robot, and so it is likely that this caused initial, unnoticeable damage and that, over the course of the years, the damage has increased until the slots malfunctioned. Thanks for your suggestions though!

[Al Skierkiewicz]

Conner,
The Crio is built like tank. I would be very surprised if the accident led to your current failures in just two slots. However, I would suggest sending in for a diagnosis. We had a Crio complete failure due to conductive fluids combined with power on. There were several circuit traces that dissolved.

[Sconrad]

We found that one of our analog slot 1 modules was not working with our other cRIO. While we have not confirmed this yet, we think this could mean that slot 1 on the broken cRIO might still work. The busted analog module's 25-pin connoector port was bent, so it is possible that it caused a short that damaged a chip on the board (I am mainly a programmer, so it is possible that I am totally wrong in what I am saying as I am totally out of my area of expertise)