cRIO interrupted during reimaging

We were reimaging the cRIO device and our network cable came loose during the “Formatting CompactRIO device…” phase.

After restoring the network link, we can ping the DS (10.0.0.5), but cannot ping the cRIO (10.0.0.2).

On the cRIO the Status light is blinking orange and the power light is green.

Restarting the cRIO imaging tool says that No CompactRIO devices were found. Verify the network connection.

Looking for advice on how to proceed…

Thanks

What exactly does the blinking look like? The pattern can tell you what error the cRIO is encountering.

From the cRIO documentation:

Table 3. Status LED Indications
Number of Flashes Indication
1 The chassis is unconfigured. Use MAX to configure the chassis. Refer
to the Measurement & Automation Explorer Help for information about
configuring the chassis.
2 The chassis has detected an error in its software. This usually occurs
when an attempt to upgrade the software is interrupted. Reinstall
software on the chassis. Refer to the Measurement & Automation
Explorer Help for information about installing software on the chassis.
3 The chassis is in safe mode because the SAFE MODE DIP switch is in
the ON position. Refer to the Configuring DIP Switches section for
information about the Safe Mode DIP switch.
4 The software has crashed twice without rebooting or cycling power
between crashes. This usually occurs when the chassis runs out of
memory. Review your RT VI and check the memory usage. Modify
the VI as necessary to solve the memory usage issue.
Continuous flashing
or solid
The device may be configured for DHCP but unable to get an IP address
because of a problem with the DHCP server. Check the network
connection and try again. If the problem persists, contact National
Instruments.

One thing you could try is to set the IP RESET dip switch to on, and powercycling the cRIO. It should now have the IP 0.0.0.0

From the cRIO documentation:

Push the IP RESET switch to the ON position and reboot the chassis to
reset the IP address to 0.0.0.0. If the chassis is on your local subnet and
the IP RESET switch is in the ON position, the chassis appears in MAX
with IP address 0.0.0.0. You can configure a new IP address for the
chassis in MAX. Refer to the Resetting the Network Configuration of the
cRIO-FRC section for more information about resetting the IP address.
You also can push this switch to the ON position to unlock a chassis that
was previously locked in MAX.

After resetting the IP, try to ping the cRIO. If it responds, see if you can reimage it now.

The flash is continous.

We are trying the Reset IP suggestion…

After the IP Reset, I still get a continous blinking status light.

The ethernet port on the cRIO shows a green light on the 10V/100 side, and a solid orange light on the link side.

Both MAX and the Imaging Tool are unable to find the cRIO.

We are linked to the DS throught ETH2 all lights are green, which is linked to the cRIO through ETH1 with a green light on the DS and the lights on the cRIO described above.

Assuming you changed your team number while re-flashing, you should change your PC’s ip address to 10.24.93.6, and then try to ping 10.24.93.2. But that probably won’t matter now that you reset the IP address.

Thanks Eric, just tried that and the request timed out.

Anyone know if NI provides phone support on Saturdays? And if so, what is the number to call?

After the IP Reset, the documentation says the IP address is reset to the factory default. I assume this is 10.0.0.2?

OK, powered down the cRIO, powered it back up and now the orange link light on the cRIO ethernet port is blinking, so I am assuming we have communication taking place. That is a step in the right direction.

The Status light on the cRIO is still doing a slow continous blink.

Still wired from the computer to the DS to the cRIO. We temporarily tried using the crossover cable which also shows blinking orange link led.

a ping of 10.0.0.2 and a ping of 10.24.93.2 (with the computer ip set appropriately) are now returning Destination Host Unreachable

Try connecting the cRio and your PC to a router with DHCP enabled. Then check the router DHCP table and see if it got an IP address from DHCP. Its possible when it was reset it changed to DHCP configuration.

You could also connect your PC to the console serial port on the cRio. On about line 30 the IP address/subnet mask information will be displayed.

To do this you need to configure your cRio and PC as described in section 5.4.1 of the control system manual. If you’re looking for another terminal client for Windows PuTTY is what I’m using.

I’ll have to go out and find a NULL Modem serial cable to try that, but in the mean time…

I did get MAX to talk to the cRIO and was able to reset the IP address from 0.0.0.0 to 10.0.0.2.

I then was able to reflash the cRIO to a point. It got to the point where it was formatting, and then I received a UDP Unexpected Error 60 (sorry for the paraphrase, I didn’t get to write the text of the message).

So now I am back to trying to reset the IP again.

Oh yeah, when I turned off the firewall on the computer, MAX has a much easier time finding the cRIO

Back to reimaging… keep your fingers crossed…

OK, so twice now we get this message during the Format phase of the reimage process:

Unable to complete operation because of unexpected error 60: UDP Open in repFPC open connection.vi->SendRebootCommand.vi->RebootSystem.vi->SetTargetConfig.vi->SetIPAddress.vi->ConfigureUtility.lvlib:SetIPAddrAndGetInfo.vi->ConfigUtility.lvlib:ReImageTarget.vi->ConfigUtilty.lvlib.Dialog.vi

Any thoughts?

Error 60 is listed as network address is currently in use. Troubleshooting the network stuff isn’t my forte, but maybe that helps you.

Here are other procedural things that have helped other teams in the past.

Hook the cRIO directly to the PC, not through the DS, router or any sort of switch.

Turn off firewalls.

Turn off other enet cards, wifi cards, and other things that could confuse the routing as the cRIO IP moves around during the formatting process.

Try the formatting on anther computer (typically the failure was due to one of the previous two, and sometimes this is easier).

Pay careful attention to the DIP switches. Flipping them on to get the cRIO IP reset may then cause problems as you try to reimage.

Greg McKaskle

Hallelujah!

We have success.

Here is what we did.

  1. Do an IP Reset
    set the IP Reset DIP switch on the cRIO to ON
    push reset button on cRIO
    Set the IP Reset DIP switch on the cRIO to OFF

  2. Power off the cRIO, then Power it back on
    2a. Turn off your computer’s firewall

  3. Connect to computer to the cRIO using the crossover cable
    IP Address on computer 10.0.0.6 SubNet 255.0.0.0

  4. Run MAX (Measurement and Automation) to reset IP address (it is now 0.0.0.0) on the cRIO
    Tools\NI-Serial\Ethernet Device Configuration
    This should find the cRIO and show the MAC address

  5. Click Properties and set the IP Address to 10.0.0.2 SubNet 255.0.0.0

  6. In MAX, click Remote Systems and then F5 to refresh and you should now see the cRIO.

  7. Expand the cRIO under Remote Systems,click on Software, and click on Add/Remove software
    Reload the NI-RIO 2.4.1 software

At this point you should be able to return to the FRC cRIO Imaging Tool and flash the cRIO

  1. Turn your firewall back on

Oh yeah, and one final step that seemed to work, we switched computers. When we did that everything worked fine.

And they say water boarding is torture. Ha, it ain’t got nothing on flashing a cRIO

The “Error 60” makes it sound like your computer was set to the IP address the cRIO wanted to use. It might be worth checking the network configuration on the first computer to see if that was your problem.

We did so many things to get this working including:

IP Resets
Loading the base NI-RIO software
Formatting the cRIO in Safe Mode (scary thing to do)
Running directly to the cRIO with the crossover cable
All kinds of address configurations (0.0.0.0, 10.0.0.2, 10.24.93.2)
The computer was always 10.0.0.6 or 10.0.0.10 (or subbing 24.93 for the zeros)
Using a different computer
Turning off the firewall
Disabling all other network connections on the computer
Clearing all COM channels (paraphrased I think) using MAX

Switching to another computer finally seemed to clear things up and make things work.

I think that was all the things we tried. It took us at least 5 hours today to finally get it figured out.

I did my best to document what I think were the relevant steps in an earlier post, hopefully to help other teams.

I sure wish I knew this morning what I know now about loading the cRIO image, things would have gone much easier

Although NI doesn’t offer phone support on the weekends (and not till Jan. 5 for FRC) we do have a large number of knowledge base articles including topics such as network communication issues with the cRIO. I recommend searching for the operating instructions for the cRIO-9074 for more information.

It also doesn’t hurt to ask questions like this on ni.com/FIRST so we can identify the gaps in our documentation (this being one of them).

One thing I would like to point out that Formatting the cRIO shouldn’t be a scary thing to do. Its more robust than you think.