Problem with System Watchdog

[JSonntag]

I'm having a problem with the system watchdog timing out whenever I click on one of the tabs in the driver station window. We loaded the unedited default code into our test bed and the problem was still there, so as far as I can tell it has nothing to do with any of the changes we have made.

I also tried to edit the timeout length of the system watchdog by using the Watchdog Configure VI (not sure if it is a standard VI or not), but then the code wouldn't load.

Is anyone else having the same problem, and does anyone know of a fix?

EDIT: I disabled the user watchdog and the error still occurred, so I am almost certain it is a problem with the system watchdog.

[Luke Pike]

I am having this issue as well. Brand new project deployed without anything changed, but the watchdog times out as indicated on the driver station console. Anyone know how to fix this?

[Greg McKaskle]

If the message is about a User watchdog, then your approach is correct, but the system watchdog is totally independent and not affected using WPI APIs.

It would be really useful to know the user actions or other conditions that provoke the system watchdog printout. The driver station uses a tab change as a hook for saving off the configuration file. Producing this file is likely delaying a control packet long enough for the cRIO system watchdog to temporarily glitch.

The diagnostic print statement to catch these short watchdog glitches wasn't even in last year's system, and were only added in December, thus we don't have enough experience with when they may occur. Internal and beta testing found and corrected a few issues that produced noticeable and reproduceable glitches. If the tabs or another action triggers a system watchdog that interferes, there are many ways of dealing with them. Once again, at this point, please describe as fully as possible, the background and steps to reproduce.

Greg McKaskle

[Mark McLeod]

I get one or more messages:

"Watchdog Expiration:System 4, User 1"

whenever I switch from the Diagnostics tab to the Operation, Setup, or IO tabs and back again. This is with the default robot project running on the robot. The messages do not begin appearing until the robot is Enabled.

I also get the message periodically, maybe 1 every 10 min., if I just let it sit doing nothing with the Driver Station.

e.g.,
(on Operation tab)

Enable

(on Diagnostics tab)

Watchdog Expiration:System 5, User 1
Watchdog Expiration:System 4, User 1

(back to Operation tab)
(back to Diagnostics tab)

Watchdog Expiration:System 8, User 1
Watchdog Expiration:System 6, User 1

(back to Operation tab)
(back to Diagnostics tab)

Watchdog Expiration:System 12, User 1

and so on as long as I choose to switch back and forth.

[Greg McKaskle]

This is helpful. Is this with the LV development environment running on the Classmate too?

The WD trigger when clicking on a tab makes sense to me and isn't really concerning. I assume that teams will not be tabbing repeatedly while driving the robot. I will also try a few simple things to see if we can eliminate this glitch.

The once per ten minute glitch. Was this on a particular tab? The diagnostics tab in particular is pinging a handful of devices.

Finally, do you have a stop button attached? Do you have joysticks attached?

Greg McKaskle

[Mark McLeod]

I kept it simple.

Driver Station & default Dashboard w/update on the Classmate.
cRIO with v19 image.
Default LabVIEW Robot Project downloaded.

Not logged into Developer account.
Nothing else is running on the Classmate.

E-stop

USB Hub
- (1) joystick
- IO board (with & without)

Classmate direct cabled to cRIO, so no router or bridge in the loop.
P.S. Camera connected to cRIO, but failing to communicate (that's what I'm really working on-workshop tomorrow).

The WD while sitting occurs on the Diagnostics tab.
I'll try leaving it for awhile on the Operation tab before switching to the Diagnostic tab.

P.S. I'll run a few more repeats to see how repeatable it is.

[Greg McKaskle]

Thanks so much. This lines up with our test results. If you find anything else, especially anything that you think would affect a match or practice, please post the conditions. We will start looking at improvements next week.

Greg McKaskle

[Mark McLeod]

The initial WD occurs each time I switch between Enable and Disable just staying on the Operation screen, but I suppose that's to be expected and seems harmless.

Of course that means I'll see one or two WD messages whenever I switch back to the Diagnostics tab to check on things. The error messages need time stamps.

[Mark McLeod]

A team had a problem running the Tank Drive example and it appears to throw a WD error every few seconds.
Both using Run and building a real-time application.
I haven't looked into why that may be.

The default robot project also throws WD errors every few seconds when the camera is active and displaying on the dashboard AND the DS is on the Diagnostics tab. It doesn't happen when the camera is active, but the dashboard is not processing the video, and it doesn't happen when the camera is disabled. It doesn't happen (edit) as much (/edit) when the DS is left on the Operations tab either. It does still throw a lot of WDs when left alone for awhile on the Operation tab.

[kamocat]

I'm having some different watchdog issues.
I doubt they're related, but I thought I might keep them in the same thread.

When I enable the robot the RSL continues flashing rapidly, but the LED that usually is in synch with the RSL is solid, with a flicker every couple of seconds.
The Jaguars remain flashing amber.

I have the updates on the DS and the cRIO re-imaged. Default code, default configuration. (It made no difference whether I ran the code "live" in the developer account, or if I built and deployed it as a startup application, running the DS in the Driver account.)
However, I had a pre-built robot running fine at Kickoff, before the update.

[Mark McLeod]

Originally Posted by kamocat

When I enable the robot the RSL continues flashing rapidly, but the LED that usually is in synch with the RSL is solid, with a flicker every couple of seconds.

The RSL continuously flashes if you only wire to two of the terminals.
You need to add a jumper wire between the center and one end terminal on the RSL (La to Lb).

The Digital Sidecar LED is correct and the flickers might reflect the WDs as they occur.
The LED also does a long-on, short-off sequence when the robot is in Teleop-Enabled (different this year) that could be an alternate explanation of your particular blink sequence. Depends on the flickering.
The Jaguars blinking mean they aren't opened or aren't enabled, so it sounds like you might have a mixture of things going on there from code to no code.

[kamocat]

Y'know what?
I found the problem, and I feel pretty silly.
Before wiring, I've learned to color-code everything so I don't get it backwards. Most of the connectors on the control system have documentation as to which way they go, right on the case. However, since that's not easy to see in low-light conditions (ie through scratched safety glasses, under a mechanism, in the pit), I color-code.
Since almost everything was labeled, and I was familiar with the control system, I didn't check the documentation before coloring.
I colored the PWM headers on the digital sidecar backwards.

So yes, all is solved. This rookie team had their first drive practice today!

[GGCO]

After spending 7 hours working on the robot today, we found a solution to our problems.

First some background:
We heard that the system watchdog issue was cause by communications lag between the driver station and the robot. I know that wireless lag is often caused by interference and other devices using the same channel. We are working in an area that has many other Wi-Fi access points. The Linksys router we got in our kit is not dual band and only operates on the 2.4Ghz frequency. The instructions from FIRST said to set the router channel to Auto. This is bad practice in my opinion. I think it’s better to know your Wi-Fi environment and set a fixed channel accordingly.

How we fixed it:
We used a program called InSSIDer to scan for access points in our area. It gives a nice graph that shows which access points overlap each other. Channels 1, 6, and 11 are the best because they do not overlap. Channel 1 had the least access points so we set our router to use channel 1. After changing the channel we no longer experienced the watchdog issue.

The short version:
Try changing your router wireless router channel to 1, 6 or 11.
- Nate (the great) from Team 3357

We also disabled the watchdog and made sure to kill it in the autonomous() and operatorControl() functions. Hope that helps!

http://twitter.com/cometsrobotics

[Nemo]

Originally Posted by GGCO

We also disabled the watchdog and made sure to kill it in the autonomous() and operatorControl() functions. Hope that helps!

Is it legal to disable the watchdog?

[Greg McKaskle]

It is legal, but is it wise? The watchdog is there to indicate when the code controlling the robot is not keeping up with timing expectations. This is a bit like pulling the batteries from the smoke alarm because it keeps going off. It solves the immediate issue, but is it wise to leave them out?

Greg McKaskle