Go to Post Many people read ChiefDelphi, not just members of FRC teams. - JaneYoung [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 18-02-2014, 23:17
AndyB871 AndyB871 is offline
Registered User
FRC #0871
 
Join Date: Jan 2012
Location: East Islip
Posts: 37
AndyB871 has a spectacular aura aboutAndyB871 has a spectacular aura aboutAndyB871 has a spectacular aura about
System Watchdog Errors disabling robot

Hey everyone, We just bagged and tagged a short while ago, but we had one weird problem that we couldn't seem to fix.

We kept getting Watchdog Expiration: System N User 0

Where N increments by 1 every 5 seconds or so.

We're using labview

At first I figured we must have done something dumb in our code accidentally. ( We discovered opening multiple PWMs or Relays twice causes things to go haywire. It would be nice if WPIlib was a little more forgiving of that, but whatever)

I had my guys disabling just about everything in our code with the Diagram Disable Structure. Didn't change a thing, so we rebooted our laptop, started Labview and created a new default robot project, compiled it and deployed it without touching any code at all. Same issue, we get incrementing system watchdog errors, that disables all our outputs for a fraction of a second, every 5 seconds.

That lead us to assume we borked our cRIO setup, so we re-ran the imaging tool and flashed the cRIO again. Same problem with the default code, We get System watchdog errors, and disabling of our output.

Next we figured maybe the hardware was dorked, maybe we had a bad cRIO or a bad sidecar or 9402 or something. To make our isolation easier we flashed last years robot with this years image, and deployed the default program. Same exact behavior. To make sure we didn't accidentally break last years robot, we re-flashed it with last years image, and deployed last years code. Errors went away.

We backed up there and re flashed last years robot with this years crio, and the error came back. At this point we decided to try switching to a different laptop, didn't change a darn thing, still watchdog errors and short output disables.

We also tried to direct connect the cRIO to the laptop to take the switch out of the loop, which also had no effect. Still watchdog errors.

Eventually we gave up and I had my guys work in some error tolerance, so our vision code didn't lose it's marbles when our LED ring clicks off once every 5 seconds.

Our code runs, but man is it annoying for every output to halt every 5 seconds for a half second or so. I've completely run out of ideas as to what we can try.

To sum up where we gave up:
1) Used last years cRIO flashed with this years image
-- Last years cRIO running last years code has no issues
2) Using a freshly generated default robot project from this years code
3) Direct cRIO to laptop connection via crossover

Anyone have any ideas? I'm really concerned that the FMS will end up disabling our robot for the entire match when the system watchdog expires...

Thanks in advance

-Andy & Team 871
  #2   Spotlight this post!  
Unread 18-02-2014, 23:59
Mark McLeod's Avatar
Mark McLeod Mark McLeod is offline
Just Itinerant
AKA: Hey dad...Father...MARK
FRC #0358 (Robotic Eagles)
Team Role: Engineer
 
Join Date: Mar 2003
Rookie Year: 2002
Location: Hauppauge, Long Island, NY
Posts: 8,906
Mark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond repute
Re: System Watchdog Errors disabling robot

The (good?) news is that FMS won't disable your robot for throwing a watchdog.
The bad news of course is that your robot will be pretty inoperable if your drivetrain keeps getting cut off every 5 seconds.

Do you have a second computer with a fresh install of LabVIEW to try?
I'd re-build default code on a second computer and try running from there to the robot.
That would identify LabVIEW installation corruption. It'd also remove any laptop/LabVIEW issues as a possible source of the trouble.
__________________
"Rationality is our distinguishing characteristic - it's what sets us apart from the beasts." - Aristotle

Last edited by Mark McLeod : 19-02-2014 at 00:02.
  #3   Spotlight this post!  
Unread 19-02-2014, 00:24
AndyB871 AndyB871 is offline
Registered User
FRC #0871
 
Join Date: Jan 2012
Location: East Islip
Posts: 37
AndyB871 has a spectacular aura aboutAndyB871 has a spectacular aura aboutAndyB871 has a spectacular aura about
Re: System Watchdog Errors disabling robot

Ah, yeah We didn't try that. I'll wipe labView from our backup laptop, reinstall this years and try to create a clean project. If that works then I guess We'll have to migrate our code into the new project...

I'll let you know what comes of it. Thanks!
  #4   Spotlight this post!  
Unread 19-02-2014, 03:58
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: System Watchdog Errors disabling robot

The Charts tab on the DS will tell you quite a bit about what your robot is doing.
It also has a button to launch the Log File Viewer so that you can review previous match and practice data. If the log file view doesn't make sense, post an image of it.

The System Watchdog happens when the robot doesn't receive or process valid info from the driver station for 100ms. It will reenable as soon as it gets valid data.

If you use one of the basic robot templates to compare against, I assume the issue will go away. You can then compare and see that it is code and not something else.

Greg McKaskle
  #5   Spotlight this post!  
Unread 19-02-2014, 17:17
AndyB871 AndyB871 is offline
Registered User
FRC #0871
 
Join Date: Jan 2012
Location: East Islip
Posts: 37
AndyB871 has a spectacular aura aboutAndyB871 has a spectacular aura aboutAndyB871 has a spectacular aura about
Re: System Watchdog Errors disabling robot

That's what we did. First thing We checked was to see if the CPU was pinned, but everything's happy, and we were using the default robot template generated by labview.

Unfortunately we don't have access to the school for the rest of this week so I can't do much more until monday. I'll take a look at the log files, I didn't think to check them out. Would there be some indication of packets being delayed? I didn't see any packet loss or drops....

Edit: Maybe we should try the default robot dashboard. We do have some complicated vision stuff that finds & selects targets from the dashboard then UDP's the selected target over to the robot. On the other hand, we were running with the dashboard off entirely. Something to look at at least.
  #6   Spotlight this post!  
Unread 19-02-2014, 21:36
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: System Watchdog Errors disabling robot

If the CPU is not loaded and you are getting system watchdog, that means the packets are not arriving. Please do look at the DB and its CPU usage. The DS is elevated in priority, but with enough overload, I suppose the DS could fail to send packets in time.

The Charts tab will show lost packets per period of time, and it will show the latency of packets that do arrive.

Greg McKaskle
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 02:44.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi