Mid-Match hang due to smart dashboard?

[DjScribbles]

In a few matches in our week one district, we observed a few seemingly random losses of control on the field, mid-match. The FTA attributed this to an issue with smart dashboard (which we were using), and informed me that similar issues were seen by a team in California in week 1, and that a fix should be expected in the coming weeks.

We disabled smart dashboard, and removed several unused sensors from code; and didn't see the issue again in our remaining ~9 matches.

In the match where it was decided that the issue must lie with Smart-Dashboard, the DS log was free of errors until robot control was lost (at which point motor watchdogs began barking), and we saw behavior that I would attribute to the main thread being stuck or crashed. No losses of communication/robot code were observed during the match.

I'm curious if this issue has been observed elsewhere, and if there are any fixes planned to resolve it. I believe this issue is separate from the one identified in the other "Serious Bug" thread because it is not occurring when a connection is severed.

[Alan Anderson]

There is indeed a patch for the C++ Network Tables issues. It is currently being validated, and ought to be available on an as-needed basis from the CSAs for teams competing this weekend. Unless something significantly bad is discovered, I would expect it to be released as an official update next week.

[Jefferson]

I'm not sure if it would help with your situation, but we write a LOT of data to the dashboard. The more we write and the longer we do it, the more issues we see with SmartDashboard.
Therefore, we link as many of the Dashboard writes as we can to a joystick button press, so the robot is only writing for a brief period. The majority of the writes are used for debug.
Now if you're using NetworkTables to send back information to the robot (like vision processing) you can recompile SmartDashboard to use the available UDP port instead of the TCP port it uses now. This seemed to help us in 2012. We haven't attempted driver's station vision processing since. We decided keeping the camera off the network was a priority.

[DjScribbles]

Originally Posted by Jefferson

I'm not sure if it would help with your situation, but we write a LOT of data to the dashboard. The more we write and the longer we do it, the more issues we see with SmartDashboard.

Actually, we have a handful of values that are pushed each teleop periodic which could easily be moved. Thanks for the tip.

[virtuald]

See this thread. You can download an official update using the link on that thread.

[DjScribbles]

Originally Posted by virtuald

See this thread. You can download an official update using the link on that thread.

That thread is about an issue that is different from what I've described; it could be related, but it didn't really seem to be.

From that thread:

The TL;DR is basically this: if you are connected via SmartDashboard, and you disconnect the connected computer's wireless while the robot is writing a value to the SmartDashboard, the robot may hang until the write times out.. which can be a few minutes.

[virtuald]

Originally Posted by DjScribbles

That thread is about an issue that is different from what I've described; it could be related, but it didn't really seem to be.

The SmartDashboard could become unsynchronized for any number of reasons, and cause writes to fail. If they do fail, then without the patch you will lose control of your robot. SmartDashboard uses a separate communications channel than the DS comms, so just because the DS didn't lose any packets doesn't mean SmartDashboard packets were not lost.

The fact that removing SmartDashboard from your code solved the problem indicates a high probability that you are affected by the issue, even if the cause isn't exactly the same.

[MrRoboSteve]

Based on the description of the issue, I agree with the other posters that you should take the patch.

Can you post DS Log Viewer snips of matches where the failure occurred? Very low CPU usage would point to this as the root cause.

[DjScribbles]

To clarify, I fully intend to apply the patch.

I simply saw a different set of symptoms than those described in the other thread; and I wanted to ensure that if the problem I've encountered is a separate issue, it is treated as such (which is why I acknowledged the thread in my OP).

I will try to get our DS logs from that match to post tonight.

[Joe Ross]

Did you have any more problems after applying the patch?

[DjScribbles]

We ran smart dashboard throughout the Lansing district event (14 matches) without removing any of our "excessive" PutNumber readouts, and did not encounter a problem on the field.

So I am fairly confident the patch fixed our issue, I'll report back if I see any sign of the issue again.