Problems with cRIO and driving while competing

[naldarn]

Hello,

This weekend, our FIRST team had their event, and it could have gone better.

We are using labview, and with our programming, we were getting poor driving performance and loss of communication. We received help from many people at the event, and still were not able to load or shoot the ball.

To troubleshoot, we swapped out the radio, cRIO, digital sidecar, switched to a different laptop from the classmate, changed network cables, disconnected the digital sidecar, ran tethered in the pits, all to no avail.

We then reimaged the cRIO, and loaded the base code for an arcade drive robot with no edits at all.

We still have lost packets that can precede latency issues, latency issues followed by packet loss, all of which can happen while the robot is not being controlled but enabled and tethered. This is the watchdog error that shows:

ERROR <Code< -44061 occurred at "Left and Right Motors" in the VI path: Robot Main.vi FRC: The loop that contains RobotDrive is not running fast enough. This error can occur if the loop contains too much code, or if one or more other loops are starving the RobotDrive loop.

During the event, we were able to drive, but not very reliably. We were told by someone to press F1 to bring the control back faster, and that seemed to work a little.

Our team is very disappointed and would like to be more competitive in our next event in 2 weeks.

Any help would be appreciated, and more info will be provided as needed.

[Kevin Phan]

My team encouters these errors every match. The only thing that helps reduce these errors is to have most of our code in periodic tasks, with one fast loop(10ms) and one slow loop (100ms) for our various tasks.

[Joe Ross]

What is the Driver Station CPU usage? It is displayed on the charts tab of the driver station (or you can open Task Manager). It is not in the DS log, however.

I helped one team this week who's netbook had unexpectedly high CPU usage. They were running around 50% usage when idle. Adding the DS and dashboard brought the CPU usage to 100%. Closing the dashboard got it to 70% and their lost packets and latency decreased dramatically.

Can you post a representative Driver Station log? See http://wpilib.screenstepslive.com/s/...og-file-viewer

[Greg McKaskle]

Please post your log files or an image of the log file. I'm curious to see your CPU usage and other errors. To Joe's point, what laptop was used for the DS. Did you happen to notice its CPU usage?

As I've said many times, reimaging is something I rarely recommend. If the controller doesn't boot, reimage it. Otherwise, the time is probably better spent debugging and experimenting with other element.

Greg McKaskle

[naldarn]

We were using a Classmate that came with the KOP, as well as a Dell XPS laptop.

Here is the base logs and our original code, and the base code for just a driving base.

Original code is the 2014 Robot Project while catapult.zip
Base code is the 2014 Robot Project drive only.zip

[Greg McKaskle]

The Thursday logs show a frequent stream of errors coming from teleop trying to read a Digital Input refnum that doesn't exist. This caused the cRIO CPU to be pegged near 100%.

On Friday, that was resolved, and the errors were just timeouts on RobotDrive during auto.

The 5:30 log on Friday has a one second disabled gap during teleop, but it isn't clear what caused that. The comms were clean and CPU was low.

The three qualification matches on Saturday look clean.

Saturday afternoon around 2:00, there were some tests in the pits where the auto timeouts started happening again

I haven't had a chance to look at the code yet, but the logs do not show communications drops, cRIO CPU usage or errors to be the cause.

Greg McKaskle

[Greg McKaskle]

Looking at the code, the first project looks exactly like the template code. Did you have issues driving with the default code?

The other project has one point that looks funny to me. The catapult motor is being controlled in two parallel loops. If the logic is consistent, then this shouldn't be a problem, but it clearly allows for funky interactions if the logic or drivers are doing things the programmers didn't expect.

I searched to see if anyone else is updating the X or Y globals. I didn't see any in the code that was posted. So I don't think this is causing any issues. Another way to do this is simply to read the joystick in the periodic loop.

So at this point, I don't see anything obvious. Can you point out periods in the log file or with specific mechanisms that were problematic? Perhaps I'm not looking at the correct logs?

Greg McKaskle

[naldarn]

The problem we were having while driving is lack of control of the robot, especially driving not responding to command.

Here is one log when we switched to another laptop. The packet loss is what we were told was causing the lack of response for steering.

[Greg McKaskle]

That is very little packet loss.

You only attached the .dslog file and not the .dsevent file, so I can't look at errors, but your packet loss never tops 5 lost packets over 1/2 second intervals.

I've seen better, but I've seen much worse. I see no evidence that your robot dropped communications. I see no evidence that your robot outputs were shutdown because of packet loss.

Greg McKaskle

[naldarn]

Here is both of the logs for our replacement laptop. The original classmate logs are attached above. If the logs are not showing any issues, why is it that the robot would not respond to the controls for seconds at time, making us a sitting duck on the field?

We ended up just driving and trying to defend, where the robot had been designed to shoot as well as pass.

[Alan Anderson]

As Joe Ross said, the logs don't record the laptop's CPU load. If it's bogged down trying to do too much, there's a good chance the Driver Station will get behind on its communication tasks. If that happens, you'll get control delays, with the lag time very likely increasing as the match goes on.

Are you using a custom Dashboard?

[naldarn]

Default dashboard. With most of the work being done on the cRIO, how much is the laptop actually doing? It is the default dashboard, shouldn't the laptop be able to do this without issue? If it cannot, why are we getting inferior equipment?

[Alan Anderson]

You aren't trying to log on to both the Developer and Driver accounts simultaneously, are you?

[naldarn]

No, only driver, and log out, not quick switch when changing between.

Sorry if the previous post sounded nasty, just really frustrated. Thanks for all the suggestions.

[Greg McKaskle]

I looked at the latest zip. You have one error related to reading the color enabled on the camera at boot up time. Other than that, it is a clean log.

From the battery voltage, I can see that your robot didn't move during auto, and there are a good number of two and three second flat spots where your robot was sitting still.

As Joe mentioned, I'd look at the laptop CPU usage and I'd also look to verify that the ribbon cable was firmly seated at both ends. I'd make sure that the power to the digital breakout board wasn't going in and out or that I didn't have a short to it. I'd also make sure the digital module is fully inserted and the clips are holding it in place.

Greg McKaskle