SOFTWARE PROBLEM!!!! PLEASE HELP!!!

[RedHeadRobotics]

We are having trouble with our controls. We have tried many things. The software will lag and respond in two or more seconds. This problem is sporatic and we think that is has been fixed, but then it returns. We have kept the same setup and it still changes. We have checked the battery, made sure that the battery we used was new, freshly charged. Our software team even had us move where the wireless bridge was to see if it received code better (probably knew this wouldn't work because it does the same while tethered.)

Even more confusing is that the mechanisms delay more than the drive train. It is not just the pneumatic devices, motors also delay more than our drivetrain does even though they are connected in the same manner.

If Anyone has any ideas or suggestions, it would be great. Thank you!

[EthanMiller]

Depending on the programming language, there's different things that could cause it. What language are you using?

Also, look to see if you have any loops that may take a couple of seconds to complete.

[birkejf]

We are using LabView.

[Mark McLeod]

If you care to zip up the entire project and post it we can help diagnose code delays. Time is getting short...

Any messages on the Driver station Diagnostic screen?
Any lengthy "Watchdog not Feed" statuses showing up in the lower left corner box?
Any oddly behaving status lights?
Does the RSL have a consistent long-on/short-off blink sequence or is it disrupted?

[bronxbomber92]

Are you constantly running the camera and processing the images for ellipses. If you don't have this running quickly, that could drop your the speed pretty drastically, I suppose.

[nick1023]

Perhaps you have a "wait" somewhere?

If there is a "wait" anywhere in your teleop take it out. Each time the loop executes you will have to wait for that time to pass.

frcmastery.com has a state machine video that shows how to use timers properly inside a loop using a shift register and a tick count.

[lscime]

There are a couple of things that could be causing this. If you are getting "watchdogs" in the box at the bottom left corner of the driver station (where it usually says enabled or disabled) it is possible that you are running a loop (say for reading sensors) that has no time delay in it. This will cause the loop to run flat-out and result in lag-times. Also there can be pretty serious lag if you are running non-deployed code (i.e. you are running the code on your programming laptop and not the on cRIO alone). Try building your project and permanently deploying your code to the cRIO and see if the lag persists.

More information would be helpful in debugging your problem.

-Luke

[jawebste]

Hello from 2145
Our team is having the same problem with the delay between when we push a button/move an axis and when the motor moves. In our code we are trying to control 7 motors and two servos could there be a limit to what the digital side car and/or the cRIO can handle? We have built our project and deployed the code but the delay continues.

I am posting our code with the hope that some one can help us. This code is pre-update before last Saturday or Sunday, so right now we are redoing it on from the Robot Project Sample in the updated Labview, but I predict that we will still have a lag if some one can help us fix it on the old code we can probably use it to fix our new code.

When we enable teleop on the driver station it flashes between Teleop enabled and Watchdog Not Feed could the Watchdog have something to do with the lag?

[lscime]

The fluctuation between watchdog and telop enabled will certainly cause lag. After briefly looking at your code I don't see anything that is likely to cause a watchdog error. Although it appears that you have forgotten to close all of your devices (ie motors) in the "finish.vi." This may cause issues.

The best way to debug this kind of issue is to go back a couple of steps. For instance, did you start noticing the lag after writing a particular piece of code?

PS - the cRIO is incredibly powerful, it is highly unlikely you are taxing its processor (remember, it is designed to have all 8 slots filled with stuff).

-Luke

[pSYeNCe]

Team 2010 is also experiencing similar problems. We're getting a 1/2 to 1 second delay from moving the controls to a response. We don't have any loops other than the teleop.vi While loop that encompasses everything, so I think I'll try to time delay that a little bit and see what happens.

[lscime]

I'm sorry for any confusion. The telop loop does not need to be delayed as it requires a lot of processor time (in other words, it won't be running flat out). The vision loop similarly should be fine without a time delay. Loops that are reading sensor values etc. don't take much processor time and so do need to be delayed. Check the "periodic tasks.vi" for such loops. Also, make sure you have fully updated LabVIEW, the cRIO, and your Driver Station.

-Luke

[Robototes2412]

We use java and get a similar problem, try a reboot of the robot

[birkejf]

Thanks for everyone's advice. It was a mis-named variable and we were getting excessive watchdog error messages! We now know what to look for next time.

[flameout]

I'd like to add that we are having the same problem. It has sprung up since the last set of updates (LabVIEW and DS.)

It is vision related, since unplugging the camera will prevent the problem. It appears to be taking around 200 milliseconds per frame, and I think it is starving the teleop task.

This applies to all our code, including our "clean" project.

Does anyone have a clue? Thank you for any help.

[Mark McLeod]

Team 2010 is also experiencing similar problems. We're getting a 1/2 to 1 second delay from moving the controls to a response. We don't have any loops other than the teleop.vi While loop that encompasses everything, so I think I'll try to time delay that a little bit and see what happens.

There shouldn't be a teleop While loop unless it is less than 1/50 sec.

Hello from 2145

Several of the Hook Motor Set Speed's are without a speed or motor reference. Of the three case statements for the Hook motor look at the False case on top, the False case in the middle, and the True case at the bottom. The pink wire needs to be wired across and the bottom speed needs a constant value.
I don't see anything else that would account for the delay you're seeing.

[pSYeNCe]

Oops, I misspoke. We don't have a while loop in the teleop code so I really don't know what's causing this. I've looked at the DS and we get a "Watchdog not fed" for a splitsecond every half second, so it must be something timed.

[Greg McKaskle]

It sounds like there is something in teleop not allowing it to finish on time. This could be a delay, time consuming computation, lots of I/O, or a combination.

Greg McKaskle

[Ether]

It sounds like there is something in teleop not allowing it to finish on time. This could be a delay, time consuming computation, lots of I/O, or a combination.

pSYeNCe,

Greg explains one possible way to monitor your Teleop throughput margin in this post:

http://www.chiefdelphi.com/forums/showthread.php?p=924284

[pSYeNCe]

Absolutely. You can read the millisecond clock simply by dropping the Tick Count block. It is used for timing the Vision code and elsewhere in the framework. You can read it at the beginning and end of Teleop, or you can read it each entry, etc. You can also make use of the Match Info.Elapsed Seconds which has millisecond resolution. It comes from the same clock, and you can use it at the beginning of Teleop to identify a missed packet. I've made notes about how each of the choices listed would impact the robot if they were the chosen method.


That's from the link you recommended, and I just don't understand what he means on how to identify the missed packet.
As a side note, we're going to reload the original code (after all the batteries recharge, which somehow all became dead at the same time :mad: ).

Edit: We're also going to try disabling Vision to see if that's causing the problem.

[Ether]

That's from the link you recommended, and I just don't understand what he means on how to identify the missed packet.


Read the clock at the beginning of teleop and compare it to the clock value from the previous iteration of teleop. If the difference is 40 ms (or 60ms or 80ms etc) instead of 20ms then you've run out of realtime and missed a packet (or 2 or 3 etc).

Or, read the clock at the beginning and end of teleop and look at the difference to see what your throughput margin (or overrun) is.

~

[theanabanana]

We figured out how to optimize it by changing the image taken to grayscale (U8). This reduced the cycle time for the while loop in the Robot Main from 300 to about 180ms. This is better, but still not very reasonable. 100ms at least would be a better goal.

It also significantly lowered the cycle time of the while loop when we changed the image resolution to 160x120. However, making the images this size is not beneficial at all because the ellipses are hard to read at that size.

Sooo while our robot's electrical mounting is taking place (therefore I cannot test/debug the cycle time), I'm studying the Vision VI in the bottom Flat Sequence Structure. There is a comment in inside its Camera: Update Status VI, written by the original programmers of the default code (not to be confused with the standard .lvlib library code) . It says:


Quote:
This subVI is reentrant so that it will work correctly if used in multpile locations. You may want to change the VI Property to disable reentrancy if debugging.

I don't believe the Camera: Update Status VI needs to be implemented anywhere else in the code really.

So, I'm planning on taking the shift register out and feed the "Enable Vision" global into the simple case structures inside the original shift register. It's one less step to go through, and could reduce the time spent in each cycle. I'm not sure if it will be a significant amount of time, but we can still shave the cycle time down in as many places as possible to reach our less-than-ideal but realistic goal of 100ms.

Ana

[theanabanana]

So the changes I said I was going to do: I implemented them and it didn't lower the cycle time at all. :(

I thought I'd show you how I got the waveform chart on my front panel of my Robot Main.vi. Well: rt-click, Graphs, waveform graph. Then go to the block diagram and implement this in the main while loop that includes your robot mode and whatnot and implement the code in the jpeg I've attached.

Still working on it,
Ana

[whcirobotics]

Unread 02-19-2010, 12:37 PM
Mark McLeod's Avatar [Mark McLeod]
Mark McLeod Mark McLeod is offline
Itinerant Programmer
AKA: Hey dad...Father...MARK
FRC #0358 (Robotic Eagles)
Team Role: Mentor

Join Date: Mar 2003
Rookie Year: 2002
Location: Hauppauge, Long Island, NY
Posts: 3,376
Mark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond reputeMark McLeod has a reputation beyond repute
Re: SOFTWARE PROBLEM!!!! PLEASE HELP!!!
If you care to zip up the entire project and post it we can help diagnose code delays. Time is getting short...

Any messages on the Driver station Diagnostic screen?
Any lengthy "Watchdog not Feed" statuses showing up in the lower left corner box?
Any oddly behaving status lights?
Does the RSL have a consistent long-on/short-off blink sequence or is it disrupted?

We keep getting "watchdog not feed" messege cntinousaly when we enable the robot. Also out switches on the kicker, are in the wrong place in the software... only in the dashboard i dont know why. AND IT IS VEEEEEEEEEERY laggy. please help

[Mark McLeod]

We keep getting "watchdog not feed" messege cntinousaly when we enable the robot. Also out switches on the kicker, are in the wrong place in the software... only in the dashboard i dont know why. AND IT IS VEEEEEEEEEERY laggy. please help
You have some actions, probably in Teleop, that are taking way too long.
Most likely While loops.

We'd have to see the code to help much more.

[Greg McKaskle]

I debugged a lagged robot at a scrimmage yesterday. We didn't get to the bottom of all the problems, but here is one easy thing to look for. The Watchdog function to delay and feed had a 1000 wired to it. The student read it as delaying 1000 ms. But of course the units to that function aren't ms, but s. So that explained why their solenoid didn't seem to run more than once.

Anyway, now is a great time of the season to do code reviews. Go screen by screen explaining the code to another person. Add comments to the code where you find yourself guessing, guessing again, and correcting yourself because you cannot even explain your own code without comments. Add comments when you find that what you are saying verbally is way more than what is in the code and you are explaining how this piece on another screen does something that this code depends on, etc.

Finally, if you are the other person with the code being read to them, ask lots of questions such as what units is that parameter or output in? How does that value get initialized? Let's go look at the configuration of that I/O and see if it matches how you are reading it. If what they said doesn't seem to match the logic or math in the code, ask them to explain it again with less hand-waving.

It is surprising how many issues you will find just by being forced to proofread the code for what is there versus what you meant to put there. With some practice, you'll find that you hardly need the other person there. You can learn to read the code with new eyes as if explaining it to another person, and you'll find yourself asking some of those same questions and fixing bugs without having to go through hours of robot observation.

Greg McKaskle

[Andy Bradley]

We are having lag problems too, and I think its related to the latest set of updates. In our crio log file (while connected to the robot, select the diagnostics tab on the driver station screen, and then click the "View Log File..." button), the robot is recording up to 12 error messages per second associated with the cypress module (which we aren't using at all). This particular problem is being discussed in this thread, http://www.chiefdelphi.com/forums/showthread.php?p=925272#post925272
but there is no solution found yet.

[pSYeNCe]

Good news, I disabled vision and the lag stopped entirely. I looked through the code and on the front panel of Begin.vi, the res is 160x120, but it is set to 320x240 on Robot Main.vi. I wonder if that was causing the problem (trying to convert the res slows it down or something). Unfortunately I won't have any time to test more, since we have to focus on more core aspects in these last few hours :O.

[Greg McKaskle]

Anytime you see values on a subVI panel such as vision, those are the defaults. In this case, you'll see that the resolution is being set to the Main VI's values via a global, and perhaps via a parameter.

There was no conversion going on, but the camera does by default use the medium image size. You can leave vision on and use the smaller image size, or you may want to play with the image compression. I believe that if you set it to around 20%, the performance will change.

Greg McKaskle

[pSYeNCe]

I believe that if you set it to around 20%, the performance will change.

Greg McKaskle

Okay, we'll have to play around with that. Thanks to everyone for all the help!:)