camera / vision system too laggy

[martin417]

DISCLAIMER: I am a mechanical engineer, and a mechanical mentor for the team. If I say something stupid about programming / computers, chalk it up to that.

Since we got the control system early, we decided to play around with programming. We mounted the system on a practice chassis and tried many different things. Using Labview turned out to be very easy once you get the hang of it. What we did find though, was that the vision system (at least through Labview) was very laggy. It was at least 1/2 to 1 second behind reality at all times. As is, this would be useless for controlling a launcher. Has anyone found a way around this limitation?

[EricVanWyk]

Try using the dashboard to watch the video stream, rather than a VI's front panel or a probe.

[Greg Marra]

Originally Posted by martin417

What we did find though, was that the vision system (at least through Labview) was very laggy. It was at least 1/2 to 1 second behind reality at all times. As is, this would be useless for controlling a launcher. Has anyone found a way around this limitation?

What you're seeing is delayed between getting from the robot to the laptop you're programming from. The actual robot controller "sees" what you see before it bothers sending the picture over to the laptop.

[martin417]

Originally Posted by EricVanWyk

Try using the dashboard to watch the video stream, rather than a VI's front panel or a probe.

Even without watching the video stream, the reaction of the robot to a change in target position was about 1/2 to 1 second behind, so I don't think it has anything to do with how you monitor the stream.

One thing I'm not sure about is the burn vs. debug mode. We started with the basic robot project for learning purposes, and there is a known bug that will not allow you to build this project to the cRIO, but only run it in "debug" mode. I was under the impression that the only difference is persistence through a power cycle, but I am not certain. Can this affect the speed at which it runs?

[StevenB]

Originally Posted by martin417

Even without watching the video stream, the reaction of the robot to a change in target position was about 1/2 to 1 second behind, so I don't think it has anything to do with how you monitor the stream.

My only experience comes with hooking the camera directly up to the computer and using the provided software. My sister was goofing around with it and pointed it at the screen displaying the image. This created an infinite "picture within a picture" effect, with each sub-picture lagging the one enclosing it. This video sort of demonstrates it. When an object was waved in front of the camera, it moved sequentially through the frames. As the object moved into the smaller frames, the frame rate increased. When the whole frame (real life) changed, it took more than a second before we saw the change on the screen, but when the fifth sub-picture changed, it took a much shorter time. I'm sorry I don't have a video of it at the moment, as it is very interesting and instructive.

My theory is that this is entirely a problem with the camera's data rate, which is dependent on how much it can compress the frame. If the camera is watching a mostly stationary scene (like most webcams and security cameras are designed to do), it will perform ok. If the camera moves much at all, expect a painfully low frame rate.

[TDohse]

Originally Posted by martin417

I was under the impression that the only difference is persistence through a power cycle, but I am not certain. Can this affect the speed at which it runs?

Yes, it can. Each front panel you have open on your PC will be in constant communication with the RT code to update controls & indicators. In some cases, having multiple front panels open with many {controls | indicators} while running in debug can result in a noticeable slow down.

One option is to build & deploy the code as an application, alternatively you can close unneeded front panels.

[Joe Ross]

Originally Posted by martin417

One thing I'm not sure about is the burn vs. debug mode. We started with the basic robot project for learning purposes, and there is a known bug that will not allow you to build this project to the cRIO, but only run it in "debug" mode. I was under the impression that the only difference is persistence through a power cycle, but I am not certain. Can this affect the speed at which it runs?

LabVIEW update 2 should not have the problem of building the basic robot project. It also optimizes some of the vision processing.

You can further optimize it by decreasing the resolution of the camera and/or decimating the camera image prior to processing.

Running in debug mode does slow down the processor, and if you have many front panels open it can do so significantly.

[Greg McKaskle]

Camera lag will always be there, but you can make it much much smaller than what you are describing. If you open and run the vision demos, you will see lag when you are viewing the entire image via the debugging tools, but it isn't 1/2 sec. If you switch it to show the mask the lag will be far smaller.

A better measure of the lag and aiming ability is to open the servo camera example. Since it is already controlling something, you can turn off most or all of the UI displays and get a better feel for the ability for it to respond. Again, the lag is still there, it always will be, but it is pretty small at that point.

You don't mention much about the setup that was producing lots of lag. Just a guess, but 640x480 images, especially if decoded and displayed on the cRIO will have pretty substantial lag.

Greg McKaskle