Chief Delphi - Fixing opencv lag

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)

- Programming (http://www.chiefdelphi.com/forums/forumdisplay.php?f=51)

- - Fixing opencv lag (http://www.chiefdelphi.com/forums/showthread.php?t=123390)

Re: Fixing opencv lag

Request a video stream from VAPIX with the "fps" HTTP parameter. Details are on page 12 of the VAPIX API doc -> http://www.axis.com/files/manuals/va...37_en_1307.pdf

And if it helps, there's an incomplete Node.js implementation of VAPIX that supports a video stream with the slowed down fps you need. https://github.com/gluxon/node-vapix

This answers your question, although I feel like the problem will still occur, just over a longer period of time with this change.

Edit: I would actually go with what virtuald said in his post for the solution. Sounds like a more complete way of handling things.

Re: Fixing opencv lag

Quote:

Originally Posted by billbo911 (Post 1315140)

I would definitely like to hear more about this approach.
I am no expert on the efficient use of OpenCV, and what you are proposing just might be the key to get rid of the minor delay we are seeing in our images. We can process 15-20 fps, but there is a small, but noticeable lag in our images.

We are using a USB camera and a PCDuino, but otherwise it is mostly the same.

Our implementation is available on my website if you want to look at it. Look for kwarqs2013.py.

Re: Fixing opencv lag

Quote:

Originally Posted by sst.thad (Post 1315111)

As for grabbing the image directly, openCV has no way of just opening a jpg over the network.

Not true. If you have ffmpeg support enabled (which it is by default in most opencv builds), you can retrieve the mjpeg stream directly from the camera via FFMPEG. Something similar to the following (error checking omitted):

Code:

vc = cv2.VideoCapture()

vc.open('http://%s/mjpg/video.mjpg' % camera_host)



h = vc.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH)

w = vc.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT)



capture_buffer = np.empty(shape=(h, w, 3), dtype=np.uint8)



while True:

    retval, img = vc.read(capture_buffer)

Re: Fixing opencv lag

Quote:

Originally Posted by sst.thad (Post 1314668)

The reason we are on a beaglebone instead of the dashboard is i have heard stories of FTA's requiring dashboards to get shut off, and we don't want to be stuck if that happens. So we want it to be onboard.

This.

In 2012, I had to tell a student that the vision software they worked on was not allowed because the FTA wouldn't let us use it. Then, we were stupid enough to try vision again in 2013, thinking that the "bandwidth limit" (which doesn't work) would prevent the FTA from banning the dashboard, but no, the FTA said we weren't allowed to use our vision stuff yet again, but this time, because a robot on the field had stopped working "because we sabotaged their network connection". What really happened? The other teams battery fell out. Not our problem, but we still couldn't use vision. Vision is not a guaranteed ability for robots. Nor are dashboards, or any amount of communication.

Re: Fixing opencv lag

Quote:

Originally Posted by magnets (Post 1315682)

... Vision is not a guaranteed ability for robots. Nor are dashboards, or any amount of communication.

I think the crux of this thread points out the error in your statement.

Yes, it is really a shame when a feature that has been pushed by FIRST is taken away, for any reason, and that feature is part of the major design a team has undertaken. The point of this thread, on the other hand, is how to avoid this possibility entirely.

Here is why this thread addresses this. If the vision processing is done by a separate on-board system, Beaglebone in this case and PCDuino in ours, it removes the wireless network entirely. All acquisition of images, processing and communication of target information remains on the robot. Honestly, for much less than $200, any team can do this! It just takes time to learn how, and the information is already readily available.

Re: Fixing opencv lag

Quote:

Originally Posted by virtuald (Post 1315655)

Code:

vc = cv2.VideoCapture()

vc.open('http://%s/mjpg/video.mjpg' % camera_host)



h = vc.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH)

w = vc.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT)



capture_buffer = np.empty(shape=(h, w, 3), dtype=np.uint8)



while True:

    retval, img = vc.read(capture_buffer)

Yeah I know I can recieve the MJPEG stream like this, and that is what I am doing. The camera still pushes frames into the stream at 30 FPS. If the vc.read does not occur faster then 30 times a second, The buffer will start to backlog and lag. In fact your code was what I started from, and I was doing exactly that. If you put at 50ms delay at the end of your processing function you will see that it starts to lag.

When people were talking about requesting images, they are talking about not opening the stream, but just directly requesting a single image, and working with that, so the camera does not push a buffer. Thats the part that does not work natively in openCV.