MJPG lag OpenCV

[Greg McKaskle]

The lag is caused by your camera running faster than the downstream pieces. Over time the buffers in the camera, network, and in your code store more and more images that they have arrived, but not been processed, until it either reaches a steady state, or until the buffers start to overflow.

I'd suggest measuring how long it takes to process an image. If it is 25ms, then your camera can theoretically run at 40fps to max out the system. But in practice it is wise to leave a bit of a safety margin. It is common to shoot for 80% utilization. This means that instead of 1/.025, you would set the camera to .8/.025, or 32 fps. This will hopefully allow the processing to catch up and be able to wait just a bit for new frames to come in.

Be sure to measure the time of the processing with realistic content. Some image processing algorithm steps are a constant amount of work. For 1,000,000 pixels, they will always perform N computation. Many are dependent on what they find in those pixels. So some datasets will not take 25ms, but may take 30 or 40.

Greg McKaskle