I've been following this thread for a while with quite a bit of interest.
Although our hardware is different, and our code is not nearly as complex, we to see a small, less than a quarter of a second, amount of lag. It is consistent and doesn't change.
Our hardware is a PCDuino v1, running OpenCV 2.3.1 and processing images from a Microsoft LifeCam over USB.
The question I keep coming back to is, is the camera free running and I am just grabbing a frame as needed, or are we actually requesting each frame one at a time. Here is some pseudo code that shows our approach. (Yash, is this how you are acquiring images?)
Code:
height = 240
width = 320
capture=cv.CaptureFromCAM(-1) ##run from camera
cv.SetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH, width)
cv.SetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT, height)
while True:
rawImage =cv.QueryFrame(capture)
Do all your processing......
cv.ShowImage('Camera', rawImage)
etc......
I would hope this either just grabs the latest frame from the camera, or requests just one frame. If it is allowing the processor to buffer all the unused frames, then we are SLOWING down the process considerably and need to take a better approach.
Currently we can acquire and process approx. 20 fps. Again, there appears to be about .25 sec. lag between acquisition and display of images.