Except for the limelight, we have 2 cameras on our robot, both of which are connected to a Raspberry Pi. One of the cameras is an RPI camera, and it works just fine, without any delay or any other issue. The other camera is connected to the raspberry via USB, and it’s a bit problematic. I didn’t manage to find the exact model, but it’s very similar to this one. Initially, because the camera has a high resolution, the stream had a very low FPS. We solved this problem by resizing the image using OpenCV’s resize function. Despite that, we have another problem: the stream has a delay of about half a second, which isn’t ideal when driving at high speeds. We don’t want to connect the camera directly to the RoboRIO because it can affect our code’s run time.
Does anyone know how to solve this issue or what causes it?
Half a second is quite a long delay. Let’s look at the processing chain…
- Camera sensor (e.g. capture & readout)
- Camera compression to JPEG (if video mode is MJPEG)
- USB transmission of camera data
- Software JPEG decompression (if video mode is MJPEG)
- Software image resize
- Software JPEG compression
- Network transmission
If the camera has a lower resolution video mode (if you’re using WPILib’s CameraServer libraries, you can get a list of video modes via the camera server webpage), it will help multiple steps in this chain (2, 3, 4, 5, and depending on the camera’s internal design, maybe even 1). Also, if the lower resolution fits you in bandwidth constraints, you can completely eliminate steps 4-6.
Sometimes changing the video mode to YUYV instead of MJPEG can help–you eliminate 2 and 4, but 3 becomes higher cost.
For USB webcams, 1 can often be a major driver as many are optimized for image quality over latency. Tweaking camera settings such as auto-exposure and going to a fixed setting can substantially help latency. Again, if you’re using the CameraServer libraries, these can be tweaked via the webpage, and once you find settings that work, you can set those settings in code.
We tried lowering the resolution via the webpage, but for some reason it also cuts the image. Besides, thanks for the tip, we will try to change the parameters like you said to decrease the latency.
Note that USB cameras will always have more latency since they must encode the image over the usb protocol and have it decoded in software. With a mipi camera the signal is received in near real time by the Pi’s video chip which leads to very good latency.
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.