Need help minimizing live camera lag

Just to start off this thread, I’d like to say that I’m not a programmer, but I am a driver who would like to help.

For previous games, it has been useful to have a camera for various reasons (most notably gear placement and acquisition), but whenever my team has set up a live feed, it has had ~1 second of lag, and that is very hard to dive with. I have asked the programming lead what the main cause of lag was, and they haven’t given me a clear answer.

I just was wanting to know what the best methods for reducing camera lag you’ve found are. Thanks.

Consider what is happening to create the video stream:

  1. The camera captures an image of the field
  2. That image is transfered to a buffer for processing
  3. The image buffer is compressed/encoded for the video stream
  4. The encoded image is transmitted to the driver station
  5. The image is decoded
  6. The image is displayed

The faster these steps complete the lower the latency. Generally speaking the easiest thing you can do to reduce latency is to reduce the size of the captured image. Reducing the resolution of the image will make everything go faster, especially if you can reduce the size of the captured image – reducing the resolution of the captured image.

Find the lowest natively supported resolution of your camera that is usable. Next (perhaps counter intuitively) reduce your frames per second to ~15 – this is to keep your wifi connection happy, remember what works great in your shop, might still not work well in the much more crowded networking environment of a competition. These two steps will get you there most of the time. If you have an AXIS camera or other similar network streaming camera that can do all of this in hardware (and can be adjusted to as listed above), that will generally be faster and simpler to setup, but will also cost more than a cheap usb setup.

Just to add in a few bits.

The default dashboard has a drop-down for setting camera values. It lets you set the size, frame rate, and compression settings and view the results. The Compression setting is sorta special in that when the slider is pushed all the way to the end - past zero, it means that the camera will be set to do its own compression. You don’t get to pick the compression level, but you take this load off the roboRIO and let the camera do it in its processing chip. This may not be as nice for image processing, but it is often the lowest latency for driving.

Greg McKaskle

You might want to consider if there are alternatives to using a camera. In the past, our driver found it distracting to have to look away from the field to look at the screen to see the camera view, even when it was elevated to near eye level. This year, we used a focused flashlight mounted on the centerline of our Gear pocket. The driver used it to line up with the Feeding Station as he drove up to it and was noticeably faster and smoother at lining up than many other teams. He also used it to line up to hang the Gear with similar efficiency.

Be careful on this.

Last years game required you to score right in front of you, so this method works well, but if next years game requires you to score in front of your opponent you’ll need to make sure your targeting device (powerful flashlight) does not interfere (blind) with the opposing drivers.

Not to mention that we’ve had a few cases recently where a target was completely obscured from the driver’s point of view (2017: gear loading for teams on boiler side; 2016: low goals behind certain defenses; 2015: almost anything after a few stacks were on the near platform). Unless you’re feeding the target info to a closed-loop navigation system, a fast low-res picture beats a laggy high-res picture for most FRC tasks. We’ve used multiple cameras on several robots recently; one or two set to stream at low resolution, low latency to the driver, and one or two at a high resolution to guild auto-targeting. The spotlight, high-speed camera, and driver control can also serve a similar function to the high-res camera.

My recommendation to consider alternatives was to get the OP to think about alternatives to what he was doing. No solution is perfect and no solution works for every game. I think we have both been around long enough to know this.