Paper: Measuring Camera Display Latency

CD-Media: Papers
1

camera_latency.pdf (1.1 MB)

With driving by camera a “thing” this year, the latency (lag) of a camera image streamed to the Driver’s Station is important. The attached paper presents a method of measuring the end-to-end latency using an LED and a couple of photoresistors. One interesting fact that came out of our work is that using Google Chrome as a display is not recommended.

4 Likes
2

Super slick work, this is an excellent contribution!!!

Just out of question, were you planning to benchmark different web browsers against each other? I’m curious if firefox/IE/Edge have any appreciable differences.

3

Nice work! There are several potential sources of latency changing when you switch from OpenCV/USB to Shuffleboard/CSCore: cscore camera connection, cscore server, wifi network, cscore client, and shuffleboard display. If you compile cscore for your Linux desktop, it might be enlightening to do the following tests to try to further isolate the source of the latency:

  1. local cscore capture, display with OpenCV (tests cscore capture latency)
  2. local cscore server and shuffleboard client (tests server+shuffleboard latency, removes network component)
  3. remote cscore server, cscore HttpClient, OpenCV display (isolates network component)
4

It would also be interesting to benchmark different video modes. For example, is it better to have the camera in yuyv and have the coprocessor compress to mjpeg, or is it better to have the camera do the compression and the coprocessor just pass the data.

1 Like
5

This is cool work. I thought the finding about Chrome was interesting.

Note that the competition field network doesn’t make any sort of latency guarantee. An interesting experiment would to run your experiments under these conditions:

  1. Wifi on 5GHz, select a channel with no traffic for your test network (NET-1)
  2. Set up a second access point on the same channel, on a separate ssid (NET-2)
  3. On NET-2, connect a client and send 4mbps traffic to simulate another robot with camera
  4. Rerun your experiment on NET-1
  5. Increase the background traffic on NET-2 to 8, 12, 16, 20mbps to simulate more robots and run the experiment again.
6

I’d be interested to see these numbers as well - Mostly because I’m curious if the extra latency of the FMS puts the differences in Camera, resolution, framerate, etc. into the noise range.

7

All cool suggestions. I did a few of the simple ones (eg local cscore and display), and don’t remember any real improvements, but was not careful enough to trust the numbers. Unfortunately, more investigation by me will probably have to wait until after competition season.

8

If you look at the FMS white paper, there’s very little between the field AP and your DS to slow packets. Nearly all of the latency will be related to the wireless interface.

1 Like
9