Low Latency Camera Discussion

[technogeek6952]

Because of the large obstructions in our field of view, we (team 4068) are thinking about using a camera to aid the drivers in tasks such as lining up the gear with the peg.

We've done a fair bit with cameras and driver visibility in the past, but it's always had a significant lag time (at least one or more seconds) between when something would move and the driver would see it on the camera. This made it hard for our drivers to use, and they mostly ignored the camera, in favor of lining things up by eye from the driver station.

I'd like to know if anyone has any insights as to how to get a low-latency (ideally only a couple hundred milliseconds) camera stream to the drivers.

We're currently thinking of skipping all of the code FIRST uses for their default camera stream, and just send raw data from the camera (compressed to save bandwidth) to the driver station over one of the UDP ports they give us for our own use. We would make a companion application on the driver station to quickly draw the frames sent over UDP. We're not sure if that would do enough to reduce latency to be worth our time, though.

Has anyone done any analysis of what contributes to latency on the camera stream? Would removing the WPILib code remove much latency? Or is more time added by some other factor? What success have other teams had in removing latency in the past?

PS: We us Java, in case that's important.