My team is trying to get the lowest latency possible and are wondering what codec are supported? It would also be helpful if you could point us in the right direction for getting the code setup.
I don’t know the answer, but I do know what the codec gurus will ask first: Is this for streaming video from a robot-mounted camera to the driver station through the 4Mb throttle? If not, then for what purpose?
it is for steaming video to the driver station
Maybe we should simplify and clarify:
What are the supported codecs for video in Shuffleboard?
The major factor in camera latency is usually not so much the codec as the camera and camera settings. Cheap USB cameras are generally designed for use as webcams, where lighting of the scene is often traded for FPS or latency. Turning off any sort of auto-exposure or auto-brightness settings in the camera generally helps a lot with both latency and frame rate. The standard CameraServer web page has interactive settings so you can experiment. Lower resolution also almost always helps speed things up, and reduces bandwidth too–use the lowest resolution you can possibly tolerate.
Codec wise, the standard CameraServer and dashboards (including Shuffleboard) only support MJPEG as a codec. The MJPEG compression can be provided either by the camera (which often has no option for setting compression ratio) or done by software. Given the typical default built-in compression settings for many USB cameras (high quality, so high data rates), you’ll almost certainly need to do software compression if you want to stream multiple cameras simultaneously. Assuming software compression is required, you do not want to set the camera to MJPEG mode, because this requires the image be decompressed in software and recompressed. I would strongly recommend you use a coprocessor such as a Raspberry Pi 3 (CD discussion here) to host your cameras if you are using multiple cameras, as it has much more CPU to work with than the RoboRio.
Outside of the standard software stack, some Ethernet and USB cameras support h.264, which will have the lowest bandwidth utilization, but I’m not sure how latency compares, and there is no standard dashboard support for h.264.
Documentation for setting up code for streaming from the RoboRIO can be found here: http://wpilib.screenstepslive.com/s/currentCS/m/vision
3 Likes