Need Help Making Driver Station Video Feed Faster

Technical Videography and Photography
, , , ,
1

I am currently using FFmpeg on a Raspberry Pi 4 Model B using an ArduCam UC-844 Rev. B as the camera. We do not need any audio and I don’t care about the quality of the video. All we need is for the stream to be as fast as possible. The video from the camera is being streamed to the driver station via FFmpeg and being picked up on the driver station via FFplay. Right now we are getting a delay that we wish could go away or be significantly shortened. These are the commands we are using.

  • Raspberry Pi: ffmpeg -i /dev/video0 -c:v libx264 -crf 45 -maxrate 1M -bufsize 1.2M -preset ultrafast -tune zerolatency -filter:v fps=30 -f mpegts -omit_video_pes_length 0 udp://10.2.33.5:554

  • Driver Station: ffplay -fflags nobuffer -flags low_delay -probesize 32 -analyzeduration 0 -f mpegts -vf setpts=0 udp://10.2.33.5:554

2

If you’re using a pi4, have you considered running PV? That has support for treating video, and you might find that to be a drop in solution for this problem.

1 Like
3

Yes, we have considered using Photon Vision but we want that to be a last resort, we are willing to upgrade to a Raspberry Pi 5 if that will lower the latency of the stream. We could also use an Odroid if that is faster as well. We also have an Nvidia Nano but that is a talk for World’s and not the Smoky Regional. Just a talk, my software mentor doesn’t want to use the Nano or PhotonVision unless we have to.

4

Is there a reason your mentor doesn’t want to use PV? It should be supported on a pi4, especially if all you’re using it for is streaming.

It also might work better if you try a different codec. I don’t have a ton of experience in that realm, but I’m sure you could find something pretty easily over google.

5

Reduce the FPS, bitrate, and/or resolution.

If you use WPILibPi or PhotonVision then you can reduce these directly from a driver dashboard’s CameraView widget.

1 Like
6

I was using WPILibPi at the beginning but it wasn’t doing what we wanted it to do so we switched to FFmpeg. If you have a better tutorial than what’s in the documentation can you send that to me?

7

What specifically were you having issues with? As long as you use a modern dashboard like Elastic, QFRCDashboard etc. you shouldn’t have any issues displaying it.

8

PV is definitely lightweight enough and can run on a Pi 4. It works really well for us

We have been using the new radios though, for practice, and noticed a significant jump in maximum fps at least (latency has never been an issue) on the steam.

1 Like
9

Also ensure that you’re on the 6 ghz band.

1 Like
10

Everyone here has convinced him to consider PhotonVision for Worlds, no longer a last resort. Gonna start testing it after Smoky Mountains.

4 Likes
11

We couldn’t figure out how to configure the camera settings on WPILibPi

12

H.264 (libx264) is currently the best solution I have found, unless we use a co-processor with a supported Nvidia GPU, then Nvidia’s version of H.264 (h264_nvenc) is the best

13

I suspect that avoiding re-encoding of the camera’s stream will be less latent then re-encoding to any other codec on any type of processor.

14

What effects will that have on the bandwidth though? And how can we get that to work on a Raspberry Pi? We don’t want the roboRio to have anything to do with the video feed

15

You’ll be at Smoky? MARS will be too… Let us know, we could possibly help you out with this at the event if you have any long-running issues by then. See you there!

16

See you then

17

We don’t have any problems with the video feed, it’s just a bit delayed, but we’ll see you at Smoky

1 Like
18

I figured out the problem, it was the ArduCam, I tried with a Microsoft camera, PlayStation 3 camera, and LimeLight camera and they were all faster than the ArduCam. Thank you for all the suggestions though.

1 Like
19

Correction, I was testing with a PlayStation 2 NEW EYETOY 5702 SOLUTION(V8.2) camera, not a LimeLight, but the EYETOY is what we are going to be switching to for faster video speeds