Camera Lag


Thanks, we did finally get the images to show. I think our issue was that we hadn’t named the 2nd camera, or that we hadn’t changed the team number.

We are having other issues though.

  1. We can’t connect to the frcvision.local wirelessly when we connect to our testbed via the radio. Is that normal?

  2. We have java code to allow for cameras to switch on a button push, which worked when the cameras are connected to the Rio, but doesn’t seem to work since we added the Pi. Any idea what to do to fix that? This is essentially the code we used: 2 Cameras in Java

  3. Also, do you know what settings for the LifeCam works well?

If we can’t get the camera switching to work, is it a big deal to stream them both simultaneously since we are using a pi now?

  1. You should be able to. That is the desired way to connect and verify connectivity/performance. Verify if the laptop can ping frcvision.local or the IP address of the FRC rPi. FRC rPi should be connected to farthest ethernet port from the power jack and RoboRio should be connected on the closest ethernet port to the power jack (on the radio). RoboRio will get an IP address from your radio - and FRC rPi will get any IP address between up to
  2. Sorry I can’t help, not a programmer. @Peter_Johnson can probably help, he’s the expert in this thread and he developed the FRC rPi Vision on RaspberryPi.
  3. in Vision Settings (on FRC rPi) we are using 160x120 with 25fps.
    Streaming two cameras simultaneously will consume bandwidth. This year we have 4Mbps. Each camera (setup with 160x120x25fps) will consume about 0.9 to 1.1 Mbps - in upper left corner of the streamed image you will see FPS and Mbps values. Two of them will consume 2 x 1.1 = 2.2 Mbps leaving 1.8Mbps for the robot. It should be enough, however, test your system thoroughly before you go to competitions. 1.8Mbps for robot communications is very tight, in my opinion.

I just realised you’re the “FRC rPi Vision” expert! Congratulations, it’s a great software and it works great for our team. Thank you for creating the RaspberryPi images.


Sometimes mDNS is unreliable. I recommend setting static IP addresses for everything on the robot when using coprocessors. The settings are accessible via each device’s webpage (RoboRio and FRCVision). Set the RoboRIO to static 10.TE.AM.2 and the rPi to static 10.TE.AM.11, netmask and gateway 10.TE.AM.1 for both. You can leave the DS as DHCP or set it to static 10.TE.AM.5.

The FRCVision image can’t access joystick data (as that only goes to the Rio), but it has built-in support for switched cameras controlled via NetworkTables. You can add a switched camera on the Camera Settings tab of the webdash and then use robot code to set the NT value, and the Pi monitors the NT value and does the switch. See my post here for some example code: WPILib FRCVision Raspberry Pi image 2019.3.1 Update

1 Like

Thanks for your help. We got the Pi to connect wirelessly. I looked at your example for the Network Tables, but I have a couple of questions. We have never done network tables before so I apologize if these are ridiculous questions.

In your example, I see that you used “PiSwitch” as the NT key, but what did you do for the name? Also, if we have 2 cameras, would we have 2 different NT keys, or would we use the same one?

Also, you previously helped us with the code to get switched cameras working on the Rio. Would we need to remove the: server = CameraServer.getInstance().addSwitchedCamera(“switched camera”); code?


On the Pi vision settings tab, you should have two USB cameras and one switched camera. You can name each of them whatever you like. The name is what appears in the dashboard list of cameras. On the switched camera settings you need to set a NT key—in my example this was set to “/PiSwitch”.

On the robot side, you don’t need to use CameraServer at all, just NT. The NT /PiSwitch value can be set to either a string (the camera name of one of the usb cameras) or a numeric index (0 for the first camera, 1 for the second).


Thanks! That all worked.


One last question, how does the Pi determine which camera is 0 and which is 1? Is it determined by the USB port they are plugged into?


No, it’s the order they’re listed in the Vision Settings tab. To avoid having cameras potentially swap places, make sure you’re using one of the “alternate” path options (/dev/by-id or /dev/by-path) for both cameras instead of /dev/video0 and /dev/video1. The mapping of /dev/video0 and /dev/video1 to the physical ports/cameras isn’t guaranteed to be stable. This can be easily changed with the button to the right of the path in the vision settings.

1 Like

Our switched cameras has stopped working. It was working, but now when we press the button nothing happens. We tried setting the alternate path on the Pi, and saving it, but even with that the cameras switch positions.
Along with that, the cameras no longer switch. Any idea why it would suddenly stop working?

Also, in our code is it better to use setString or setDouble for the camera select?


Is the rPi successfully connecting to the Rio through NetworkTables? Do the individual camera streams work? SetDouble may be a bit more reliable, as it avoids potential typos in the string name?

Which alternate path did you use? If the cameras are the same make/model you should use by-path rather than by-id.


It is connecting successfully I assume. We can pull the stream from the web dashboard for the pi. We had the switching working the other day, but turned the testbed on today and it didn’t work. We hadn’t changed anything between times.

I’ll check the names and use the path if we are not.

Also, we are still experiencing lag on the camera even with the pi. We have the settings at 240x160 (I think) 25fps and it still lags.


How much lag are you seeing? You could try switching the cameras to YUYV mode and using software compression, since the Pi has a much faster processor than the RoboRio, that conceivably could be faster than the camera’s hardware compression. Also make sure you have turned off auto-exposure and auto brightness etc; all of those things can slow down frame rate / increase latency.


I’m not sure how to quantify the amount of lag. It seems to be about a 1-2 second delay when it lags it doesn’t always lag, but when it does it’s noticable.

In the pi dashboard, how do you put the network key in? Is it “/PiSwitch” or just /PiSwitch without the " ?

Edited to add: we are testing with Microsoft lifeCams.


Don’t include the quotes when typing in the text box.