Camera Lag

FIRST General Forum
1

We have a 170 degree fish eye cam we got from amazon and are trying it out: https://www.amazon.com/gp/product/B01CVGBVM6/ref=ppx_yo_dt_b_asin_title_o02__o00_s00?ie=UTF8&psc=1

When we hooked up the camera to the Rio we noticed that the camera has very bad lag, to the point of being unusable. Amazon has various performance specs listed for the camera, and I imagine the issue is that the resolution is too high. Is there a way to set the resolution in code? We are using Java. Or can it be set on the smartdashboard?

1 Like
2

Since you’re using Java, you can go to the camera stream webpage at http://roborio-TEAM-frc.local:1181/, which will list the available modes and also provide interactive settings for things like exposure which can drive frame rate. The camera stream on that webpage doesn’t set resolution or compression and should show the “best possible” lag for the currently active stream. Setting either resolution or compression from the dashboard will dramatically increase latency as that requires the Rio decompress/recompress the stream. You can’t change the selected video mode via the webpage, but you can set it in code by using setVideoMode() on the VideoSource object returned by CameraServer.startAutomaticCapture().

1 Like
3

So would I use the VideoMode.PixelFormat in the setVideoMode to define my height, width and frame rate?

4

The PixelFormat just defines the pixel format from the camera. Use mjpeg pixel format to have the camera do the compression (otherwise the Rio will need to do it in software). The width, height, and FPS are all separate parameters from the pixel format. Cameras support a set of combinations of all 4 parameters.

5

You should theoretically be able to call CameraServer.getInstance().startAutomaticCapture().setResolution(...) to set the height and width of the camera stream.

We’re using a similar camera (180Âș) but a totally different way of streaming it (on a Jetson TX2 via g-streamer over UDP) and we’re getting decent results (~100ms latency) with 640x360 @ 30fps.

6

This does not work when I try to use it. I am connected to the Rio wirelessly. Do I need to be connected via USB instead?

7

Either should work. It’s possible mDNS isn’t resolving; you can try the Rio IP address instead of roborio-TEAM-frc.local (should be 10.TE.AM.2 on Ethernet or 172.22.11.2 on USB). It’s basically the same way you get to the RoboRIO dashboard, just on a different port number.

8

Ok, I got into the Rio with the ip, but I don’t see anything about a camera stream

9

I still haven’t found the camera stream via the Rios IP, but did put the code you suggested in Java. The camera is still very delayed. I put
camera1.setVideoMode(pixelFormat.kMJPEG, 640, 480, 30);

Could I be having issues because of using 2 cameras and using the kKeepOpen connection strategy for both?

10

You need to put in both the IP and the port number. E.g. http://172.22.11.2:1181/ when connected via USB. 1181 for the first camera, 1182 for the second, etc.

Using kKeepOpen doesn’t cause issues unless you run out of USB bandwidth, in which case you’d be getting console errors and the camera wouldn’t even connect.

11

Thanks I got the camera stream. The camera only supports 2 pixel formats, mjpeg and yuyv. 640*480 is the Lowest resolution for either type at 30fps.

When viewing the camera there it is still lagging if we pick it up and move it or wave our hand in front of it.

Are there other settings in there that should be tweaked?

1 Like
12

The one setting that might help FPS is making sure it’s manual exposure. However, the 640x480 lowest resolution poses a challenge. You’ll probably run into bandwidth constraints quickly even if you do get the native FPS higher. Resizing in software to reduce bandwidth will also slow things down. I would recommend a different camera (see my post here: What are teams doing about the Logitech C920's minimum resolution problem?). Either way, I would recommend moving it to a coprocessor (see my post here WPILIB CameraServer Input/Output error).

13

We’ve been having similar issues with a similar camera (https://www.amazon.com/180degree-Fisheye-Camera-usb-Android-Windows/dp/B00LQ854AG). The lowest res it supports in MJPEG is 320 x 240 @120fps but setting it to that spec weirdly results in a bit over 2.2mbps used and a choppy 10-15fps. Would this suggest that I’m doing something wrong with config or am I SOL?

I’m hoping that the rio can just cut out frames and not lag as 120fps is obviously not happening

[I saw your recommended fish-eye camera but we already bought 3 of these :frowning: ]

14

Thanks, changing the exposure didn’t help. I guess we will go back to the drawing board. Do you have a recommendation for a wide angle camera that we can run on the Rio?

15

On what platform? Rio or rPi? Definitely try rPi if you haven’t already.

1 Like
16

We are using FRC RaspberryPi and the results are very good - https://wpilib.screenstepslive.com/s/currentCS/m/85074/l/1027241-using-the-raspberry-pi-for-frc , you can get the image from here (https://github.com/wpilibsuite/FRCVision-pi-gen/releases) and burn it on a SD card. First time it will take probably a minute to come up, subsequent reboots will take 20 seconds. It will survive a robot shutdown because the SD card is in read-only mode. We’ve set the image at 160x120 with 25fps which consumes 0.9 to 1.1Mbps out the 4Mbps allowed this year during the game. We are using SmartDashboard (yes, I know it’s deprecated) but had really good experience with it. The drive team is very happy. It shows video in real time, no delays.

17

What kind of camera are you using?

18

We use the RaspberryPi camera V2.0 – https://www.amazon.com/Raspberry-Pi-Camera-Module-Megapixel/dp/B01ER2SKFS/ref=sr_1_4?keywords=raspberry+pi+camera&qid=1552482147&s=gateway&sr=8-4 . But you can use any camera, USB included. We’ve tried the FRC recomended one as well, Microsoft, and it worked fine. We chose to use the RaspberryPi camera, to have less wires. Its in the same case with the Pi. It’s like a module, just power and Ethernet.

19

We decided to try out a rPi on our testbed. We downloaded and flashed the image and connected via frcvision.local and had a camera image, but when we connect the Pi to the testbed we aren’t getting any images on the dashboard.

Edited to add: When we try the Pi on another computer it says the ethernet does not have a valid IP and wont connect to frcvision.local

Also, we cannot view a second camera on the frcvision.local:1182. We get a message saying the site can’t be reached.

20

To get image on the FRC SmartDashboard you need to have the RoboRio (running), FRC RPi and the radio (optional) in the same network. If you just want to see the image and configure various settings, go to http://frcvision.local or if that doesn’t work try to run arp -a in command prompt. Run that command several times until you see on the ‘Physical Address’ column an address starting with b8-27-eb-
 Next to it will be the IP address of your RPi.
Alternatively, you can download “Angry IP scanner” and scan your network/subnet.
After that, go to Vision Settings -> Open Stream, next to Camera rPi Camera 0. It should open another tab http://frcvision.local:1181.
Launch the FRC Driver Station then click on the ‘gear’ and select SmartDashboar from the Dashboard Type menu. Once it launched (it takes few seconds), in the SmartDashboard go to View, select Editable. Then View, Add, CameraServer Strream Viewer. In few more seconds you should see the image from FRC rPi. Drag a corner and make the image as big as you need. Right-click over the image ‘Properties’ and select ‘Degrees of rotation’ put a number in there if you need to : 0, 90, 180, 270. Hope this helps.