Microsoft USB camera lagging, low resolution, and low fps

Hello chiefdelphi

At our last competition, our camera was giving us about 0 fps, so we had to turn the resolution down all the way to our lowest setting. Then we got about 2 fps. We also had a lot of camera lag, so we couldn’t put the hatch panel on the front of the cargo ship reliably.

We tried to replicate this situation at home, but we couldn’t because we didn’t have an fms. We get about 8 fps on the second lowest setting, but there’s still a bit of camera lag. Is there any way we can fix this? This is the camera we’re using: https://tinyurl.com/y27bnhwl

Can you share code?

UsbCamera camera = CameraServer.getInstance().startAutomaticCapture();

That’s it.

What Smart Dashboard are you using to view the camera?

Also we’re running into the same problem. From looking at a few posts from Chief Delphi and Reddit, I found a few teams saying they skipped the Smart Dashboard completely and switched to using a browser to view the camera stream.

To view the camera stream through a web browser:

  1. Turn on your robot and connect your driver station laptop to the wifi.
  2. Turn on the driver station software and wait until you have communication to the robot. Leave the robot disabled.
  3. Browse to the following Webpage (TE would be the first 2 digits of your team number, AM would be the last 2 digits of your team number. So if your team was 1122, it would be 10.11.22.2): http://10.TE.AM.2:1181/?action=stream
  4. Now you can enable and try driving with that camera.

The main pitfall is you’ll have to refresh the page every time someone downloads new code or reboots the robot. It appears to be more responsive, haven’t figured out how to get the FPS to print via this method yet. I’m crossing my fingers that this takes care of it.

Also Add this to your code after the start Automatic Capture line.

camera.SetConnectionStrategy(VideoSource.ConnectionStrategy.kConnectionKeepOpen);

That tells the Camera Server to keep the camera stream open as much as possible.

I would caution against using a browser, although if nothing else works, of course go with it. My testing shows that Chrome will buffer one frame, while Shuffleboard and SmartDashboard do not. So Chrome has 30 msec or more lag. With your 8 FPS, that would really be an additional 125ms.

The lag and FPS are probably closely related. 8 FPS will lead to ~250ms of lag. Solve the frame rate and the lag will be much better. You should be able to get better than 8 FPS in the shop, so try to debug that first.

This fixed our teams low fps issues, first also do the same to resolution(i.e. set it to -1 or just don’t set it at all). After this make sure to set the fps otherwise your fps will be minimized. Finally don’t run smart dashboard and shuffle board at the same time and try to have as little stuff open as you can. This is what we did to fix our lag issues however we didn’t see anything as significant as 0 fps(we were getting about 6 now we get 24) so this might not fix it. If you get a chance take a look at your bandwith and try testing different subsystems and look for one that may be eating it up or something.
also @Peter_Johnson would probably be your best bet.

Definitely try the above recommendations first. The biggest hit to FPS is typically from setting things on the dashboard such that you end up with software compression instead of using the frame directly from the camera–setting the compression level to other than -1 or the resolution different than the camera resolution will cause this to happen.

Once you’ve connected to the web dashboard, try setting everything manual you can (e.g. manual exposure instead of automatic). These settings can be made permanent with code on the VideoSource object returned by startAutomaticCapture(), but the web dashboard lets you experiment with settings.

Also, it’s possible the camera itself is malfunctioning. It’s worth trying a different one if you have one (even if it’s the same model).