Edit: Issue seems to be related to camera motion and auto-exposure slowing down the camera. You can watch frame rates change just by moving the camera. Even turning off autoexposure didn’t fix it, just made it better… I think it’s just a cheap webcam processor dropping frames when behind.
Hello all, I didn’t know if I should make this a github issue or a topic here, so I started here.
As with many others I have been playing with AprilTags, and I think I came to a roadblock that should be looked at. I think the CameraServer.CsSink.grabFrame() function has an issue, and I think it’s affecting anything using CameraServer (both WpiLib and PhotonVision).
I was suspicious because while we are limited to ~7FPS with AprilTags reading, we were also only using ~40% CPU and why we can’t make data processing more parallel.
So, I made a VERY simple vision loop (posted below) in Python; just grabbed a frame and did nothing. Then I took the time between loops and logged it. This is a plot of the FPS, notice it groups really bad at 30, 15, and 7.5 FPS.
# some WpiLib vision initialization before csInst = CameraServer.getInstance() vs = NetworkTables.getTable('Vision') # Get the Vision NetworkTable # start cameras for config in cameraConfigs: cam = startCamera(config) cvSink = csInst.getVideo(camera=cam) #Begin Image processing loop frame = np.zeros(shape=(640, 480, 3), dtype=np.uint8) while(True): t1 = time.perf_counter()#Used for testing performance FPS = (1/(t1-t0))#Used for testing performance vs.putNumber("FPS", FPS) ret, frame = cvSink.grabFrame(frame) t0 = t1
I’m definitely not CPU bound, running the WpiLibPi v2022.1.1 image on a Pi 4b, 2GB of RAM, LifeCam HD-3000 camera, configured in WpiLibPi as a 640x480, 30 FPS cam. No other streams were running on the Pi (verified by the Network traffic from Pi). Data collected on my laptop using the Robot Simulation (only to run a NetworkTables server and using WpiLib logging, used AdvantageScope to plot the data)