So our team is pretty certain were gonna use the new Raspberry Pi image for image processing this year. Our question is what is the best camera to use to operate during the sandstorm and efficiently process the images to locate the vision targets on the field.
Our first choice so far is probably the Microsoft Lifecam 3000, which we have plenty of laying around and have used in the past.
We are aware of the 7mb/s bandwidth limitation for data to the roborio, so whats the best combination of fps and resolution to use? Also, does the same limitation apply for the camera to the raspberry pi? For example could we get a 720p 30fps stream to process the images?
The Lifecam is a decent choice for vision processing.
You don’t need to worry about bandwidth limitations if all the vision processing is done on the raspberry pi (or elsewhere on the robot) and don’t send the stream to the driverstation. The bandwidth limit only applies to traffic between the robot and the driverstation.
Vision processing for FRC doesn’t need a high resolution - I believe the limelight has a 160x90 resolution. Low resolution images are faster to process than higher resolution ones, so try to use smaller images as often as possible.
Alright, thanks for the responses, I now understand that the bandwidth limit is only for the robot to the driverstation. Another quick question is, I just tried to plug in a Lifecam directly to the robot to mess around with resolutions and frame rates on the driverstation display, however it seems with the small quality adjustment box I cant get it to go above 7 or 8 fps, no matter where the slider is. Why is this? Can I manually lock it in the code?
That limit is probably from the camera. Cameras only support a fixed range of settings - resolutions paired with native framerate. So a camera could do 640x480 at 30fps, but at 1080p could only do 10fps. It’s just a limit of the camera hardware.
The stream page (which should be roborio-2783-frc.local:1181 for you) will show a table of these settings.
You can set the resolution output of the camera and a target framerate (which may not be hit if it’s above the camera’s supported framerate for that resolution) in your robot program.
I have not used this in official play, but the Playstation eye camera is hardware built for computer vision and boasts some great frame rates at usable resolutions (the bottleneck with this camera would probally be whatever device it’s piping these frames to). It is also cheap and plays well with linux.
From Wikipedia:
The PlayStation Eye is capable of capturing standard video with frame rates of 60 hertz at a 640×480 pixel resolution, and 120 hertz at 320×240 pixels,[1] which is “four times the resolution” and “two times the frame-rate” of the EyeToy, according to Sony.[10] Higher frame rate, up to 320×240@187 or 640×480@75 fps, can be selected by specific applications (Freetrack and Linuxtrack).
How exactly do you do either of these? I looked at the stream page and none of the settings there were resolution or framerate, and the class “CameraServer” in our java code has no methods to change them. Also, the lifecam’s specs are 720p at 30fps. We checked this by plugging it into my computer and using the camera app.
The CameraServer.startAutomaticCapture() function returns a UsbCamera object. That object has methods to set the resolution, FPS, and frame format, as well as other settings.
Nice, that worked perfectly. However, I can’t get it to set both a framerate and a resolution, if i use startAutomaticCapture twice it treats it as two streams. Also, I don’t see a method to set the compression, using the slider in the driver station is the only way I can get a reasonable MB/s.
Call ((MjpegServer) CameraServer.getServer()).setCompression() to force a particular JPEG quality, or setDefaultCompression() to set the JPEG quality to use if the dashboard doesn’t specify the quality. Note that if the camera is set to MJPEG mode, you’ll see higher CPU and latency compared to running the camera itself in uncompressed mode, since the software has to both decompress the image and recompress it.
Note: the getServer() method only works for the first camera; if you have multiple cameras you’ll need to use getServer(camera name). If you need this, I recommend you pair this with startAutomaticCapture(camera name) rather than using the built-in name generation.
Hmm, its telling me “MjpegServer cannot be resolved to a type”. Also, the getServer() isn’t static, so I assume you meant to use the CameraServer object instantiated before?
Ah I see, Ctrl+. wasn’t working for some reason. Now its telling me the method getCompression() isn’t visible, and when I look at the source WPILib code it shows it’s not public. Sorry for so many questions, I just assume this is gonna be faster than trying to figure all this out on my own.
