1418 has two lifecam cameras so the operator can view the front and back of the robot, and at CHCMP we added a third lifecam camera so that they could reliably score in autonomous mode. We also added a USB hub and a memory stick so that we could log images and tune on them if necessary.
The first problem of course that we ran into was USB bandwidth and opening the third device failed. We were able to overcome this by switching the two viewing cameras to 160x120, and kept the vision camera at 320x240. So far so good, and after various bugs and tuning, in our last match it finally scored in autonomous (woo!).
However, in the first quarterfinal match, the USB devices decided to randomly swap, such that the vision camera was /dev/video0, and the other two were 1 and 2. This hadn’t happened at all prior to that point, and some quick testing in the pits showed it happening again. After another autonomous failure in the next match it was decided to let the Robobees score from the low bar, the vision camera and usb stick were removed, and the remaining two cameras had no issues for the rest of the day.
We’d like to have the vision camera back by the time CMP rolls around. I’m curious if anyone else has tried using this many devices on the roborio, or have had similar issues, and how have you resolved them?
I know that you can use udev rules to make sure that devices are assigned to static symlinks, but I haven’t looked into it too deeply yet. It appears that the lifecam devices do not have serial numbers associated with them, so it might be a bit harder to statically assign them. I’ve attached the output of lsusb with all devices attached.
We tried using a hub, and even before opening any cameras for capture we’d only see two: one from the hub and the one directly connected (order didn’t matter; current draw wasn’t the issue; we tried two different hubs). We just gave up on the idea of three USB cameras.
What hub did you use, and how exactly did you do it?
Well, if you look at lsusb.txt you’ll see exactly what we used in more detail than anyone would want. It’s some random usb hub that we’ve been using on our DS for a few years now. According to lsusb.txt, its a “Bus 001 Device 003: ID 05e3:0608 Genesys Logic, Inc. USB-2.0 4-Port HUB”.
The two USB ports are a camera and the hub. The other two cameras are plugged into the hub, and the usb stick is also plugged in.
We use mjpg-streamer to interact with the cameras. There are potential USB bandwidth issues to be mindful of – we were not able to open the camera devices until setting the resolution to 160x120, 160x120, and 320x240. I believe mjpg-streamer opens the devices in mjpg mode, as opposed to YUV (which an image processing framework such as NIVision or OpenCV would be more likely to open it in).
What did you use to determine that only two devices were detected? We used two different random hubs and had no issues detecting all cameras on either of them.
We also had three cameras, connect to an RPi2, and hit the same device-swapping issue at some point. One of our programming students came up with a workaround that involved hard-coding the serial numbers of each attached camera. I’ll see if he can post here or summarize the process for me to pass on.
Heads up from 836, we had some issues with the ms cameras too. Not sure why, but colors were changing on us in some cases. I plan to go back to the axis for Champs and investigate the usb cameras more over the summer.
As to the question of running multiple cameras, we wrote some beta code from scratch. I’ll send you some system calls that let you track cameras by usb port.
Hm, so it definitely looks like the lifecam devices don’t have serial numbers or unique identifiers of any kind on them. There are symlinks setup at /dev/v4l/by-id and /dev/v4l/by-path. As you might imagine, the by-id symlinks are useless (only one appears even with multiple devices plugged in), but the by-path symlinks appear to be related to the position on the USB hub, and the information I’ve found appears to indicate that it should be a stable path.
Initial experiments on a roborio with a hub indicate that the by-path positions are indeed stable across device insert/removals and reboots. I’m not 100% sure of the best way to test this, but I’ll add the information to mjpg-streamer’s documentation anyways.
We were using three cameras on our robot this year (Forward drive, intake alignment, and reverse). They were spare HP cams the school had lying around, all identical. They also didn’t have any identifying characteristics from the USB side of things and the RoboRIO would somewhat randomly assign them cam0 cam1 and cam2.
What we did was to use the SmartDashboard to set the camera names and the code that toggled the camera feeds used those values. Took an extra 15-30s once FMS came up on the field to switch through the cameras and make the settings changes, but it worked fine from there on.
Could put a queue in the image (dots on lens or card in field of vision with image or color you could detect with code) or put a relay on the USB wires and connect the cameras one at a time then check the relay output against the device names that appear.
That user seems to have a similar problem, namely that there is no serial number. The answers seem to confirm that the by-path identifier will be stable across reboots, so I think I’ll go with that until it can be shown that it won’t work.
Unfortunately, it looks like at the moment OpenCV won’t allow using devices other than /dev/videoX. I’ve patched OpenCV and pushed a pull request to the OpenCV repo, hopefully they’ll take it and it’ll work in 3.2.
By the way, NIVision does allow you to open the UVC camera in any mode that it enumerates. The WPILib VIs parse and compare the size and framerate. The DB in HW mode uses the JPEG mode while the SW uses YUV and compresses with the roboRIO. I don’t have a ton of experience with multiple cameras, but NIVision enumerates and attempts to keep the cameras in order and returns lots of device info to validate.
This, by the way is the acquisition lib, called IMAQdx, and not the processing lib.
To work around this problem, we manually resolve the symlink and parse the index from the resolved filename (/dev/videoX), which is then passed to OpenCV.