Just returned from Central Illinois Regional, and wanted to post something about camera issues we had on the field.
We have two Logitech C920 USB cameras on our robot, connected to a Raspberry Pi 2. We have two instances of mjpg_streamer running on the Pi, sending HTTP feeds to SmartDashboard on the DS for the driver to use. We’re using this SmartDashboard extension to display the feeds.
This worked great in the shop before CIR, and when tethered in the pits and practice field. When we hit the competition field the feeds didn’t work.
After a lot of help from FTA Nick Luther, we set the RPi, Roborio and DS laptop to static IPs. This also seemed to work great off the field, but failed when connected to FMS.
One of our students went through the FMS whitepaper again and saw that the port range we were using for those feeds (5800 and 5801) are TCP/UDP only. He tried changing them to ports 80 and 443, which are dedicated to HTTP traffic. The very next match the feeds worked flawlessly on the field. The FTA was surprised to learn this, since HTTP just uses TCP under the hood anyway.
Trouble is, it only worked for that one match. The matches that followed the feeds refused to display in SmartDashboard again. We did not resolve this at CIR, so we plan to dig into it this week before MWR. One thing I did notice was SD has a “use mDNS” option that’s on by default. That’s one thing we can try next time we’re on FMS. We’re also going to try changing mjpg_streamer to use UDB instead of HTTP.
Just wanted to post this in case anyone had similar setup/experiences/advice to offer. It’s frustrating not being able to reproduce issues like this off the field.
Just an FYI, we use a very similar setup except we use two MS LifeCam 3000 HDs.
We had a few “panic” moments on the field where we though the feeds weren’t working, but for some reason it took a long time for the SmartDashboard extension to pick up and start displaying the feed after the robot was powered on on the field.
We would stand there staring at two pink screens until just seconds before the match started, and the feeds would come alive. They always did without fail, but it was nerve-wracking.
We kept the RPi set up on DHCP, with a hostname of “raspberrypi” - which is how we addressed it in SmartDashboard. We broadcast over ports 5800 and 5801. FYI, HTTP traffic is actually broadcast over TCP, so that restriction is not the issue.
I think it is just a time-out/retry delay with the SmartDashboard extension being a tad too long.
Just before our matches, if we were really nervous, we would open a browser on our DS, and navigate to the MJPG Streamer web interface (http://raspberrypi:5800 and http://raspberrypi:5801). This showed the streams working perfectly fine, even though the SmartDahsboard extension hadn’t kicked in yet.
Thanks. Our streams weren’t just slow to start, though, they flat-out didn’t appear for the entire match, except that one.
Maybe the switch to ports 80 and 443 prior to the match where it worked were just coincidental. I did baffle me and the TSA why that would fix anything, given HTTP still uses TCP.
We were using a pretty similar setup at CIR too - dual mjpg streamers running to smartdashboard. Never saw any issues related to coms on the field. Some crucial differences: We stuck with the default ports of 5800 and 5801, did not see any issues. We addressed our rio with the roboRIO-1736-FRC.local address. Our streamers were hosted on the RIO itself, not a coprocessor. Additionally, we had made some modifications to that same smartdashboard plugin to allow it to stream from either feed and only display one, depending on which direction we were driving. Finally, we had generic wide-angle cameras, not the Logitech ones you had.
I’ll try to keep you posted if we run into any issues - next field we’ll be on will be at Midwest. The non-reproducible problems are always the worst to try to solve…
We had issues at Buckeye, never did get it working on the field with FMS. It worked great at home wireless, tethered, tethered during calibration time on the field, but never during an actual match.
We are running an Axis IP camera, and using Labview. I’m not at all a programmer, but have some understanding. The connection is camera via ethernet to the radio, ethernet from the radio to roborio. We do still get the live feed during the match, but no vision tracking.
I also believe the FTA did something with static IP’s with us. I would really like to get this working, so if anyone has any suggestions, it would be greatly appreciated.
The other obvious cause would be the video stream is trying to use too much bandwidth and the field drops it.
Images compress differently depending on how complicated the picture is, so if the camera happens to point at an easily compressed picture the bandwidth use can be small, but if the field image is too complicated and it isn’t easily compressed then it takes up a lot more bandwidth. I’ve seen camera bandwidth vary on the field by 2 or 3 Mb/s depending on what the camera was looking at.
As a test for the next time you are on the field, temporarily cut your image stream way down to 320x240 / 10fps / 50% compression. And in the Pit measure your bandwidth using the standard Windows Task Manager->Performance tab.
Those are exactly the settings we use for our two streams. They were reasonable off the field and in the one match our feeds worked we didn’t notice any loss of controls response or other symptoms of hitting the bandwidth limit.
I would say we looked in the log, but on a related note it looked to us (and the FTA) there’s a bug in the Driver Station logging this year that causes the bandwidth value to always be zero. Can anyone confirm? We’re on the required DS 16.0.2.
There won’t be any loss of controls evident due to the camera exceeding bandwidth limitations.
Command packets are prioritized over all other traffic through the robot radio.
If the camera traffic gets dropped by the field then the bandwidth log won’t show any of that, it’ll just be the control packet and other non-camera user traffic.
We also switched to static IPs at the event. We used mjpgstreamer on the Nvidia TK1 on port 5801. We had a small Chrome window connecting to the static IP of the Jetson. Something like HTTP://10.41.43.11:5801/&stream
That worked good for us.
If you still have trouble at Midwest, I’ll help you out.
Is this “filtering” of packet types for all traffic through the radio, Lan traffic only, or WiFi traffic only?
Will it depend on which port on the radio the camera and/or RoboRio are connected to?
What impact on throughput in the radio does this packet prioritizing have? Is it progressively restrictive as non-command traffic incresses, or does it just kick in at a certain threshold?
I would highly recommend dumping the Raspberry Pi and running MJPG-Streamer directly on the RoboRIO using the link below. I installed using offline instructions which can be found in the readme at the root of the repository. You’ll also have to SSH in and edit the /etc/default/mjpg-streamer file and then restart to enable streaming of the second camera and change resolution. This is what we had running all of CIR and it worked great. I helped 4156 install this on practice day and I believe 4143 and one other team were running it. No issues at all as far as I know. None of us were using a static IP for anything. Come talk to me on Thursday at Midwest if you’re still having issues or just want help getting it set up.
