Hello everyone. At my teams last competition, we decided to use the FRC Vision image on the Raspberry Pi to try to stream high frame rate, low latency camera streams. While it worked on our practice bot, when we went to competition. The stream would not show up on Shuffleboard. The Pi was using a static IP address. Through FMS, I could use the frcvision.local to open a stream through a browser. However, the latency was too high to be usable. After discussing with a CSA, I was unable to get the stream working without having to directly wire the Pi to the driver station. Is there any known cases of the Raspberry PI stream working and are there any known solutions to this problem I’m having? Thanks in advance.
Did you make sure that you aren’t exceeding the 4mbps bandwidth limit imposed by the FMS?
If you’re able to connect via a web browser and seeing high latency there, bandwidth utilization is the most likely culprit. The main system status page on the webdash shows how much bandwidth the Pi is using.
Shuffleboard not seeing the stream would be almost certainly due to NetworkTables not connecting from the Pi to the RoboRio. Were you using the built-in multi camera streaming or did you customize it (e.g. with Python)? Was the RoboRio set to the static .2 address? We’ve also occasionally seen this happen with some radios; did you try swapping radios? This should also have been visible in the console logging on the Vision Status page–reconnection attempts continuing to show up there / never getting a “NT: CONNECTED” message would indicate NetworkTables is not connecting.
Yes. The cameras only reach 2mbps.
Edit: Only on Shuffleboard. I wasn’t able to figure out how to control compression on the pi.
We were using the built-in multi camera streaming. The RoboRio I believe was set to a .2 address although I have no way to tell right now due to the robot being in a bag. However, I did not check the Vision Status output. I should have remembered to have don that. I did swap radios. I was told by some CSAs at the competition that the new radios were known to have issues with the second port. I will try to confirm soon what radio we have on our teams practice robot.
It may be worth trying to put an Ethernet switch on your robot. The downside is that it introduces another potential failure point, but it would be a useful data point. Or try swapping to the old radio.
It’s possible to control compression on the Pi on the camera settings page of the webdash (this was added in 2019.2.1 so make sure you have the latest version). There’s also a way to do it with the stream URL, but this is easier.
Thanks for the quick reply!
We did have an ethernet switch at the competition but not materials to wire it and power it sadly. Would I wire the Rio and Pi to the switch and then to the radio or does the Rio have to have a direct connection to the Radio. I was also thinking about switching to an older radio.
About the compression, would I use the default compression, compression setting. I believe remembering there were two different setting for compression and how would I set it?
Edit: Also. would going through the URL be better than sending it to the network tables in terms of performance or would it be virtually the same?
It’s possible hooking up just the Pi to the switch could fix it, but it’s much more likely that the fix would be to have both the Rio and the Pi hooked up to the switch, and the switch hooked up to the radio. Doing the latter unfortunately makes the switch system-critical–if it loses power you’ll lose comms to the Rio.
I know it’s a little confusing to have two settings. In this case you want the one that’s not labeled “default”. Basically “default compression” is what’s used if the camera itself does not provide MJPEG. The “compression” setting forces recompression even if the camera provides MJPEG and the stream doesn’t request different compression (which is what happens when you use the webpage). The MjpegServer docs describe the difference too.
NetworkTables is only used to publish information about where to find the stream so dashboard apps such as Shuffleboard can find it. The stream itself is always fetched through HTTP. So there’s no difference in performance.
Thanks for the insight. I’ll be sure to try it and report back.
I have one last question. I am not quite sure how having the just the Pi to the switch would help as their would still be direct connection between the Pi and the RoboRio without having to go through the second radio port. Could you elaborate on how just having the Pi the switch would be wired and work? Thanks.
Your current network topology is Rio - Radio - Pi.
If it’s Radio - Switch - Rio + Pi, then the Rio and Pi can talk directly to each other through the switch without going through the radio at all.
If it’s Rio - Radio - Switch - Pi, the Rio to Pi traffic does indeed still go through the radio. I agree with you that in general this shouldn’t be any different than your current topology without a switch. However, there’s definitely some oddities with the radio with regards to how link is established and how switching occurs between the ports of the radio and things like link establishment ordering (e.g. I’ve seen the Pi not be able to communicate, but simply unplugging it and plugging it back into the radio fixes communication). Adding the switch means that the link between the radio and the switch will come up pretty much instantly, and not be subject to the same delays/changes of the radio to Pi link.
Cool. Thanks you so much for the help. As soon as I get a chance, I will test this out.
I was able to test the “bad” radio after reconfiguring it. When I plugged in the pi and rio directly into the ports of the radio, the camera stream showed up on Shuffleboard, but when I checked the console output on frcvision.local, it said:
NT: connect() to 172.22.11.2 port 1735 timed out
NT: ERROR: could not resolve roboRIO-xxxx.FRC.frc-field.local address (TCPConnector.cpp:99)
This makes me think that there could be a problem with the configuration for the radio at competitions.
I was able to successfully change the compression through frcvision.local and get a working video stream to Shuffleboard through the Radio - Switch - Rio + Pi topology. However, the switch went nuts and stopped working after a while, which I have never seen before so it could just be a problem with the switch itself.
Edit 1: I just retested the Radio - Switch - Rio + Pi topology. The switch went nuts most likely due to overheating. I also was able to check the console output here.
The network tables showed
NT: Clint: CONNECTED to server 10.xx.xx.2 port 1734
NT: ERROR: select() to 172.22.11.2 port 1735 error 113- No route to host (TCPConnecctor.cpp:173)
which I believe is an improvement and would allow the stream to show up at competitions although I am mainly betting on streaming through URL.