GStreamer-Shuffeboard - A plugin in beta

With the many teams pursuing H.264 encoding on coprocessors, I decided to create a Shuffleboard plugin. The plugin is still in very early beta, but it should be operational and I’m posting this here for teams to test (our team has furloughed for the summer) and for constructive criticism and suggestions.

Installation instructions - download the zip file for your platform (Windows only, 32 or 64 bit) here and unzip the contents to your Shuffleboard plugin directory (~/Shuffleboard/plugins).

To use this plugin, you must put the RSTP link of your camera on the /GStreams/$cameraName/streams/$link. An example script with pynetworktables can be found here.

If you do not know how to setup an RTSP stream, this post by rianadon is a great resource.

I have not used it, and we don’t have plans for test H.264, but please allow the user to set an arbitrary URL. It really annoys me that the default MJPEG plugin only accepts URLs from NetworkTables. No reason a team could not use a off-the-shelf Ethernet camera which does not understand NetworkTables.

The camera doesnt have to understand networktables to use it, you just need a link in the networktable which points to the stream, that link can be put there with any method. IIRC the Rio has a method for adding IP cameras, but you could just as easily use pynetworktables or the outline viewer.

1 Like

When we sat with the wpi folks at champs, we found out that adding IP cameras in your robot code actually funnels the camera network traffic through the Rio. Does anyone else get nervous about available cpu and ram resources as a result?

1 Like

I can see this being correct if you use the built in WPI method for adding an IP camera, however, there is nothing stopping you from directly adding the stream address to the networktable. While perhaps more involved, it will get around using Rio resources.
Interesting point though!

Heads up, you committed your .idea folder for a Gradle project.

Woo, more H.264! Definitely interested in reverse-engineering this… We tried to integrate with SmartDashboard to mixed success.
Did using Kotlin make things more difficult? I wouldn’t think so, but GStreamerPlugin.kt looks like a “dark magic shim” of sorts.

Nah Kotlin’s interop made things a breeze, and being able to keep stuff in Java (mainly the JavaFX) is great. The only “dark magic” part would be onLoad(), as I have yet to figure out a better way to package the DLLs.

Note: GStreamer-Java uses JNA and not JNI, which makes packaging a lot more painful compared to CScore or FFMpeg.

If you aren’t doing the actual vision processing on the robot, there is very little overhead from adding a camera. If you are so close that it matters, then you are way too close to maxing the Rio to begin with.

Thanks for the opinion! We didn’t want to take a chance in Detroit. We’ll take a look in the off-season.

This is only true when doing switched cameras. If you add just a single IP camera the camera traffic does not pass through the Rio (unless you are tethered via USB), as it publishes the IP address of the camera to NetworkTables and the dashboard will connect to the camera directly.

The only reason this would have more than negligible impact on CPU and RAM resources is if the Rio is recompressing the stream (e.g. if you have set different resolution or compression on the dashboard).

1 Like

I understand, but I really don’t see why I should have to fuss with forcing in a URL in one place, so I can read it out in another. If I know my camera’s URL, and it is never going to change, why can’t I just type it in to Shuffleboard and it is done “forever”. I can even read the stream if the NetworkTables server is not functioning (ie in a test setup with no Rio).

I also wish for this functionality. We were running two Limelights (front and back) and wanting to switch feeds in shuffleboard so we could save on bandwidth. At first the add switched camera seemed like a good idea

@prensing @vargoose84 You’ll be happy to know I’ve added the ability to set the GStreamer URI via text field in. Just drag a GStreamer widget out and right click to set the properties.

2 Likes

Just curious, why not use h265, our team uses it and compared to h264 it cuts the bandwith in half. Its really cool though

The RTSP pipeline is codec agnostic, so you should be able to run any codec, assuming I have the right DLLs.

ah okay

Thanks. I appreciate the flexibility and attention.

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.