Viewing a custom camera stream on Shuffleboard

Hey all, I have a custom MJPEG stream set up at an URL, say http://10.61.35.19:1180/stream?topic=/main_camera/image_raw&quality=20&width=320&height=180. I want to be able to view this camera stream on Shuffleboard. I looked around a bit and found some documentation. I tried to set the /CameraPublisher/MainCamera/streams entry in NetworkTables to ["mjpeg:http://10.61.35.19:1180/stream?topic=/main_camera/image_raw&quality=20&width=320&height=180"]. Shuffleboard is able to detect the stream, but it just shows up as a black box when I try to display it. How do I fix this? Any help would be appreciated.

P.S. Here’s some code:

    public static final String FRONT_CAMERA_URL = "mjpeg:http://10.61.35.19:1180/stream?topic=/main_camera/image_raw&quality=20&width=320&height=180";
    private static final NetworkTableEntry mainCameraUrlEntry = NetworkTableInstance.getDefault()
            .getTable("CameraPublisher").getSubTable("MainCamera").getEntry("streams");

    public static void setMainCameraURL(String url) {
        mainCameraUrlEntry.forceSetStringArray(new String[] { url });
    }

// ...

setMainCameraURL(FRONT_CAMERA_URL);

I don’t know much about streaming from cameras or having camera streams on Smartdashboard, so I can’t help too much there.

I would say you should try to view the stream in another way (like making a basic html page with just the stream in a canvas) to see if the stream itself is working

If I remember correctly, the stream url can’t include quality, width, or height, as it unconditionally adds them. So you would have duplicate copies, which whatever server you’re using might not support.

Hmm yeah that sounds like it might be the cause. I’ll definitely try it once I get the bot again. Thanks!

I’ve done some experiments with my stream (it’s provided by the web_video_server ROS package) and it seems to accept duplicate copies of parameters (it just uses the last one I think). It’s still not working even after removing all the parameters from the stream URL (i.e. using mjpeg:http://10.61.35.19:1180/stream?topic=/main_camera/image_raw instead). The stream itself is working fine in a browser.

If I launch Shuffleboard in a terminal and attempt to view the camera stream, it prints

CS: WARNING: MainCamera: disconnected during headers (HttpCameraImpl.cpp:241)

What does it mean? Is there something wrong with the HTTP headers? If I do a curl -I I get

HTTP/1.0 200 OK
Connection: close
Server: web_video_server
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0
Pragma: no-cache
Content-type: multipart/x-mixed-replace;boundary=boundarydonotcross
Access-Control-Allow-Origin: *

does this look right?

That error will only happen if the connection is closed while cscore is trying to read the frame header. The header you dumped above is actually not the header in question; a MJPEG server will output another HTTP header in between each frame following the multipart boundary. If you can capture a segment of the actual stream that would be helpful for trying to reproduce the problem.

If you can capture a segment of the actual stream that would be helpful for trying to reproduce the problem.

Thanks for the clarification. How would I do this?

Run curl http://... -o test_stream.dump and ctrl-C it after a couple of seconds. The test_stream.dump file should have a mix of text (multipart boundaries and headers) and binary (the image data).

Thanks for the help! Here’s the dump:

stream.dump (96 KB)

I’ve given up and just wrote my own plugin for a simple MJPEG Stream Viewer instead…

Never mind DONT USE IT it crashes

1 Like

Shuffleboard plugins are a travesty, and they’re badly documented with minimal support. With that being said, I’ve suffered the pain of it once, so what’s the crash?

Well I’m not exactly sure why it happens. Basically after running for a few minutes the camera stream would begin to freeze (although the FPS and bandwidth indicators still indicate that it’s transmitting data), and then a few mins later it would crash Shuffleboard.

My guess is that this is probably caused by me not taking into account all the painful multithreading stuff. I have a background thread that gets the images from the stream, which has an ObjectProperty<Image> that my Widget controller class can bind to in initialize(). The listener just redraws the image, and I’m guessing that the listener will be called on the worker thread instead of the JavaFX thread, which JavaFX is not happy about. As it stands right now, it also has another problem - every instance of the stream viewer widget starts its own background worker thread, and since I have no idea when the Widget is deleted, these background daemon threads never get stopped and just hang around until the VM exits. If anyone can offer any suggestions on how to fix this (or how to make the official CameraServer plugin work), it would be greatly appreciated.

My challenge is not being able to reproduce the problem. If you set me up with a step-by-step of how to get a web_video_server ROS node running with video so I can run it in a docker image, that would go a long way towards enabling me to figure out what the issue is.

Unfortunately, most of this was set up by a friend of mine so I’m not exactly sure what are the minimum requirements for getting a web_video_server node up and running. But currently I’m doing it by cloning the entire repository for our team’s 2019 Jetson code and then building and running it to get the stream. There are some instructions on how to build and run it here. This should get you a camera stream at http://localhost:1180/stream?topic=/main_camera/image_raw.

Note that this program is designed to be used with two cameras, so you’ll have to modify src/bot/launch/cameras.launch to get rid of the second camera, so you end up with something more like this:

<launch>
	<!-- Web video server -->
	<node pkg="web_video_server" type="web_video_server" name="main_video_server" respawn="true">
		<param name="~port" value="1180" />
	</node>
	
	<!-- Cameras -->
	<node pkg="usb_cam" type="usb_cam_node" name="main_camera" respawn="true">
		<param name="camera_name" value="main_camera" />
		<param name="video_device" value="/dev/video0" />
		<param name="camera_info_url" value="package://bot/config/main_camera.yaml" />

        	<param name="~framerate" value="30" />
		<param name="~pixel_format" value="yuyv" />
	</node>
</launch>

also depending on the camera, you might also have to get rid of the <param name="~pixel_format" value="yuyv" /> line. If you have any problems reproducing this please let me know. Thanks!

Seems like the problem with the camera stream viewer plugin I wrote was caused by multithreading. I have changed it to use JavaFX’s concurrency API, and this one has been tested to not crash: https://github.com/Arctos6135/StdPlug/releases/tag/v0.2.0