Access USB Camera stream from OpenCV app on driver station

My team’s robot has two USB cameras plugged into the RoboRIO, which are streamed to Shuffleboard using wpilib’s CameraServer library. I would like to create a custom Java application which runs on the driver station, which can read the camera streams from network tables and use OpenCV to do computer vision tasks on individual frames from the streams. The processing is resource intensive, and the outputs are shown to the drivers but not sent back to the robot, so it doesn’t make sense to do this processing on the RoboRIO. However, after looking through the CameraServer documentation and the wpilib guides, I’m not sure how to create my own CameraServer client. This page covers creating a custom networktables client, but I haven’t been able to extend it to include camera streams. Has anyone had success with this approach before?

Noah - Team 1157 Programming Captain

We do not have documentation on how to do this directly, but there are several options. The important thing to realize is the camera streams themselves are simply MJPEG over HTTP; there’s a NetworkTables key per camera that provides the available URLs published by CameraServer. This means there are two parts to this task: reading the URLs from NetworkTables (a small bit of parsing code) and actually reading the camera stream. For the second part, you can use any library that can read MJPEG-over-HTTP streams. The CameraServer library provides this with the HttpCamera class, but other libraries do too. For an example of what this code looks like using the CameraServer library, you can look at Shuffleboard’s CameraServerSource source code.

This is where I’ve historically started - just hardcode the consuming program with the URL’s of the streams.

The NT layer on top allows for greater flexibility, especially if you rearchitect your system later (or, equivalently, if you are making a program for distribution to other teams).

Thanks so much for you quick and thorough reply! I wrote a simple wrapper for the HttpCamera class and it worked perfectly.