Patched Widget for USB Camera in SmartDashboard

[RyanCahoon]

Hi all-
After hearing of and experiencing the issue with streaming the USB Camera to the SmartDashboard, I investigated the issue some, and present here what I feel is an optimal solution.

The problem appears to be that the frames being sent from the camera driver are being extracted from an MJPEG stream, and thus are missing a part of the JPEG header called the Huffman Table.

As previously noted, one solution is to use the code from the Intermediate Vision template, which works because it re-encodes the JPEG before sending it to the driver station. However, this means that the RoboRIO has to decode, then encode the JPEG image for each frame the camera sends. Also, the Intermediate Vision example places the image processing in the main control loop, which can interfere with robot control functions. Ideally, we would be able to use the CameraServer class, which doesn't process the image data and runs in separate threads from the main robot loop, reducing latency for both the camera stream and robot control.

Tl;Dr: I've created a modified version of the "USB Webcam Viewer" applet that repairs the images coming from the USB camera. Download it here, and extract it in your user directory (usually C:\Users\<username>), then add the "USB Webcam Viewer (Compat)" in SmartDashboard. More instructions here.

You can download the source here.

It reinserts the Huffman Table at the correct place in the image data before decoding the JPEG for display. This lets the driver station computer (which is likely much faster than the RoboRIO) do the work of repairing the image data, and does it much more efficiently than having to re-encode the entire image.

A slightly more efficient solution might be to use a JPEG decoder that's agnostic to the Huffman Table being missing, but the Java JPEG decoder is so entwined with the AWT framework that replacing it in a manner that wouldn't lead to more inefficiencies would be extremely difficult. In addition, the presented solution requires only a handful of memory reads and two block memory copies, so it should be very efficient.

Thanks to @TheChewanater on Reddit for posting the original code to the USB Camera Viewer widget. The algorithm to fix the JPEG header was adapted from here.