mjpg-streamer running on roboRIO

[taichichuan]

Here at the Orlando Regionals while waiting in the pits for our robot to start working, I ported mjpg-streamer to the roboRIO. It was pretty funky since the source doesn't lend itself to cross-compilation. So, the answer? Compile it on a Beaglebone Black running Angstrom and copy the files over.

Result? A working camera feed that takes only 1% of the CPU. And, with a little Javascript magic, you can have multiple cameras running at the same time. There was only one little problem in that the libjpeg code isn't available on the roboRIO. So, simply copy the libraries from the BBB and place them in /usr/lib. Open source is a wonderful thing.

Then, untar the mjpg-streamer code and run it ala the instructions.

Enjoy,

Mike

[AlexanderTheOK]

Multiple cameras? Last I tried to use mjpg-streamer for anything productive I found that only a select few models didn't hog the entire USB bandwidth on their own (causing a second camera to fail because there "isn't enough bandwidth" . Most cameras that are "uvc compatible" just aren't because of the underdevelopment by the companies that make them. What cameras are you using?

[taichichuan]

Multiple cameras? Last I tried to use mjpg-streamer for anything productive I found that only a select few models didn't hog the entire USB bandwidth on their own (causing a second camera to fail because there "isn't enough bandwidth" . Most cameras that are "uvc compatible" just aren't because of the underdevelopment by the companies that make them. What cameras are you using?

The Logitech C920 does hardware H.264 encoding. Mjpeg-streamer understands that and doesn't repeat the process. This significant;y lowers CPU utilization.

HTH,

Mike

[virtuald]

This sounds great. I've been contemplating finding something like this and compiling for the roborio (though I was considering ffmpeg et al), now I don't have to! :)

I've been nervous about running the streaming stuff in the same process as our robot code (particularly since we're in python and there are GIL contention issues), so having a standalone executable sounds great.

[cbf]

Have you guys run this during a competition yet? I would be concerned about the bandwidth taken by two simultaneous video streams. In the past, we've accidentally saturated the feed between our laptop and the robot with a single 640 x 480 stream.

[EmileH]

How do you display it on the PC, thru smartdashboard or VLC? And does the FMS allow for communications over the ports used?

[taichichuan]

Have you guys run this during a competition yet? I would be concerned about the bandwidth taken by two simultaneous video streams. In the past, we've accidentally saturated the feed between our laptop and the robot with a single 640 x 480 stream.

Actually, you drop a Javascript page onto the ./www directory that has a button that allows you to switch between the streams. So, you're not getting both streams simultaneously unless you want them.

HTH,

Mike

[taichichuan]

How do you display it on the PC, thru smartdashboard or VLC? And does the FMS allow for communications over the ports used?

Actually, a simple web browser will do the display just fine. You can compile mjpg-streamer for vlc, udp, rtsp or one of several other options. However, we've just been using the output_http.so plugin and Chrome, Firefox or IE to display it on the DS. You can even make it start automatically just like the dashboard normally would if you want to.

HTH,

Mike

[virtuald]

We're actually doing our DS in HTML/JS, with a python interface to forward networktables to/from the HTML page, so the mjpg-streamer is exactly what we're looking for. We played with it a little bit today and it's pretty sweet, but haven't wired it into our interface yet.

I'm curious, what javascript magic do you need to do to switch streams? Is there a delay? We setup two servers, and were able to stream two cameras that way. Didn't try the JS magic yet.

[virtuald]

Multiple cameras? Last I tried to use mjpg-streamer for anything productive I found that only a select few models didn't hog the entire USB bandwidth on their own (causing a second camera to fail because there "isn't enough bandwidth" . Most cameras that are "uvc compatible" just aren't because of the underdevelopment by the companies that make them. What cameras are you using?

We tested two Lifecam 3000 cameras this afternoon running two servers, and at 160x120 were at about 5% CPU usage. 320x160 was roughly the same. Haven't tried measuring the network bandwidth yet.

[cbf]

I'm trying to set this to test. Where did you guys install the ./www pages on the roboRIO?

Easiest thing that I can see is just to put everything under /var/local/natinst/www, since that would avoid having to edit the web config file (which seems to be at /etc/natinst/NISystemWebServer.conf).

The other alternative would be to add a second document root, but while that NISystemWebServer.conf bears resemblances to Apache .conf files (perhaps version 1.X?), it pretty clearly has proprietary NI directives in it, and I'm hesitate to make changes without real documentation (which I suppose I could hunt for on NI's site).

Also, did anyone notice the mjpeg-stream stuff at /usr/camera_server/mjpg-streamer-r63? It appears that's where the NI-IMAQdx routines get their USB webcam support from. It seems to be an older(?) version of the same MJPEG streaming libraries you guys are playing with.

[cbf]

When I try to run this, I'm getting an error opening the /dev/video0 device. I have a Logitech P930e plugged in, which should certainly support the requested mode.

admin@roboRIO-2877:/var/local/natinst/www/mjpg# source start.sh
MJPG Streamer Version: svn rev:
i: Using V4L2 device.: /dev/video0
i: Desired Resolution: 640 x 480
i: Frames Per Second.: 5
i: Format............: MJPEG
Unable to set format: 1196444237 res: 640x480
Init v4L2 failed !! exit fatal
i: init_VideoIn failed

admin@roboRIO-2877:/var/local/natinst/www/mjpg# ls -l /dev/video0
crw-rw-rw- 1 admin ni 81, 0 Dec 31 1969 /dev/video0

[virtuald]

What you want to execute is something like this:

LD_LIBRARY_PATH=`pwd` ./mjpg_streamer -i "./input_uvc.so --device /dev/video0 -f 10 -r 160x120" -o "./output_http.so --port 5800 -w www"

Of course, you're going to want to write scripts and such to make it easy to run, and more importantly start on robot boot. I did all of this already, so I put install instructions + the scripts I created into a gist, you can access it at https://gist.github.com/virtuald/c8835244759e53314211

Feedback/comments welcome!

[cbf]

Actually, it turns out the default shell script was working for me. I had forgotten to shut down FRCProgram, which also opened the camera!

But your scripts for starting, etc. will be helpful if we decide to go this route.
Right now at 640 x 480, a stream from my Logitech P930e is taking 2.8Mbps, which is higher than the stream from our robot program, which uses the NI MAXQdx libraries to send a 640 x 360 stream at 24fps and onlyh takes 2.0 mbps. I'm still leery of how this will perform through the FMS.

But you said something about doing an entire Dashboard in JavaScript. That really intrigues me. And a Python implementation of NetworkTables. Is that running server side or client side?

[virtuald]

But you said something about doing an entire Dashboard in JavaScript. That really intrigues me. And a Python implementation of NetworkTables. Is that running server side or client side?

Check out the latest RobotPy project, pynetworktables2js (https://github.com/robotpy/pynetworktables2js). It forwards networktables traffic to/from an HTML page via a websocket. It hasn't really had a proper release announcement, but we'll be using it in our dashboard at our comp next week.

We run it on the driver station.

[cbf]

It's worse than I thought. My earlier comparison was with the MJPG streamer at 10 fsp and my robot program at 24fps. The MJPG streamer uses about 3 times the bandwidth as the WPILib streamer running on the robot.

It looks like I'm getting an MJPEG stream from my Logitech P930e (similar to the Logitech P920 but with a wider field of view), rather than an H.264 stream.

Do you know if there's any way I can persuade input_uvc.so to get an H.264 or MPEG-4 stream from the camera?

[virtuald]

It's worse than I thought. My earlier comparison was with the MJPG streamer at 10 fsp and my robot program at 24fps. The MJPG streamer uses about 3 times the bandwidth as the WPILib streamer running on the robot.

It looks like I'm getting an MJPEG stream from my Logitech P930e (similar to the Logitech P920 but with a wider field of view), rather than an H.264 stream.

Do you know if there's any way I can persuade input_uvc.so to get an H.264 or MPEG-4 stream from the camera?

What quality setting are you using? That could affect it. (-q option). Also, I'm curious, what are you using to measure the bandwidth usage?

[virtuald]

For anyone who wants to read data from mjpg-streamer, I've posted python code that can read from the stream using urllib + opencv.

http://www.virtualroadside.com/blog/index.php/2015/04/03/better-python-interface-to-mjpg-streamer-using-opencv/

We ended up not using any image processing at our competition, but because our dashboard was created using HTML/js, it was trivially easy to connect to the camera and display its output. Highly recommend mjpg-streamer as a solution if you're only needing to display output on the Driver Station.

[cbf]

What quality setting are you using? That could affect it. (-q option). Also, I'm curious, what are you using to measure the bandwidth usage?
I believe I tried a few -q choices, but it was a while ago now.

I measured the bandwidth using the Windows task manager. Since there was nothing else on the WiFi between the laptop and the robot router (usual DLink 1522B), I believe the bandwidth measurements are reliable.

[virtuald]

FYI, I packaged mjpg-streamer into an IPK file that can be installed by opkg. It includes an init script that automatically starts mjpg-streamer when the RoboRIO starts.

Installation: https://github.com/robotpy/roborio-packages
Notes: https://github.com/robotpy/roborio-packages/tree/2016/ipkg/mjpg-streamer

[taichichuan]

FYI, I packaged mjpg-streamer into an IPK file that can be installed by opkg. It includes an init script that automatically starts mjpg-streamer when the RoboRIO starts.

Installation: https://github.com/robotpy/roborio-packages
Notes: https://github.com/robotpy/roborio-p.../mjpg-streamer

Thanks! This is a big help.

MikeA
FRC #116

[team-4480]

FYI, I packaged mjpg-streamer into an IPK file that can be installed by opkg. It includes an init script that automatically starts mjpg-streamer when the RoboRIO starts.

Installation: https://github.com/robotpy/roborio-packages
Notes: https://github.com/robotpy/roborio-packages/tree/2016/ipkg/mjpg-streamer

I am trying to install it on our RIO, but I get this message: http://imgur.com/wzr8bJj

I did update to the latest version of RobotPy on the robot.

Thanks!

[virtuald]

The opkg architecture has changed from 2015 to 2016, so if you don't have the 2016 image installed, then you will get that error.

[team-4480]

The opkg architecture has changed from 2015 to 2016, so if you don't have the 2016 image installed, then you will get that error.

Oh yea, I still have to update. So once I get it installed, do I just plug in the webcam and I will see it on the driver station? Or is there some other setup I have to do yet?

[virtuald]

I haven't tried to integrate mjpg-streamer with the LabVIEW dashboard (and it's not really something I'm planning to do).

To see the camera feed, point a web browser at http://roborio-4480-frc.local:5800/ and it should show up there. If it doesn't, then reboot your roboRIO with the camera plugged in.

[team-4480]

I haven't tried to integrate mjpg-streamer with the LabVIEW dashboard (and it's not really something I'm planning to do).

To see the camera feed, point a web browser at http://roborio-4480-frc.local:5800/ and it should show up there. If it doesn't, then reboot your roboRIO with the camera plugged in.

Awesome! Is it possible to connect two cameras with this or would that eat up too much bandwidth?

[virtuald]

Awesome! Is it possible to connect two cameras with this or would that eat up too much bandwidth?

This package supports launching up to 4 instances of mjpg streamer (the second would be at :5801 instead of :5800). You'll need to edit /etc/default/mjpg-streamer on the roborio, followed by '/etc/init.d/mjpg-streamer restart' or a robot reboot to enable this.

Refer to the FMS Whitepaper (http://wpilib.screenstepslive.com/s/4485/m/24193/l/291972-fms-whitepaper) for bandwidth approximations. Last year we were able to run two 120x160 streams at 10fps with no complaints from the FTAA (we doubled the display size of the image on our HTML dashboard).

[team-4480]

This package supports launching up to 4 instances of mjpg streamer (the second would be at :5801 instead of :5800). You'll need to edit /etc/default/mjpg-streamer on the roborio, followed by '/etc/init.d/mjpg-streamer restart' or a robot reboot to enable this.

Refer to the FMS Whitepaper (http://wpilib.screenstepslive.com/s/4485/m/24193/l/291972-fms-whitepaper) for bandwidth approximations. Last year we were able to run two 120x160 streams at 10fps with no complaints from the FTAA (we doubled the display size of the image on our HTML dashboard).
I am not able to see the page, firefox just says unable to connect when I try "roborio-4480-frc.local:5800". I did reboot so I am wondering if the config file is off because I set it up over USB using 172.... instead of Wifi. I imagine there is a setting in the config file, but I don't know how to get into it since my favorite editor is not installed(nano).

[virtuald]

I am not able to see the page, firefox just says unable to connect when I try "roborio-4480-frc.local:5800". I did reboot so I am wondering if the config file is off because I set it up over USB using 172.... instead of Wifi. I imagine there is a setting in the config file, but I don't know how to get into it since my favorite editor is not installed(nano).

To diagnose you'll need to SSH into the roborio (user: admin, password: just hit enter).


Does /dev/video0 exist? If not, then your camera isn't recognized by the roboRIO and mjpg-streamer won't be running. You can use 'dmesg' to look at the kernel messages and try to divine what happened. Not all cameras will be supported -- but I'm not 100% sure what ones are supported, or how to get the list (probably some linux magic)
Is mjpg-streamer running or did it die? "ps -Af | grep mjpg" will show you
If it's running, is it listening on the right port? "netstat -ln" can show you what ports are being listened on, 5800 should be in the list.
If /dev/video0 exists and mjpg-streamer dies, then there's something else wrong and you'll need to run mjpg-streamer on the command line. Unfortunately, I haven't put anything in the package to do this using the settings, so it's a bit verbose...

[team-4480]

To diagnose you'll need to SSH into the roborio (user: admin, password: just hit enter).


Does /dev/video0 exist? If not, then your camera isn't recognized by the roboRIO and mjpg-streamer won't be running. You can use 'dmesg' to look at the kernel messages and try to divine what happened. Not all cameras will be supported -- but I'm not 100% sure what ones are supported, or how to get the list (probably some linux magic)
Is mjpg-streamer running or did it die? "ps -Af | grep mjpg" will show you
If it's running, is it listening on the right port? "netstat -ln" can show you what ports are being listened on, 5800 should be in the list.
If /dev/video0 exists and mjpg-streamer dies, then there's something else wrong and you'll need to run mjpg-streamer on the command line. Unfortunately, I haven't put anything in the package to do this using the settings, so it's a bit verbose...

Alright, so there is video zero

With the "ps -Af | grep mjpg", I get

1602 admin {grep} /bin/busybox.nosuid /bin/grep mjpg


There is not 5800 on the port list

Thanks for all the support!

[virtuald]

K, then mjpg-streamer has decided to die for some reason. You should be able to execute this and see what the error is...


mjpg_streamer -i '/usr/local/lib/mjpg-streamer/input_uvc.so --device /dev/video0 -f 10 -r 160x120' -o '/usr/local/lib/mjpg-streamer/output_http.so -w /usr/local/share/mjpg-streamer/www -p 5800'

[team-4480]

K, then mjpg-streamer has decided to die for some reason. You should be able to execute this and see what the error is...


mjpg_streamer -i '/usr/local/lib/mjpg-streamer/input_uvc.so --device /dev/video0 -f 10 -r 160x120' -o '/usr/local/lib/mjpg-streamer/output_http.so -w /usr/local/share/mjpg-streamer/www -p 5800'


I get this error:

ERROR: could not find input plugin
Perhaps you want to adjust the search path with:
#export LD_LIBRARY_PATH=/path/to/plugin/folder
dlopen:libjpeg.so.8: cannot open shared object file: No such file or directory

[virtuald]

I get this error:

ERROR: could not find input plugin
Perhaps you want to adjust the search path with:
#export LD_LIBRARY_PATH=/path/to/plugin/folder
dlopen:libjpeg.so.8: cannot open shared object file: No such file or directory


Sounds like a bug in the installer, oops. Is there a 'libjpeg*.ipk' in the opkg_cache directory? You should copy that to the roborio, and 'opkg install name_of.ipk'. Then it should work.

[virtuald]

The problem is that libjpeg didn't get installed due to an error in the control file. I'll rebuild the package later, but for now if you use the installer to install libjpeg8 then it should work.

[Greg McKaskle]

The default dashboard, which is simply a build of the template code of the LV dashboard, has four options for the camera display -- Off, USB SW, USB HW, and Axis IP.

I believe that USB HW is compatible with the mjpeg-streamer, but it expects the stream on port 1180, which was the initial image port. I don't have stuff to test with, but I suspect that changing the .cfg file would make it compatible. From the code, I suspect that USB SW would also work. The difference between these is that for USB SW, the roboRIO acquires uncompressed images, compresses them according to the DB settings, then transmits. USB HW requests compressed images from the camera, and transmits. It is not possible to control what that compression rate will be for USB HW.

The default DB does send a two 32 bit integers and one 16 bit to the stream port to specify the framerate, compression rate, and size. I think the streamer will ignore these and work fine. For LV teams, I'm not sure that this streamer is much different from the one in the palette and in the default robot template.

Greg McKaskle

[virtuald]

I believe that USB HW is compatible with the mjpeg-streamer, but it expects the stream on port 1180, which was the initial image port. I don't have stuff to test with, but I suspect that changing the .cfg file would make it compatible. From the code, I suspect that USB SW would also work.

Nope, just tried it, doesn't work when I switched it to 1180.

[Greg McKaskle]

Some of the code precedes me, and it wraps the jpeg with a header that contains a version number of 1,0,0,0. I can't really look at the arriving stream, but I guess that the version is something he added that is not part of the standard stream and that is what is causing it to be rejected.

When I can get to it, I'll see if that is the issue. But unfortunately, it will need a slightly modified VI and will not work with the default DS.

Greg McKaskle

[virtuald]

I updated the package, so team-4480 if you just use the installer to download/install mjpg-streamer again, it should install libjpeg now too. I also added some scripts to the package so that you can do 'mjpg_streamer_cfg1' to run the first configuration by hand, instead of having to guess what the command line args look like. :)

[team-4480]

I updated the package, so team-4480 if you just use the installer to download/install mjpg-streamer again, it should install libjpeg now too. I also added some scripts to the package so that you can do 'mjpg_streamer_cfg1' to run the first configuration by hand, instead of having to guess what the command line args look like. :)

Thanks! I appreciate all your time and effort!

[Joe Ross]

This package supports launching up to 4 instances of mjpg streamer (the second would be at :5801 instead of :5800). You'll need to edit /etc/default/mjpg-streamer on the roborio, followed by '/etc/init.d/mjpg-streamer restart' or a robot reboot to enable this.

Should the port number increment automatically? /etc/default/mjpeg-streamer uses 5800 for both. I was only able to get the second camera to work when I set the second port number to 5801.

[virtuald]

Should the port number increment automatically? /etc/default/mjpeg-streamer uses 5800 for both. I was only able to get the second camera to work when I set the second port number to 5801.

I believe that would be a copy/paste error on my part.

[Joe Ross]

Some of the code precedes me, and it wraps the jpeg with a header that contains a version number of 1,0,0,0. I can't really look at the arriving stream, but I guess that the version is something he added that is not part of the standard stream and that is what is causing it to be rejected.

When I can get to it, I'll see if that is the issue. But unfortunately, it will need a slightly modified VI and will not work with the default DS.

Greg McKaskle

I think there's more to it. It's streamed over http, which means you have to send the right headers. In that respect, it's closer to the axis-camera. In my quick look at axis-camera, there's several vis that would have to be hacked up to get it to work with mjpg-streamer.

[Greg McKaskle]

Naively, the default DB works with IP cameras, and the jpeg-streamer is a virtual IP camera. So I agree. The USB camera server is also very similarly sending a sequence of jpegs, but without the http initiation -- I didn't write and had totally forgotten about the version wrapper.

So I think the major problem is that the existing IP stuff needs to know the camera IP. For USB, the camera IP is the robot IP. I'll see about getting this running on a roboRIO and documenting the changes needed, but it clearly won't work without code changes to the DB. Perhaps a fifth camera option. I was, of course hoping that it would work, or we could tweak the config.

Greg McKaskle

[Joe Ross]

I made a smartdashboard extension for viewing the camera stream from mjpg-streamer.

https://github.com/Beachbot330/MJPGStream_SDExtension

[team-4480]

Is there a way to set at what resolution a camera goes at? We are going to have two cameras and we want both to be sent back to the Driver Station. One camera is going to be for vision so we would probably want that one to be 640x480. The second camera is just for driving so a low resolution would do just fine. I just don't know how to go about changing the resolution in mjpg-streamer?

[virtuald]

Change /etc/default/mjpg-streamer. The default contents look like this:

https://github.com/robotpy/mjpg-streamer/blob/master/mjpg-streamer-experimental/init/default#L11

[team-4480]

Change /etc/default/mjpg-streamer. The default contents look like this:

https://github.com/robotpy/mjpg-streamer/blob/master/mjpg-streamer-experimental/init/default#L11

What text editor does the RoboRio have installed? Or can I use something like FileZilla?

[virtuald]

What text editor does the RoboRio have installed? Or can I use something like FileZilla?

vi.

You could use filezilla. Better in the long term to learn how to use vi.

[Joe Ross]

I got viewing the stream from mjpg-streamer working on the LabVIEW dashboard also. Use "BB_CameraDirectly from MJPG-Streamer.vi" from the attached zip.

[team-4480]

Does anyone know how save the exposure settings? We are using the control panel in the web interface in mjpg-streamer. The settings work but they don't save over a reboot. Any ideas?

Thanks!

[jreneew2]

Does anyone know how save the exposure settings? We are using the control panel in the web interface in mjpg-streamer. The settings work but they don't save over a reboot. Any ideas?

Thanks!

We would also like to be able to set the exposure manually through this as well. I was looking through the code and I only saw a setControl function and could not figure out where that came from. I am not very good at javascript or html so I would appreciate some guidance.

I was thinking that if we could find where those values are defined, we could manually set them. But I think they are in the hardware of the camera, so I'm not sure if we could set them. Or you could make a script to automatically set the values at start. I'm not sure how to do that either...

[virtuald]

Does anyone know how save the exposure settings? We are using the control panel in the web interface in mjpg-streamer. The settings work but they don't save over a reboot. Any ideas?

Thanks!

I will bring a camera home tonight and make it work.

[virtuald]

I've uploaded mjpg-streamer release 2016.2.0 to the ipkg repo, which will allow you to pass the following settings to the input plugin (via the INPUT[1] line of /etc/default/mjpg-streamer):


Optional parameters (may not be supported by all cameras):

[-br ].................: Set image brightness (auto or integer)
[-co ].................: Set image contrast (integer)
[-sh ].................: Set image sharpness (integer)
[-sa ].................: Set image saturation (integer)
[-cb ].................: Set color balance (auto or integer)
[-wb ].................: Set white balance (auto or integer)
[-ex ].................: Set exposure (auto, shutter-priority, aperature-priority, or integer)
[-bk ].................: Set backlight compensation (integer)
[-rot ]................: Set image rotation (0-359)
[-hf ].................: Set horizontal flip (true/false)
[-vf ].................: Set vertical flip (true/false)
[-pl ].................: Set power line filter (disabled, 50hz, 60hz, auto)
[-gain ]...............: Set gain (auto or integer)
[-cagc ]...............: Set chroma gain control (auto or integer)


Additional notes can be found here (https://github.com/robotpy/roborio-packages/blob/2016/ipkg/mjpg-streamer/README.md).

[jreneew2]

I've uploaded mjpg-streamer release 2016.2.0 to the ipkg repo, which will allow you to pass the following settings to the input plugin (via the INPUT[1] line of /etc/default/mjpg-streamer):


Optional parameters (may not be supported by all cameras):

[-br ].................: Set image brightness (auto or integer)
[-co ].................: Set image contrast (integer)
[-sh ].................: Set image sharpness (integer)
[-sa ].................: Set image saturation (integer)
[-cb ].................: Set color balance (auto or integer)
[-wb ].................: Set white balance (auto or integer)
[-ex ].................: Set exposure (auto, shutter-priority, aperature-priority, or integer)
[-bk ].................: Set backlight compensation (integer)
[-rot ]................: Set image rotation (0-359)
[-hf ].................: Set horizontal flip (true/false)
[-vf ].................: Set vertical flip (true/false)
[-pl ].................: Set power line filter (disabled, 50hz, 60hz, auto)
[-gain ]...............: Set gain (auto or integer)
[-cagc ]...............: Set chroma gain control (auto or integer)


Additional notes can be found here (https://github.com/robotpy/roborio-packages/blob/2016/ipkg/mjpg-streamer/README.md).

Thanks! I will test this tomorrow!

[vScourge]

I've uploaded mjpg-streamer release 2016.2.0 to the ipkg repo, which will allow you to pass the following settings to the input plugin (via the INPUT[1] line of /etc/default/mjpg-streamer)

Thanks for this. Is this build specifically for running on the RoboRio? We use mjpg-streamer on the Raspberry Pi, but we'd like to use a build that has these image control options.

[virtuald]

Thanks for this. Is this build specifically for running on the RoboRio? We use mjpg-streamer on the Raspberry Pi, but we'd like to use a build that has these image control options.

The source code is at https://github.com/robotpy/mjpg-streamer if you wish to compile a version for a different platform. I also made a pull request to the most active mjpg-streamer fork (https://github.com/jacksonliam/mjpg-streamer/pull/12).

[NegaNexus]

I'm having connectivity issues when the mjpg-streamer daemon service starts at boot. I placed the files in /home/lvuser/mjpg-streamer-182. I tried to set it up so that the server would startup at boot by adding


#!/bin/sh
cd /home/lvuser/mjpg-streamer-182/
sh start.sh


to a new CAM.sh script in /etc/init.d and then typing into the shell:$ update-rc.d /etc/init.d/start.sh defaults The interesting part is when I attempt to reboot and connect to the robot. It never connects until I unplug the camera, which kills the mjpg-streamer daemon. I can then manually start mjpg-streamer over SSH without issue, but no dice with an at boot start. Does anyone have any ideas?

[virtuald]

Does anyone have any ideas?

Sounds like you're going about it the hard way. Install the mjpg-streamer package from https://github.com/robotpy/roborio-packages and it'll automatically boot at startup.

[jreneew2]

How would you connect opencv code running on the roboRIO to the jpeg stream. (in c++). I've seen your code in python, but where are you getting that IP from? I've tried numerous ips and can't seem to connect it. I can connect the the page fine and see the stream at http://roboRIO-2053-frc.local:5800/?action=stream, but I don't know what IP to enter in the code. do you have to set a static ip for the camera?

EDIT: I've also tried doing this (after looking on google for a bit):
videoCapture.open("http://roborio-2053-frc.local:5800/?action=stream?dummy=param.mjpg");

Nothing. Any suggestions?

[jreneew2]

Also, I know the code works, because it connected to an axis camera fine. I'm pretty sure it cannot recognize the stream or it is not "looking" for it in the right place.

[virtuald]

To connect to an mjpg stream with OpenCV's VideoCapture object, OpenCV needs to have libav/ffmpeg support compiled in. The version of OpenCV that I distribute does not currently have libav support (see issue #2 (https://github.com/robotpy/roborio-opencv/issues/2)). That's interesting that you were able to connect to an Axis camera, as that would also require libav.

When running the code on the RoboRIO, you should be able to use this URL: http://localhost:5800/?action=stream

EDIT: It's possible that OpenCV 3.x does not require libav to connect to an mjpg stream.. but I know OpenCV 2 did require it. If you're connecting to the stream separately then that's would work too.

[jreneew2]

To connect to an mjpg stream with OpenCV's VideoCapture object, OpenCV needs to have libav/ffmpeg support compiled in. The version of OpenCV that I distribute does not currently have libav support (see issue #2 (https://github.com/robotpy/roborio-opencv/issues/2)). That's interesting that you were able to connect to an Axis camera, as that would also require libav.

When running the code on the RoboRIO, you should be able to use this URL: http://localhost:5800/?action=stream

EDIT: It's possible that OpenCV 3.x does not require libav to connect to an mjpg stream.. but I know OpenCV 2 did require it. If you're connecting to the stream separately then that's would work too.

Alright, thanks. I'm not sure what you mean when you said "If you're connecting to the stream separately then that would work too". Are you saying that over a web browser?

EDIT: So, is there an issue with linking libav to opencv? Also, I had an axis camera working on opencv 2.4.10, not 3.0. Maybe this (https://github.com/Team2168/2168_Vision_Example) will help you?

So, running the vision code with mjpg streamer on the roboRIO is out of the question at the moment? If so, would the next logical step to have vision run on the driver station computer? If so, how would I go about doing that?

Thanks again,
Drew

[virtuald]

Alright, thanks. I'm not sure what you mean when you said "If you're connecting to the stream separately then that would work too". Are you saying that over a web browser?

So, running the vision code with mjpg streamer on the roboRIO is out of the question at the moment? If so, would the next logical step to have vision run on the driver station computer? If so, how would I go about doing that?

Thanks again,
Drew

Well, you said you could connect to an Axis stream? If you can connect to that, then the mjpg-streamer stream should work too, if you have the right URL.

When I say "separately", I mean roll your own code to make the HTTP connection. Like this python code (https://github.com/frc1418/2015-vision/blob/master/robot-vision/mjpg_client.py). It's pretty easy in python because it's fairly easy to create http connections... I'm not sure what I would use in C++, it's been too long.

FWIW, the last time that I used OpenCV to connect to an mjpg stream (an Axis, actually) via ffmpeg, I ran into this bug: https://trac.ffmpeg.org/ticket/2343

[virtuald]

Heh, mjpg-streamer has an HTTP input plugin which reads images from another mjpg-streamer instance (https://github.com/robotpy/mjpg-streamer/blob/master/mjpg-streamer-experimental/plugins/input_http/mjpg-proxy.c). You could grab that code and use it to read from the stream, it seems to be pretty simple.. though, it is GPL so if you used it your code would automatically become GPL too.

[hardcopi]

I was able to install the mjpg streamer without issue, works great. The issue (well not really issue more of question) is: Is there a good way to check overall bandwidth usage. I have camera 1 running through Labview for vision tracking and camera 2 running through the streamer. I can adjust the frame rate, resolution, etc, but I want as high as possible without getting into the danger zone. For what we have planned for the 2nd camera even 1 or 2 frames a second is sufficient but I'd prefer it as high res as I can get it.

[NegaNexus]

Sounds like you're going about it the hard way. Install the mjpg-streamer package from https://github.com/robotpy/roborio-packages and it'll automatically boot at startup.

So I did this, but I still have issues with connectivity. I have two cameras set up currently, if that helps.

[virtuald]

So I did this, but I still have issues with connectivity. I have two cameras set up currently, if that helps.

Same issues? What kind of cameras are you using?

Are you trying to access the cameras from mjpg-streamer and another app (like the robot code)? That won't work, I think you can only access a camera from a single app at a time.

[NegaNexus]

Same issues? What kind of cameras are you using?

Are you trying to access the cameras from mjpg-streamer and another app (like the robot code)? That won't work, I think you can only access a camera from a single app at a time.

We're just accessing the cameras mjpg-streamer. The problem occurs intermittently.

UPDATE: It seems to be working fine now. I post again if we have problems

[imdunne8]

In case anyone is running 2 cameras and wants a convenient way to switch between them, I have a forked branch of the SmartDashboard plugin that allows it. It also has the ability to add a simple "crosshair" circle from SD.

https://github.com/RobotCasserole1736/MJPGStream_SDExtension

[virtuald]

FYI, I created an OpenCV 3.x input plugin for mjpg-streamer that allows you to capture from OpenCV and filter the input via a separate plugin. More information on github (https://github.com/mjpg-streamer/mjpg-streamer/blob/master/mjpg-streamer-experimental/plugins/input_opencv/README.md). And of course, there's a filter plugin that comes with it that allows you to write a python 3 script and use that as a filter, or you can write your own in C++.

Haven't done much testing with it, but it seems to work on my desktop. I have not pushed a working build into roborio-packages yet... I have packaging setup for it, but want to test it on the RoboRIO first. Will do that tonight.

[hardcopi]

What we ended up doing is running mjpg as well as an ip camera. 2 cameras at once just wasn't gonna happen no matter how much we throttled the fps and resolution. We ended up with a nice system of a single ip camera (Axis) and a generic 180 degree camera we got on Amazon. Pictures are excellent and we have a very good field of view. There is a 1/2 a second or so lag which the driver had to get used to though.