2 Cameras Setup (Lifecam HD 3000)

Technical Java
#1

Hello all,

With the game this year being hard to navigate through, I would like to set up two cameras on our robot (in Java of course). However, I was only able to figure out how to get one set up. When both are plugged into the roborio, we do get 2 different camera anmes (“cam0” and “cam1”), so I attempted to set up the second camera using a similar method as the first, however this did not work. We are attempting to use the SmartDashboard for Java, and only one of the cameras showed up (with a second USB camera module on the dashboard, the first camera was duplicated). As was mentioned in the title, we are using two Microsoft Lifecam HD 3000s, and we do not have an axis camera.

Does anyone know how to get this working? I’m open to all suggestions – I’ll update this information if you guys request it.

Thank you all,
–aweso_meme, Team 4687 programmer

#2

The following post has some sample c++ code that handles 2 USB cameras. It might point you the right direction.

#3

My team is using two USB Cameras and Java. However, we are swapping between them, rather than steaming both at the same time.

Our code can be found here: https://github.com/iron-claw-972/FRC2016/blob/master/src/org/usfirst/frc/team972/robot/Robot.java

Look at lines 182-194 and 409-480 (may change in later commits)

#4

We are also streaming only one camera at a time. There are a couple of tricks to this:

  1. In Robot.init, instantiate a USBCamera instance for each camera. Then when you call CameraServer.startAutomaticCapture, pass in the default camera.

  2. You’ll need a Command, probably triggered by a button, to switch cameras. The natural thing to want to do is to simply call startAutomaticCapture again, passing in the other USBCamera instance. That won’t work, because of the way startAutomaticCapture is implemented. To solve this, you’ll have to create your OWN version of CameraServer (copy, paste, rename), and change the startAutomaticCapture to stopCapture on the camera already in use, if any (m_camera), set the new camera, then startCapture on the new camera.

If you need more details, let me know.

#5

How would you code the stopCamera() function without using NIVision?

#6

Sadly I can’t really answer that yet, as we haven’t dug into the new OpenCV-based library. However, you could start by looking at the UsbCamera and CameraServer classes’ source code to see if you can find where the streaming thread attaches to the camera. I’ll be working in this area in a couple of days, so if I find anything I’ll reply again.

#7

CameraServer has changed substantially for 2017, so the discussion/suggestions from 2016 no longer apply (I’m the author of the 2017 rewrite). To start two cameras, simply do (Java):


UsbCamera cam0 = CameraServer.getInstance().startAutomaticCapture(0);
UsbCamera cam1 = CameraServer.getInstance().startAutomaticCapture(1);

Note the no-arguments variant of startAutomaticCapture always starts camera 0. Thinking about this, it may be a good idea for us to make this auto-increment in the future to make this a bit easier to use.

There is no stopCamera() in 2017; the camera automatically starts and stops depending on what’s connected to it.

You may run into USB bandwidth limits running two cameras simultaneously. Keeping the resolution low is a good way to avoid this. Streaming one camera at a time and switching between them to conserve USB bandwidth isn’t well-supported at present (there’s nothing to prevent one camera starting before the other stops and thus hitting the bandwidth limit); that’s something we will be addressing in a future update.

#8

Why is switching difficult? What’s wrong with having two cameras and one mjpeg stream output:


VideoCamera cam0 = UsbCamera("Camera 0", 0);
VideoCamera cam1 = UsbCamera("Camera 1", 1);
MjpegServer server = new MjpegServer("Output to dashboard", 5800);

And switching between them using server.setSource()? We switched between two streams last year by modifying the CameraServer class, which worked fine. However, looking at the CSCore library, this seems like the “obvious” way to do it. I’m about to test this in a few hours, am I going to run into problems?

#9

How did your testing work out. Last year we had three USB cameras and were able to nicely switch among them. I would prefer that to consuming bandwidth by running two camera streams simultaneously.

#10

Any word on the ability to switch between cameras?

#11

I have this exact code running just fine with the occasional non fatal error.

public void robotInit() {
    	
    	Thread t = new Thread(() -> {
    		
    		boolean allowCam1 = false;
    		
    		UsbCamera camera1 = CameraServer.getInstance().startAutomaticCapture(0);
            camera1.setResolution(320, 240);
            camera1.setFPS(30);
            UsbCamera camera2 = CameraServer.getInstance().startAutomaticCapture(1);
            camera2.setResolution(320, 240);
            camera2.setFPS(30);
            
            CvSink cvSink1 = CameraServer.getInstance().getVideo(camera1);
            CvSink cvSink2 = CameraServer.getInstance().getVideo(camera2);
            CvSource outputStream = CameraServer.getInstance().putVideo("Switcher", 320, 240);
            
            Mat image = new Mat();
            
            while(!Thread.interrupted()) {
            	
            	if(oi.getGamepad().getRawButton(9)) {
            		allowCam1 = !allowCam1;
            	}
            	
                if(allowCam1){
                  cvSink2.setEnabled(false);
                  cvSink1.setEnabled(true);
                  cvSink1.grabFrame(image);
                } else{
                  cvSink1.setEnabled(false);
                  cvSink2.setEnabled(true);
                  cvSink2.grabFrame(image);     
                }
                
                outputStream.putFrame(image);
            }
            
        });
        t.start();
Switch cameras after Sandstorm
#12

To reduce bandwidth, and if only one camera at a time is needed, the following approach seems to work without errors:

  1. Create 2 Usbcamera using new (not startAutomaticCapture)
  2. Create a single sink and source
  3. Use cvSink.setSource to change cameras as needed.

When I look at the Driver Station variables, it looks like only a single stream is created, and only the Current View shows as Camera option

Abbreviated code:

UsbCamera cam0 = new UsbCamera ("USB Camera 0", 0);
cam0.setResolution(320,240);
cam0.setFPS(20);
UsbCamera cam1 = new UsbCamera ("USB Camera 1", 1);
cam1.setResolution(320,240);
cam1.setFPS(20);
 CvSink   cvSink   = CameraServer.getInstance().getVideo(cam0);
 CvSource cvSource = CameraServer.getInstance().putVideo("Current View", 320, 240);
 while(!Thread.interrupted() && !cameraKillf)
 	{
   	//Grab image
   	cvSink.grabFrame(source);

    //If cam switch button pressed, switch cams
    //choose either 
    // cvSink.setSource(cam1);
    // cvSink.setSource(cam0);
    
    //Display result to screen
  	cvSource.putFrame(output);
 	}
#13

thecoopster20

Does your posted code still work under the new WPILib 2017.3.1? We have been having problems on Camera switching that worked fine until the most recent lib version, and when tried your code, I am getting the same problem (driverstation keeps resetting). Thanks.

#14

I was able to get the posted code working on our robot, but I had to add delays in between the things done before hitting the “while” loop. I haven’t gone back to see which of the delays solved the problem or to see how short a delay will work. My clue that delays might help was that the code ran beautifully when I stepped through it in the debugger. Also, I was tethered to the roboRIO–haven’t tried it through the radio yet.

#15

The only delay that I’ve had to add was a short delay after the button press that switches the cameras.

#16

You think it would still work for switching between multiple cameras?
Make allowCamx disable all cameras but x? Right now I got 4 cameras all plugged into a jetson. Now ill just have to figure out how to implement it within the vision processing. Probably wouldn’t have to change much.

#17

I was thinking of this. I’ll try testing it later


public void robotInit() {
    	
    	Thread t = new Thread(() -> {
    		
    		boolean allowCam1 = false;
            boolean allowCam2;
            boolean allowCam3;
            boolean allowCam4;
   
    		UsbCamera camera1 = CameraServer.getInstance().startAutomaticCapture(0);
            camera1.setResolution(320, 240);
            camera1.setFPS(30);
            
            UsbCamera camera2 = CameraServer.getInstance().startAutomaticCapture(1);
            camera2.setResolution(320, 240);
            camera2.setFPS(30);
            
            UsbCamera camera3 = CameraServer.getInstance().startAutomaticCapture(2);
            camera3.setResolution(320, 240);
            
            UsbCamera camera4 = CameraServer.getInstance().startAutomaticCapture(3);
            camera4.setResolution(320, 240);
            
            CvSink cvSink1 = CameraServer.getInstance().getVideo(camera1);
            CvSink cvSink2 = CameraServer.getInstance().getVideo(camera2);
            CvSink cvSink3 = CameraServer.getInstance().getVideo(camera3);
            CvSink cvSink4 = CameraServer.getInstance().getVideo(camera4);
            CvSource outputStream = CameraServer.getInstance().putVideo("Switcher", 320, 240);
            
            Mat image = new Mat();
            
            while(!Thread.interrupted()) {
            	
            	if((Robot.oi.joySecondary.getRawButton(5)) {
            		allowCam1 = true;
            	}
                if((Robot.oi.joySecondary.getRawButton(6)) {
            		allowCam2 = true;
            	}
                if((Robot.oi.joySecondary.getRawButton(7)) {
            		allowCam3 = true;
            	}
                if((Robot.oi.joySecondary.getRawButton(8)) {
            		allowCam4 = true;
            	}
            	
                if(allowCam1){
                  cvSink2.setEnabled(false);
                  cvSink3.setEnabled(false);
                  cvSink4.setEnabled(false);
                  cvSink1.setEnabled(true);
                  cvSink1.grabFrame(image);
                } else if(allowCam2) {
                  cvSink1.setEnabled(false);
                  cvSink3.setEnabled(false);
                  cvSink4.setEnabled(false);
                  cvSink2.setEnabled(true);
                  cvSink2.grabFrame(image);     
                } else if(allowCam3) {
                  cvSink1.setEnabled(false);
                  cvSink2.setEnabled(false);
                  cvSink4.setEnabled(false);
                  cvSink3.setEnabled(true);
                  cvSink3.grabFrame(image);     
                }else if(allowCam4) {
                  cvSink1.setEnabled(false);
                  cvSink2.setEnabled(false);
                  cvSink3.setEnabled(false);
                  cvSink4.setEnabled(true);
                  cvSink4.grabFrame(image);     
                }
                
                outputStream.putFrame(image);
            }
            
        });
        t.start();
#18

Oh I forgot to change the other values to false.


    while(!Thread.interrupted()) {
            	
            	if((Robot.oi.joySecondary.getRawButton(5)) {
            		allowCam1 = true;
                    allowCam2 = false;
                    allowCam3 = false;
                    allowCam4 = false;
            	}
                if((Robot.oi.joySecondary.getRawButton(6)) {
            		allowCam2 = true;
                    allowCam1 = false;
                    allowCam3 = false;
                    allowCam4 = false;
            	}
                if((Robot.oi.joySecondary.getRawButton(7)) {
            		allowCam3 = true;
                    allowCam1 = false;
                    allowCam2 = false;
                    allowCam4 = false;
            	}
                if((Robot.oi.joySecondary.getRawButton(8)) {
            		allowCam4 = true;
                    allowCam1 = false;
                    allowCam2 = false;
                    allowCam3 = false;
            	}
#19

What happens if you don’t have that delay built in?

Wouldn’t holding the button down in the code you mentioned rapidly toggle the allowCam1 switching variable back and forth as long as the button is pressed? Could that “spam” the camera switching code to the point where it gets angry with you?

I think what you’re looking for is an alternating “one-shot” action that only toggles once until you release the button and press it again. This would guard against that “spamming” condition above.

Perhaps something like the following would be useful?

if (not camchange_sw) camswitchlock = 0;

if ( (camchange_sw) && (camswitchlock == 0) )
{

  allowCam1 = !allowCam1;

  camswitchlock = 1;

}

#20

Unfortunately I tried to deploy this tonight (basically a copy/paste) so our robot could switch between two LifeCams, but I kept getting fatal errors.

Strangely, it did work once, the first time. But most of the time, I got an error that would cause the robot code to reset, over and over again.

I kept seeing an OpenCV error: Assertion failed (!fixedSize() something or other)

Has anyone else seen the same thing?

In the meantime, I’m just creating two UsbCamera instances, doing the startAutomaticCapture() call on both, and setting up the dashboard to show both feeds simultaneously. We dropped the resolution down to 160x120 which, while crude, gives the necessary views without taking up a lot of bandwidth. Still, we’d really like to get this switching code working.