LifeCam USBCamera changing settings from Java

Technical Programming
1

We are trying to use the LifeCam and the USBCamera class for our vision processing. One thing we need is the ability to control exposure and brightness so we get a good image going into our image processing pipeline.

So far I have been able to get an image on the SmartDashboard, provide a way to enter Exposure and Brightness values on the SmartDashboard and send them to the USBCamera class. The problem is, when I change the values, the image doesn’t change brightness or exposure. I know the image is updating because I see the motion in new frames.

Here is what we’re currently doing:

Startup:


USBCamera targetCam = new USBCamera("cam0");  // create connection to camera
NIVision.Image frame = NIVision.imaqCreateImage(NIVision.ImageType.IMAGE_RGB,0); // create frame buffer
targetCam.openCamera(); // open the camera connection
targetCam.startCapture(); // start the frame capturing process (internal to USBCamera)

Loop:


targetCam.getImage(frame);  // retrieve a frame from the USBCamera class
CameraServer.getInstance().setImage(frame);  // push that frame to the SmartDashboard using the CamServer class

The above works as expected, and is pretty cool to boot.

Now adding the brightness/exposure control to the loop
Loop:


int exposure = Preferences.getInstance.getInt("camExposure", 50);
int brightness = Preferences.getInstance.getInt("camBrightness", 50);

if (brightness <= 100 && brightness >=0)
   targetCam.setBrightness(brightness);

if (exposure <=100 && exposure >= 0)
   targetCam.setExposureManual(exposure);

targetCam.UpdateSettings();

updatedBrightness = targetCam.getBrightness();
SmartDashboard.putNumber("Current Brightness", updatedBrightness);

targetCam.getImage(frame);  // retrieve a frame from the USBCamera class
CameraServer.getInstance().setImage(frame);  // push that frame to the SmartDashboard using the CamServer class

The above updated code, with brightness and exposure still gives me updated frames from the camera, I can also change the brightness and exposure from the SmartDashboard and the getBrightness() will return the NEW value and display it on the SmartDashboard, so I know the USBCamera class THINKS the brightness is changing BUT the actual brightness and exposure of the video does NOT change.

Does anyone have any experience on getting this to work? Any hints, tips or suggestions? We may have to punt with the USBCam and switch to the Axis Cam, which would be a shame since the LifeCam is so compact and “simple”

2

and cheap. Haven’t found an axis camera for <$170. Got a lifecam for <$25.

Assuming I don’t have any fires to be putting out at our teams meeting tonight, this was actually going to be my priority to troubleshoot. First year back on the programming subteam for awhile and and first year ever using Java, but hopefully I’ll have something to report back tomorrow to help you out.

3

Robert,

I was actually able to get almost exactly what you have to work. The only difference is I made sure to update the camera before opening the camera and starting the capture.


public class Robot extends IterativeRobot {

    Preferences prefs;
    CameraServer server;
    USBCamera targetCam;
    public static int g_exp;

    /**
     * This function is run when the robot is first started up and should be
     * used for any initialization code.
     */
    public void robotInit() {

    	prefs = Preferences.getInstance();
    	g_exp = prefs.getInt("exp", 1);
    	SmartDashboard.putNumber("Exp", g_exp);
    	
    	targetCam = new USBCamera("cam0");
    	targetCam.setBrightness(5);
//     We actually still had trouble setting exposure. It didn't actually like values 0-100. We found that setting the brightness did enough though.
//    	targetCam.setExposureManual(g_exp); 
    	targetCam.updateSettings();
    	
    	SmartDashboard.putNumber("Brightness", targetCam.getBrightness());
    	
    	targetCam.openCamera();
    	
	server = CameraServer.getInstance();
        server.setQuality(50);

        server.startAutomaticCapture(targetCam);
    }

We then don’t get anything in the teleopPeriodic. In order to change the parameter for brightness (or exposure) you could change the value on the smart dashboard, click on the tab 2nd from the top on the left side of the driver station and click robot code restart (or whatever it’s called). You don’t need to do a full reboot.

Now if you find anyway to turn off the autofocus…that’d be handy.

4

THANK YOU THANK YOU THANK YOU!!! So frustrated with grip and this code might actually help us!

5

I spent more time on it last night and discovered a few things. I looked at the source code for the USBCamera class - it’s just a wrapper for the NIVision.IMAQ functions - that’s not a bad thing, but it is good to know.

Info:

The USBCamera constructors call ‘openCamera’ internally. You don’t need to explicitly call openCamera unless you explicitly call closeCamera.

UpdateSetting is automatically called by getImage() and getImageData() if one of the settings (exposure, brightness, fps, etc) had previously been changed. There is no need to call updateSettings() explicitly unless you are not using getImage() or getImageData().

The setBrightness() and setExposure() functions do nothing but set a USBCamera member variable. The settings are not sent to the camera/driver until the updateSettings is called (which will be the next time you call getImage() or getImageData())
[LIST]
The getBrightness() does nothing but return that member variable.

[/LIST]

It’s good to know what the functions are actually doing :wink:

Bottom line - I removed the openCamera() call and removed my calls to updateSettings() and relied on the USBCamera class to manage it. This works better, but not perfectly. I can adjust the brightness and see things change, but exposure seems to have no effect.

I took it a step further. I created my own USBCamera class and added a getExposure() and modified the getBrightness() to get the values from the camera/driver using the NIVision.IMAQ calls. This proved to me that the exposure and brightness ARE getting changed, AND I validated that I am NOT in AutoExposure mode. Unfortunately, I still cannot get the exposure changes to show any appreciable effect on the LifeCam image. Brightness goes from a decent image with Brightness = 0, to a white image with Brightness = 100. Exposure changes seem to have no effect.

Has ANYONE been able to get images out of the life cam, using ANY method, that comes anywhere close to being as dark as the sample images for the FRC vision processing? I’m beginning to think that the LifeCam is simply not adjustable to that low of an exposure.

6

See my followup. The openCamera() call is superfluous, it is opened when you created the object. Your use of setBrightness(), setExposureManual() and updatedSettings() is correct.

Were you able to get a good DARK image, or just an image that is good for a driver to look at? I want a DARK image that a driver would find unusable, but the image processing will be very happy about (because the retro-reflectors will show up nicely).

7

I just confirmed it is possible - I’m running on a Mac and I just installed “Webcam Settings Panel” from the app store. It allowed me to adjust the exposure to a BLACK image. It also seems to confirm that it is not doing any of this in software, but is actually querying the camera for it’s capabilities. The setting options are different between the LifeCam, my laptop iSight and my monitor’s iSight, for example, the LifeCam has a backlight compensation slider while the iSights show up as simply on/off options.

I also confirmed that the settings are saved in the camera (or at least in the driver) - quit the settings panel (so I know it wasn’t restoring the values) and unplugged the cam and plugged it back in and the image was the same as when I had unplugged it. This is all good news, now if I can just figure out how to use the USBCamera, NIVision.IMAQ or something on the RoboRIO to make these same adjustments, I would be happy.

8

CORRECTION - I THOUGHT I had closed “Web Cam Settings” - I hadn’t. IT was storing and re-storing the settings. I made sure I quit it and tested again. When I replugged the camera, it reverted to Autoexposure

9

It was dark enough that I don’t anticipate GRIP having a problem with it - based on configuring the settings similarly with the USB camera is plugged in to the laptop.

Using the microsoft software linked to in the other thread, it was very obvious that the camera can be set up to change a lot more than we have access to with the class. I could turn the autofocus off, mess with the white balance, and all sorts of things that were useful in making the retroreflective tape pop even more. Unfortunately, I think we’ll be limited to creating a good vision processing filter with just the brightness modified, though I’m going to keep messing with the exposure. But I am seeing the same thing you are - changing the exposure doesn’t seem to have any affect, in fact at one point that camera stopped updating and I had no idea until I waved my hand in front of it to test something else.

Good to know it’s just a wrapper…might be more investigating to do with the IMAQ functions.

10

Was this running on the roboRIO or on your desktop PC? What microsoft software?

11

https://www.microsoft.com/hardware/en-us/d/lifecam-hd-3000 was the software - with the camera plugged into the laptop. The settings did not appear to get saved to the camera much as you saw from the webcam settings panel.

12

Thanks for posting this code. One question that I have is how can you use GRIP when you have this code running in the robot. We have included similar code to turn off auto white balance and to turn off auto exposure in Java using the USB camera class. However, we have found that GRIP will not run if we have opened the USB camera in our java robot code. So then we tried closing the camera when we were done setting up the camera, but it seems like it does not retain the settings in this case. Is there some way to change the settings in the USB camera and then run GRIP without losing your changes?

13

The Javadoc notes that exposure and brightness go from 0 - 100 are completely wrong. I forget what the actual range is but this issue frustrated a room of students and mentors for most of a Saturday until we played with the MS software for the LifeCam which actually presents sane values for the 3 different settings. Brightness is something like -15 to 4 (don’t trust that number, I’m going from rough memory) but not 0-100 at all. None of them documented to be 0-100 actually are.

14

I think it was mentioned in another post that the values actually go from 0-20,000. if that is the case, then using command will only max the value at 100/20,000.

15

Using the NIVision IMAQ commands, the min/max values returned are 5 and 20,000 respectively. I duplicated the USB Camera class (too bad they made all the variable private instead of protected) and replaced the setExposureManual() with code that allowed me to set the value explicitly. It turns out that anything above about 40 or 50 will give you quite a bright image. I’m actually using 10 I think for the exposure and 10 for the brightness.

As for losing the settings - as long as you don’t power the camera off, you will not lose the settings. It’s tricky, but you can use Robot code to configure the camera, then disconnect, then run GRIP and it will have the setting you just set. There are some real caveats in all this and I think I finally have all the cases worked out. I’m probably going to post the code when I get it working.

Caveat #1 - The robot code cannot use CameraServer - if it does, GRIP will not be able to publish a stream. Unfortunately, while you CAN disconnect from the USBCamera, you CANNOT kill the CameraServer stream without rebooting the robot or manually killing the robot code.

Caveat #2 - don’t forget that during development, if you use your robot code to set the settings, then launch GRIP, if you reload robot code, GRIP is still running and your robot code will throw an exception trying to connect to the USBCamera - you have to either reboot your roborio or kill the GRIP process. I’ve written code to kill the GRIP process if the USBCamera open() method throws and exception.

16

Just wanted to say thanks for reporting your findings on all of this. I’ve been busy taking care of some other mentor duties, and this is a life saver in allowing some students still get some help in trouble shooting what is going on with limited software experience

17

When you say you can use the Robot code to configure the camera, then disconnect, then run GRIP, what do you mean by “then disconnect”? Does the mean calling the CloseCamera() function?

18

I don’t suppose anyone has done something similar in C++? I tried to translate it but it crashes the rio.

19

I gave up trying to get image processing working on the roboRIO. I ended up having really bad problems - either GRIP or the robot code was crashing and generating a core dump (a HUGE file). This core dump was taking up all the storage space on the roboRIO which in turn would prevent the robot code from launching (you’d see exceptions that it couldn’t write certain files because there was no room left on the device). I ended up moving everything to a Raspberry Pi. We now have a very stable system that works quite well and gives us all the control we need. I intended to post detailed documentation when I get a chance.

As for doing the above in C++, I did get that part of the process working in Java. C++ should be quite similar. Where is your code crashing?

20

Our code isnt crashing. But we get a whitewashed image. I can adjust cam settings in the ms util, but they dont reliably stick. With the brightness turned down, we have great results. But a random number of power cycles or code reloads, the settings revert.