Toggle Between Two Cameras in Vision Processing

[Greg McKaskle]

Good news and bad.

The vision team now has a reproducible case of this. It only occurs on the ARM linux target with certain cameras.

Unfortunately, the camera that shows it most easily is the MS 3000. I had four camera types and didn't do all combos of two, but did enough to see that some worked and some resulted in the internal error in Start. I saw it fail with two MS 3000 and one MS3000 and one high end Logitech. I saw it succeed with one MS3000 and a lower end logitech. And saw it succeed with two Logitech cams.

My only suggestion at this point is a workaround of trying there models or makes of camera. Sorry. Thanks for reporting the bug. They should have it fixed for an upcoming release. But that will probably be after the season ends.

Greg McKaskle

[Greg McKaskle]

I have an update. The error message is occurring when the addition of this camera stream will overwhelm the USB bus. If the camera framerate or resolution are lowered, they claim that the error won't happen. It is a pretty lousy error message, and that is what they will try to improve.

This matches my earlier testing in that my earlier tests were with a lower res camera.

Please let me know if this doesn't solve your issues.
Greg McKaskle

[PhilBot]

Originally Posted by Greg McKaskle

I have an update. The error message is occurring when the addition of this camera stream will overwhelm the USB bus. If the camera framerate or resolution are lowered, they claim that the error won't happen..
Greg McKaskle

Hmm

Do you mean that this is error occurs if the system calculates that the total stream requirements are too high? In other words, it's a protective issue rather than an actual detected error condition?

I'm a little surprised, since my setup was only trying to run the cameras at 320x240, with a reduced frame-rate (15fpm) and reasonable compression (30).

Do you know if it uses the MAX throughput requirements, or the limited throughput based on the Camera's current settings?

The reason I ask is that I would think that two 320x240 streams would have less throughput requirements than a single 640x480 stream.

I guess I need to do a super limited test.

Phil.

[PhilBot]

OK, some updated information.

I can get both USB Camera streams to start if I go low enough in image size.

The requested frame rate or compression don't seem to have any impact.

If I set BOTH cameras to 160x120 they work....

However, if I put just ONE of them up to 320x240, the second stream does not start.

This is oddly annoying, since the system is quite capable of sending one stream at 640x480. If the system can handle an uncompressed 640x680 image (307K pixels), the you'd think it could handle two 320x240 images (153K pixels), or certainly a 320x240 image plus an 160x120 image (115K Pixels).

Thanks Greg, for the insight.

Phil.

[adciv]

Are you running at 30fps or 15 fps?

[PhilBot]

Originally Posted by adciv

Are you running at 30fps or 15 fps?

I tried a range of FPS. all the way down to 5 FPS with no apparent change to the Start error.

Is there anything magic about 15 FPS. I may not have tried that value: I did try 12,10,5 etc..

Update: I just tried 15 FPS. No change. Still fails with two 320x240 videos.

[adciv]

The MSHD3000 cameras natively support only two framerates. One is 30fps, the other is 15fps. In all other cases, the FRC LabView VIs are running the camera at a higher FPS and only taking "every so many" frames. I wonder if the cameras are both be running at 30fps and the VIs are just notifying at the lower rates. I'll have to dig out my code on how to set them at the lower fps.

The cameras, as is, have about 103 settings for video. This is a large combination of Frame Rates, Resolutions, and if the image is compressed or not. The FRC VIs don't directly show these and the native IMAQ one which does it is something like Imaq Get All Camera Resolutions.

[Greg McKaskle]

I believe that this is based on the camera mode at the time of start. It is also affected somewhat on the lower level drivers, as we tested this on an Intel embedded system and didn't see this error. So this is an error propagated from the lower level drivers saying that they will not start because they cannot or will not try to attempt the USB transfers.

I no longer have the cameras, since the other group wanted to borrow them. Please open the camera and set the resolution, framerate, and compression before doing the start. See if that will convince it to startup.

Please keep me updated and when I get the cameras back, I'll do the same.

Greg McKaskle

[PhilBot]

Originally Posted by Greg McKaskle

Please open the camera and set the resolution, framerate, and compression before doing the start. See if that will convince it to startup.

Greg McKaskle

Hi. That's how I was testing them. Although I was setting up cam 1 and then starting it, followed by setting up camera 2 and then starting it.

It occurred to me that if it's the start that is causing the problem, then the start and stop could be moved into the source switching loop, and the whole open/setup/close could be left out of the loop. This may shorten the time taken to switch cameras to a less noticeable pause.

Since I'm just discovering all this code, what is the purpose of the "notify" code in the error case? The original only checks the notification status of USB 0. Does this indicate when the camera is up and running again?

[PhilBot]

Some updated information.

I'm explicitly setting the FPS to 15, however, if I enumerate the Camera modes and display the current mode, I get mode 29, which is 320x240 at 30FPS. So there is a disconnect here somewhere.

[adciv]

I'm basing this on memory, so I may be a little off. The way the VIs select the camera mode is they enumerate all modes and then search for the first one which meets the resolution asked for. I suspect the 30fps modes come first. In order to select a 15fps mode, it may have to be done by explicitly selecting which mode you want to use and sending that to the appropriate IMAQ VI for setting up the camera.

[Greg McKaskle]

The code I just stepped through picks the smallest fps that is bigger than the requested image. With the Logitech I have plugged in, it lists 7 fps rates for 320x240.

Inside of WPI_CameraSet Framerate, you can prove the array of videoModes returned from the enumeration function and you can probe the videoMode sent into the GetSafe function on the right. Use the index sent into getSafe to look at the array description.

Greg McKaskle

[GuyM142]

Was there any significant progress by now?

I have tried to use the exact same code earlier this season and got the same error on the Open vi.
We now open the camera which we want to use on each switch and the lag is very noticeable.
We use 2 Lifecams 3000, one for far shooting and one for close shooting and switching during a match makes the driver wait for the other camera to show an image.
We really need a solution for this!

Will using a different camera model fix this?

[sraque]

Originally Posted by Greg McKaskle

Good news and bad.

The vision team now has a reproducible case of this. It only occurs on the ARM linux target with certain cameras.

Unfortunately, the camera that shows it most easily is the MS 3000. I had four camera types and didn't do all combos of two, but did enough to see that some worked and some resulted in the internal error in Start. I saw it fail with two MS 3000 and one MS3000 and one high end Logitech. I saw it succeed with one MS3000 and a lower end logitech. And saw it succeed with two Logitech cams.

My only suggestion at this point is a workaround of trying there models or makes of camera. Sorry. Thanks for reporting the bug. They should have it fixed for an upcoming release. But that will probably be after the season ends.

Greg McKaskle

Which model Logitech cams seem to work with this? We have been having a couple flakey results with the Lifecams:

1. Exposure setting VI. Zero sets image pretty dark (but seems workable), 1-100 appears to all be the same.
2. Sometimes exposure settings "take" on boot-up, sometimes not. No rhyme or reason. With low sample size, restarting robot code from DS seems to fix this. I suspect a boot-up sequence issue with camera settings being set before camera is up and running. We tried implementing a button to set the exposure again to Zero during teleop...does nothing.

We saw this behavior with both 1 and 2 camera scenarios.

-Steve

[adciv]

Originally Posted by GuyM142

Was there any significant progress by now?

I have tried to use the exact same code earlier this season and got the same error on the Open vi.
We now open the camera which we want to use on each switch and the lag is very noticeable.
We use 2 Lifecams 3000, one for far shooting and one for close shooting and switching during a match makes the driver wait for the other camera to show an image.
We really need a solution for this!

Will using a different camera model fix this?

Please see this thread here for a beta project which may meet your needs.

http://www.chiefdelphi.com/forums/sh....php?p=1560711