Low light and low framerate with Photonvision

Technical PhotonVision
#1

Hi
I’m trying to use PhotonVision, and first I am trying to set up the PiCam. However, the image is incredibly dark, no matter what resolution I use. I saw in other places stuff like “oh, framerate is too low so not enough light gets in”. This is not the case here, the problem persists at whatever framerate. Changing the exposure helps slightly, but the final is still way too dark. And yes, I am in driver mode to get raw footage. Also, everything is purple. I expect that with exposure at the max, this image should be very bright (see the USB cam image following for example)

picam
picam1894×1080 655 KB

I know that this is an issue with Photon specifically because when I use other camera software such as raspistill or just an online webcam test, the image appears perfectly fine, at a reasonable framerate.
Is there any solution to this problem?

Also: I have a completely different issue with using a USB camera (this one). In this case, the exposure is fine but the FPS (in driver mode) is awful. It runs at about 2 frames per second. All the resolution settings, including mjpg or yuyv run the same framerate. Again, I tested this using other software on the Pi and it works fine there.

usb
usb1885×1065 696 KB

These two problems are making it impossible to use vision, so any help would be greatly appreciated.

We are using a Pi 3B+

#2

This is a known behavior of the custom picam driver photon uses, and there are a few open issues to correct the default behavior.

Try dropping the gain down to about 18 or so. That should get you closer to normal color balance.

#3

Ok, that is good to know. I guess that works for us since we will be doing reflected light on that cam. We thought that this might be a symptom of a larger problem, but if this is it that is fine.

Anything on that USB camera?

#4

That one I am less certain on. I haven’t tried out that camera yet.

When changing the resolution or stream type settings, does it visibly change the stream? I’d be interested to see the .zip created by exporting settings . It should have some log files, and I’m wondering if there are any errors in it.

#5

Changing the resolution does change the image. However, I identified why the FPS was low on the camera. A note with it said “Changing the light level will change the FPS”. We thought this was a mistranslation, but apparently it is true. An exposure that is lower is the only way to get good framerate. Honestly though that makes no sense since it worked fine in other software using both good FPS and good resolution…

We tried to use a Lifecam instead, and it gives good framerate in reflected light mode. However, when used for shape tracking it drops down to 7 FPS. This is with all other cameras disconnected, so it should be a bit faster right? It needs to be able to chase balls in real time.

#7

Even with Driver Mode on, so no processing, the LifeCam is only at 7 fps (at 160x120). The PiCam is at 100 FPS doing the same thing.

Shouldn’t just showing the lifecam stream always be at 30 fps, since it doesn’t have to go through any processing?

#8

I’m sorry, I’m really not sure what’s going on. The behavior you describe doesn’t align with what I’ve seen on my pi3 with picam’s or different USB cam’s.

Inspecting the log files is probably the best way to see if there’s an obvious issue you’ve exposed. Reaching out on the discord server may get more immediate attention.