Good check with the Raspistill - I concur, the camera hardware looks reasonably functional.
Can you screenshot the settings page? What software version does it indicate is running, and what camera hardware?
Also, if you are able to post the .zip produced by “export settings”, there are some log files we could look into.
Strange… It’s possible that the white balance gain is out of whack; what happens if you adjust the “Gain” slider? Another possibility is that (for some reason) the GPU acceleration has stopped working. Have you updated your Pi recently? We currently don’t support Bullseye because that version broke MMAL for us.
If you could go to settings and click “Export settings” and upload the ZIP file here then that would help a lot too. A screenshot of the settings page @gerthworm asked for would be great too.
Here is a screenshot of the settings page:
See attached ZIP for a settings export. photonvision-settings.zip (23.0 KB)
I did have Bullseye on this Pi, but I am running the PhotonVision image now.
Zero gain*:
100 gain*:
Maybe related, but maybe not… Why is the image cropped so much? Notice how much more visibility there is in the shot from raspistill.
The image is cropped because raspistill is capturing in a different “video” mode. The different video modes have different fields of view both because of sensor binning and on-camera cropping. I’ll take a look at your logs in a bit.
I talked a bit with Matt - the gain slider was previously not hooked to the expected camera parameters. Once it got hooked up correctly (fairly recently), the behavior changed.
Current behavior limits exposure time based on the framerate. Choosing a higher resolution in the UI should reduce the framerate, which will help get the higher exposure back (based on your pipeline names, sounds like you’re looking to detect cargo?). An upcoming release should add the ability to reduce framerate at lower resolutions too.
In general, I try to keep the brightness slider at or near 50%, that should prevent some of the grey haze you’re seeing in the images.
I don’t think that’s it. When I use Driver Mode I’m getting >100 fps but the image is still not working. I have the resolution at the lowest possible value.
I tried flashing a fresh copy of the PhotonVision Pi image and I’m still getting same results.
Sorry, I think I misunderstood. You wanted me to choose a higher resolution, not lower. There are three options: 320x240 (see screenshot above), 640x480 (did not improve image), and 1920x1080. At the 1920x1080 the stream froze. I tried to change it to a lower resolution but the UI did not do anything. I refreshed the browser but it would not load fully. I attached another export.
photonvision-settings (2).zip (7.9 KB)
Sounds good, thanks.
The frozen UI I’ve seen from more than one root cause. One has been the browser itself - some PC/browser combos were not happy about having a full 1920x1080 mjpg stream thrown at them, and bog down to the point where they stop rendering.
There’s at least one stack trace in there I know I could go dig into:
[2022-01-13 22:54:15] [VisionModule - VisionModuleChangeSubscriber - mmal_service_16.1] [ERROR] Failed to invoke camera settable method: setVideoModeIndex
[2022-01-13 22:54:15] [VisionModule - VisionModuleChangeSubscriber - mmal_service_16.1] [ERROR] java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.photonvision.vision.processes.VisionModuleChangeSubscriber.onDataChangeEvent(VisionModuleChangeSubscriber.java:170)
at org.photonvision.common.dataflow.DataChangeService.dispatchFromQueue(DataChangeService.java:64)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException
at org.photonvision.vision.camera.ZeroCopyPicamSource$PicamSettables.setVideoModeInternal(ZeroCopyPicamSource.java:168)
at org.photonvision.vision.processes.VisionSourceSettables.setVideoMode(VisionSourceSettables.java:58)
at org.photonvision.vision.processes.VisionSourceSettables.setVideoModeIndex(VisionSourceSettables.java:75)
... 7 more
[2022-01-13 22:54:17] [General - ConfigManager] [DEBUG] Saving to disk...
[2022-01-13 22:54:17] [General - ConfigManager] [INFO] Settings saved!
[2022-01-13 22:55:02] [General - ConfigManager] [DEBUG] Saving to disk...
[2022-01-13 22:55:02] [General - ConfigManager] [INFO] Settings saved!
[2022-01-13 22:55:15] [General - ConfigManager] [DEBUG] Saving to disk...
[2022-01-13 22:55:15] [General - ConfigManager] [INFO] Settings saved!
[2022-01-13 22:55:44] [VisionModule - VisionModuleChangeSubscriber - mmal_service_16.1] [ERROR] Failed to invoke camera settable method: setVideoModeIndex
[2022-01-13 22:55:44] [VisionModule - VisionModuleChangeSubscriber - mmal_service_16.1] [ERROR] java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.photonvision.vision.processes.VisionModuleChangeSubscriber.onDataChangeEvent(VisionModuleChangeSubscriber.java:170)
at org.photonvision.common.dataflow.DataChangeService.dispatchFromQueue(DataChangeService.java:64)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException
at org.photonvision.vision.camera.ZeroCopyPicamSource$PicamSettables.setVideoModeInternal(ZeroCopyPicamSource.java:168)
at org.photonvision.vision.processes.VisionSourceSettables.setVideoMode(VisionSourceSettables.java:58)
at org.photonvision.vision.processes.VisionSourceSettables.setVideoModeIndex(VisionSourceSettables.java:75)
... 7 more
Additionally, I do have the symptom replicated on my end.
1920x1080 - framerate is slow enough for reasonable illumination -
320*240 - framerate too high to keep the shutter open long enough to get good illumination.
There’s two related issues that I think you’re being hit by:
FWIW most of us have been testing with picam v1.3’s - not to say for sure you should switch, but just be aware of some of the image binning implications of the v2
I updated two issues to help track this:
Additionally - and this is a bit of a hack but to get you going for now… do you have a USB webcam you could try? We’ve gotten good reports from those doing colored shape detection.
Thanks @gerthworm. I’ll sit tight in the hopes the open source community will come to the rescue.
I’ve had some good success identifying the colored balls with my laptop webcam and we’re going to try using a Limelight with PhotonVision tomorrow. We have a couple ELP cams on order, so maybe we’ll have better luck with those on the Pi.
We are planning to use attached USB life cam for balls and limelight for retro reflective tape. That camera worked well for balls through the limelight USB
@gerthworm @pietroglyph Are there any instructions for building PhotonVision? I was going to poke around, but when I run gradlew build I get an error related to a native lib:
A problem occurred configuring project ':photon-lib'.
> Exception thrown while executing model rule: Photon(org.gradle.nativeplatform.NativeLibrarySpec) { ... } @ photon-lib/build.gradle line 52, column 9 > all()
> Could not create an instance of type edu.wpi.first.nativeutils.dependencies.DelegatedDependencySet.
> java.lang.NullPointerException (no error message)
Build Instructions - PhotonVision Docs is the instructions for the main monorepo. The Picam driver repo is separate and has it’s own instructions… I believe it’s best built directly on a Pi?
FWIW I also ran in to the same thing but never traced it down, as I thought it was a fluke (I was building on a Jetson, not an x86 PC). It seems to work fine in CI at the very least, so it points to something different between CI and a “local” environment.
We’ll look in to it.
To be sure, what platform were you building on? Windows/Linux x86, Jetson, RPi?
I tried first on a RPI and then on Windows 11 (with WPILib tools) and I got the same error on both.
I’ve been unable to reproduce this issue on Windows 11 or WSL Ubuntu 18.04, so I’m not sure what else to try. It seems like the WPILib NativeUtils tool has an issue, but only in certain environments. I talked to @Thad_House on the FRC Discord and he will be taking a look next week.
I pulled the latest and successfully ran gradlew buildAndCopyUI. I was running gradlew build. Maybe that target does some setup I was missing.
Thanks for pointing me at the build documentation, I missed it. I was looking in the repo at the README thinking it would be there.