Romi + PhotonVision + Startup Errors

Short Story:

I am trying to install the Photon Vision Client Portal to the Raspberry Pi I am using on the Romi and having multiple issues:

  1. The most frustrating symptom of this is that sometimes, for a very short while, the Photon Vision Client Portal recognizes and displays the camera feed. Then stops doing so after the next reboot and will not work again until I restart the install from scratch. And it will not show every time.
  2. Manually starting Photon Vision generates several errors in the log including:
    • Couldn’t load shared object photonlibcamera
      • At this point, I am pretty sure this is my immediate issue.
    • Could not deserialize apriltag layout!
    • Could not connect to the robot!
    • Failed to create or connect to Pigpio Daemon socket: Connection refused
    • Exception while setting DHCP!

Your mission, if you choose to accept, is to help me figure out what I am doing wrong.

Seriously? My post is too long? Ok…

Details:
I have followed the process below several times with what I think are minor differences. Each time leads to a final step of rebooting the Romi, seeing the WPILibPi Portal come up as expected and the Photon Vision Portal come up but not recognize the camera. Sometimes, intermediate steps do recognize the camera but then “loose” it on subsequent reboots. Here is my process.

Getting Started with Romi

  • Getting Started with Romi — FIRST Robotics Competition documentation
  • I really liked these instructions, they were well written and clear.
  • Substeps that were all successfully performed:
    • Download WPILibPi_64_image-v2023.2.1-Romi.zip for the image.
      • In previous attempts I used the 32 Bit version. That choice does not to appear to impact behavior
    • Flash SD
    • Update Firmware
    • IMU Calibration
    • Attach Life Cam and Rename
    • Update SSID

At this point I had an operational Romi that I could program and use and my basic Romi Functionality project runs. This always works.

The next steps are for installing Photon Vision.

Photon Vision Romi Installation - Preparation

  • Romi Installation - PhotonVision Docs
  • Another good set of documents. However, I have tried to follow all these instructions several times and it has not worked. So, I will say that I did exactly what they said except:
    • “The pi must remain writable!”
      • I missed this several times. I ended up executing these commands before doing anything else:
      • sudo nano /etc/fstab
      • Change:
        • PARTUUID=9ad66ae4-02 / ext4 defaults,ro,noatime
      • To:
        • PARTUUID=9ad66ae4-02 / ext4 defaults,noatime
  • I then made the change to
    • sudo nano /home/pi/runCamera
      • Although, why that sleep delay is 2.8 hours is not at all clear.
  • And rebooted.

At this point, the WPILibPi Portal came up fine, the system was already in writable mode and the camera stream was no longer available, all as expected.

Photon Vision Other Debian-Based Co-Processor Installation

And this may be where I am starting to go off the rails. Looking at the entire log, I see some things that look like errors:

Installing libopencv-core4.6 on aarch64
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package libopencv-core4.6
E: Couldn't find any package by glob 'libopencv-core4.6'
E: Couldn't find any package by regex 'libopencv-core4.6'

Rebooting at this point, the WPILibPi Portal comes up, the camera is connected there, but viewing the stream is disabled, as expected. Also, at this point the Photon Client is also up and running. And here is my issue.

There is no recognized camera.

So, I disabled the service and rebooted. The Photon Client did not come up as expected. I then ran:

  • /usr/bin/java -jar /opt/photonvision/photonvision.jar

So I could see the logs. The logs are below, but the first line is:

[2024-01-28 13:26:29] [Camera - LibCameraJNILoader] [ERROR] Couldn't load shared object photonlibcamera: /tmp/libphotonlibcamera.so: libcamera.so.0.1: cannot open shared object file: No such file or directory

I cannot find any documentation on photonlibcamera, but suspect this is what would load the camera.

So, this is where I am stuck.

Any suggestions?

Here are parts of the s the photonvision.jar log:

pi@photonvision(rw):~$ /usr/bin/java -jar /opt/photonvision/photonvision.jar
[2024-01-28 13:26:29] [General - Main] [INFO] Native libraries loaded.
[2024-01-28 13:26:29] [Camera - LibCameraJNILoader] [ERROR] Couldn't load shared object photonlibcamera: /tmp/libphotonlibcamera.so: libcamera.so.0.1: cannot open shared object file: No such file or directory
java.lang.UnsatisfiedLinkError: /tmp/libphotonlibcamera.so: libcamera.so.0.1: cannot open shared object file: No such file or directory
        at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
        at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2389)
        at java.base/java.lang.Runtime.load0(Runtime.java:755)
        at java.base/java.lang.System.load(System.java:1953)
        at org.photonvision.raspi.LibCameraJNILoader.forceLoad(LibCameraJNILoader.java:64)
        at org.photonvision.Main.main(Main.java:348)
[2024-01-28 13:26:29] [General - Main] [ERROR] Platform does not support RKNN based machine learning!
...
[2024-01-28 13:26:31] [Config - SqlConfigProvider] [ERROR] Could not deserialize apriltag layout! Loading defaults
...
[2024-01-28 13:26:31] [General - NetworkTablesManager] [ERROR] [NetworkTablesManager] Could not connect to the robot! Will retry in the background...
[2024-01-28 13:26:31] [General - PigpioSocket] [ERROR] Failed to create or connect to Pigpio Daemon socket: Connection refused
...
[2024-01-28 13:26:31] [General - NetworkManager] [INFO] Setting DHCP with team 0
[2024-01-28 13:26:31] [General - NetworkManager] [ERROR] Cannot manage hostname without root!
...

I am seeing something similar I think. I took my romi vision project from last year(2023) and updated it for 2024 wpilib. When I ran it got a crash saying photonlib versions do not match. So I updated photonvision on my romi raspberry pi. I can load the photonvision UI but there is no lifecam appearing.

[2024-02-02 18:47:03] [General - Main] [INFO] Native libraries loaded.
[2024-02-02 18:47:03] [Camera - LibCameraJNILoader] [ERROR] Couldn't load shared object photonlibcamera: /tmp/libphotonlibcamera.so: libcamera.so.0.1: cannot open shared object file: No such file or directory
java.lang.UnsatisfiedLinkError: /tmp/libphotonlibcamera.so: libcamera.so.0.1: cannot open shared object file: No such file or directory
        at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
        at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2389)
        at java.base/java.lang.Runtime.load0(Runtime.java:755)
        at java.base/java.lang.System.load(System.java:1953)
        at org.photonvision.raspi.LibCameraJNILoader.forceLoad(LibCameraJNILoader.java:64)
        at org.photonvision.Main.main(Main.java:348)
[2024-02-02 18:47:03] [General - Main] [ERROR] Platform does not support RKNN based machine learning!
[2024-02-02 18:47:03] [Camera - MrCalJNILoader] [INFO] Successfully loaded shared object libmrcal_jni.so
[2024-02-02 18:47:04] [General - Main] [INFO] Logging initialized in debug mode.
[2024-02-02 18:47:04] [General - ShellExec] [DEBUG] Executing "cat /proc/device-tree/model"
[2024-02-02 18:47:04] [General - ShellExec] [DEBUG] Got exit code 0
[2024-02-02 18:47:04] [General - Main] [INFO] Starting PhotonVision version v2024.2.2 on Linux Raspbian 64-bit (Pi PI_3)
[2024-02-02 18:47:04] [General - Main] [DEBUG] Loading ConfigManager...
[2024-02-02 18:47:04] [Config - SqlConfigProvider] [DEBUG] Using database /home/pi/photonvision_config/photon.sqlite
[2024-02-02 18:47:04] [Config - SqlConfigProvider] [INFO] Using correct database version: 2
[2024-02-02 18:47:04] [Config - SqlConfigProvider] [DEBUG] Loading config...
[2024-02-02 18:47:05] [Config - SqlConfigProvider] [ERROR] Could not deserialize apriltag layout! Loading defaults
[2024-02-02 18:47:06] [Camera - CameraConfiguration] [DEBUG] Creating camera configuration for UsbCamera Microsoft LifeCam HD-3000 (AKA Microsoft_LifeCam_HD-3000) at /dev/video0

...

[2024-02-02 18:47:13] [General - ShellExec] [DEBUG] Executing "nmcli -t -f GENERAL.CONNECTION,GENERAL.DEVICE,GENERAL.TYPE device show"
[2024-02-02 18:47:13] [General - ShellExec] [DEBUG] Got exit code 8
[2024-02-02 18:47:13] [General - NetworkUtils] [DEBUG] Found network interfaces:
[]
Exception in thread "DataChangeEventDispatchThread" java.lang.UnsatisfiedLinkError: 'boolean org.photonvision.raspi.LibCameraJNI.isLibraryWorking()'
        at org.photonvision.raspi.LibCameraJNI.isLibraryWorking(Native Method)
        at org.photonvision.raspi.LibCameraJNI.isSupported(LibCameraJNI.java:65)
        at org.photonvision.raspi.LibCameraJNILoader.isSupported(LibCameraJNILoader.java:77)
        at org.photonvision.common.configuration.PhotonConfiguration.toHashMap(PhotonConfiguration.java:145)
        at org.photonvision.server.UIInboundSubscriber.onDataChangeEvent(UIInboundSubscriber.java:46)
        at org.photonvision.common.dataflow.DataChangeService.dispatchFromQueue(DataChangeService.java:65)
        at java.base/java.lang.Thread.run(Thread.java:833)

Following up after asking on wpilib discord about this:

Peter | WPILib — Today at 10:48 AM
I missed the CD post.  The problem is the 2023 wpilibpi image is too old of an OS version to run PV and/or doesn’t have the libcamera library correct version installed
I have a PR in progress to update the pi image to 2024 but the web-based network configuration gui is broken with it and will take some time to fix

mine is like this:

Did you solve it? :sob:

Same boat, was excited to see an easy setup for Romi but looks like there’s more work to do here.

Reopening this old thread, since not much development has happened on Romi side, and the problems described here are still happening. I do see people refer to this thread in various forums (e.g., on the FRC Discord channel), so it looks like other people (like myself) are still trying to make this work.

So, after stumbling upon all the errors described here, and with the hint from loafdog’s last message + a little bit of info in the most current Photovision installation page (Romi Installation - PhotonVision Docs), it looks like the trick is to install a 2023.4.2 version of photonvision. I’m copying here the info I compiled responding to a question in the FRC Discord channel:

  1. When I run sudo ./install.sh -v 2023.4.2 it was failing with this error:
PhotonVision v2023.4.2 is not available
See ./install --list-versions for a complete list of available versions.

When I run ./install --list-versions it will indeed list versions only ranging back to some time mid 2024. But last night I managed to work around that and now I finally have photon vision working on the Romi. I’m adding notes how that worked out in case anyone else stumbles upon this:

  • Romi is imaged with WPILib version from with “_Romi” in the image name (the last, AFAICT) as per instructions here Imaging your Romi — FIRST Robotics Competition documentation
  • I’ve went through hoops and loops to enable the Bridge mode, since that seemed broken as well. DM me if you want my messy notes on this
  • I then got to installing the PhotonVision script, starting with downloading the script (while SSH-ed in the Romi’s PI):
$ wget https://git.io/JJrEP -O install.sh
$ sudo chmod +x install.sh
  • Here, before running the install script, I went in with sudo nano install.sh and commented out the following two lines. This is so it actually just downloads the real_install.sh and not delete it on the spot:
# Comment out these lines:
# ./real_install.sh "$@"
# rm ./real_install.sh
  • Finally, I went into the real_install.sh with sudo nano real_install.sh and commented out the lines (l286-l289) that check for the existance of the old package:
# if ! is_version_available "$VERSION" ; then
#  die "PhotonVision v$VERSION is not available" \
#      "See ./install --list-versions for a complete list of available versions."
# fi

FIRST Robotics Competition Documentation

Imaging your Romi

The Romi has 2 microprocessor boards: A Raspberry Pi that handles high-level communication with the robot program running on the desktop and, A Romi 32U4 Control Board that handles low-level motor …

So now running sudo ./install.sh -v 2023.4.2 again, skips the version check and happily installs the old version. Few more reboots and sudo systemctl restart photonvision.service later, the service finally came to life!! NOTE: The service takes the liberty to rename the hostname of the pi, so it is not accessible as http://wpilibpi.local or any custom hostname you might have had for it but with http://photonvision/

1 Like