I am the senior mentor for Team 4464, working with the team to help them implement RobotPy. They decided that they wanted to use photonvision and were able to get the coprocessor up and running with no issues, but the library used to access the photon cameras do not seem to be complete. I noticed the talk about this earlier and was wondering if we would have to build our own wrapper for photonvision to make this work. Does anyone have photonvision working with python?
I think @gerthworm made photonlibpy
1 Like
There are a few teams out there according to my github search.
I believe that gerthworm’s team is 1736 Robot Casserole…but I’m sure they can confirm.
We’re not quite there yet, hopefully by this weekend we’ll have something.
1 Like
Looks to be correct. Thanks.
Visited their space. There is a wrapper there that looks great.
Yup
Yup.
A bit of history and future direction…
photonlib is the PhotonVision-developed library in Java and C++ for interfacing PhotonVisioninto robot code.
In generally, robotpy’s design is to wrapper the C++ libraries into python. Historically, this is what has happened for wrappering photonlib into python.
In late 2023, it was discovered that opencv dependencies in photonlib were breaking this process. The robotpy team did not have bandwidth to address it.
The PhotonVision team had some bandwidth to address it, but did not have the skillset needed to resolve the robotpy build issues. Instead (Given the timeline) the path was chosen to create a new, pure python implementation of a minimum subset of photonlib, called photonlibpy (distinct name chosen to help avoid namespace collisions).
The PhotonVision team will continue to build and support photonlibpy in the CI pipeline until robotpy build issues are fixed. However, the library represents a step backward both in how photonlib and robotpy maintain parity with other languages through good build processes to avoid duplicate implementations. I do expect there will be some ‘bumps’ in the road in terms of keeping feature parity.
In 1736’s 2024 codebase, you can find how we’re using photonlibpy to read some basic info from the cameras. That wrapper is just how our team likes to interact with PhotonVision- one way among many.
Docs and other examples remain major TODO’s. Hopefully soon, with the main constraint being developer bandwidth given that it is indeed build season.
Thanks. I am going to have to take a look at this on the team’s system. When I run python scripts with photonlibpy on my personal system, I get one of two errors. One is an error on packet.py line 7, about the object not being subscriptable (my pyenv is usually set to version 3.8.3 for reasons I forget) or failure to find module cv2 from the later versions. I will sort that out later, it does not matter, as long as the team’s systems work.
While robotpy supports older pythons, it’s recommended to use python 3.12 on your machine as the Rio always uses 3.12. At least pyenv local makes it really easy.
pip install opencv-python will make OpenCV available locally.
The local install of opencv-python fixed me up. I was also naive enough to believe that I could just import photonlibpy and access the members. I had to follow the wrapper at 1736 and use the whole line
from photonlibpy.photonCamera import PhotonCamera
to successfully import the class. Luckily we are never too old to learn.
1 Like
I did a bit of work on photonlibpy to implement the PhotonPoseEstimator. Right now, only the LOWEST_AMBIGUITY and MULTI_TAG_PNP_ON_COPROCESSOR strategies work, but that should be enough to do effective AprilTag pose estimation.