Photon and custom 3d targets

We are still just getting started with Vision in general, but I was wondering if it is possible, and if so, what is involved in adding custom targets to the 3d module of Photon Vision.

Currently, we are still working with mini-bots, so mostly the Romi will be our device of choice. It would be wonderful if we could put in some of the targets for our Game Design and the like to learn how to use the solvepnp in our pipelines.

I asssume, we would at least need to know the object’s size, shape, and height off the ground.

I totally understand if…
A, We are biting off more than we can chew, or
B, This is not possible at this juncture.

1 Like

PhotonVision actually used to support custom targets, but we removed it.

The main reason was that the user interface is tricky to get right; we supported uploading targets as CSV files but we would’ve had to document the format, or we would’ve had to create a much more complicated user interface. This was right before kickoff and our first release window, so we removed the custom target feature because we didn’t have time to flesh it out. Usability is one of our main goals, so we really like to avoid including half-baked features. We also thought that it seemed like a pretty niche feature (it’s just our luck that it’s something someone actually wants to use :slightly_smiling_face:.)

There are a couple of paths from here for you (in order of difficulty):

  1. Use an existing target model that’s already included. I think the target for the 2020 loading station is pretty simple and small (please correct me if I’m wrong here), so that could work well.
  2. Stick to 2D mode and trig-based distance estimation—if your target is above the height of the camera then 2D-distance estimation can be very easy and accurate, and you don’t need to make any special ‘target model’. To boot, PhotonLib will also do all the math for you here.
  3. Make a custom build of PhotonVision with your custom target model. New targets are pretty easy to add to the code (targets are currently defined here in the PhotonVision code); the difficult part is really understanding what points you need to put into your target model. This will probably require reading the OpenCV docs. You’ll also have to understand how PhotonVision picks target points for your specific target, so this’ll probably also require reading and understanding some PhotonVision code too.

If there’s more community interest for this feature we could also re-add it with a better UI. We’re nearing the end of the season, and after we finish adding the colored shape pipeline and docs for Limelight support we’ll probably make a small release. Then, we’ll probably take a look a roadmap for larger features to add in the offseason, and it’s possible that custom PnP target support could be included if there’s a lot of interest.

4 Likes

Thank you. That is exactly what I was looking for and incredibly helpful.

I totally understand. The fact that Photon works so easily on the Romi is incredible and opens many doors for us. I understand about the priorities.

Out of curiosity, If your target is below the camera height by a couple inches, is option 2 just as accurate?

The first piece for us would be a smaller ball, so that may be a good start for option 3 if we decide to go that route.