PhotonVision 2021 Official Release

After months of hard work and beta testing, the PhotonVision team is excited to announce the first full release of PhotonVision! With a focus on robustness and usability, this feature-packed release is a big step towards making vision processing accessible to all teams by improving ease of use and decreasing cost. With everything from GPU acceleration (on supported devices), a Raspberry Pi image, and a custom vendor dependency, to a reimagined web dashboard and vendor support on devices like the Gloworm, PhotonVision is ready to take on the 2021 game. Our most important features are detailed below:

GPU Acceleration

  • PhotonVision can offload large portions of processing to the GPU. This allows it to run 2x faster than similar software running on the same Pi 3/Pi Compute Module 3 hardware. This release unlocks the following performance improvements on your Gloworm or Pi 3:
Resolution PhotonVision on Gloworm Limelight
320 x 240 90 FPS 90 FPS
640 x 480 85 FPS Unsupported
960 x 720 45 FPS 22 FPS
1920 x 1080 15 FPS Unsupported

Note: to consistently get the above performance at greater than 480p you’ll need to have well-tuned HSV thresholds. Also, on the Pi Camera v2 PhotonVision can reach up to 120 FPS at 320x240.

PhotonLib

  • Vendor library makes interfacing with your camera as simple as creating a PhotonCamera and getting the latest result – no more remembering confusing NetworkTables keys
  • SimPhotonCamera allows teams to accurately simulate and unit test the vision component of their robot code with the WPILib simulator
  • Complete code examples and example projects make getting up and running easy
  • PhotonUtils, included in the vendor library, abstracts common functions like distance estimation, as well as utilities for estimating target and robot pose without SolvePNP

Vendor Support

  • Vendors like Gloworm have images that “just work” on their devices with no user configuration needed
  • Raspberry Pi-based devices support LED control over hardware PWM

PhotonVision Usability Improvements

  • Comprehensive documentation on installation, pipeline tuning, and robot code (with full examples for common uses)
  • A premade Raspberry Pi image makes setup fast and simple; just download the image and flash it onto an SD card
  • Offline access to documentation and code examples means easy access on-field
  • Completely reimagined user interface allows you to view more information at once; every element has a helpful ‘hint’ that shows up when you hover over it
  • Take snapshots on-field and view them later
  • Supports streaming thresholded and color streams simultaneously
  • Settings can be exported to a zip backup and later imported
  • Key metrics such as memory usage and CPU utilization can be viewed from the web interface
  • Test mode (–test-mode) (seen below) makes trying out PhotonVision as simple as running a JAR file (no camera required)

The PhotonVision team is excited to bring a new level of accessibility and ease of use to FRC vision processing. If you have a Raspberry Pi laying around we encourage you to flash the image onto it and try out PhotonVision. If your team is looking for a more “plug and play” experience, then you should check out the Gloworm, which can run PhotonVision. And if you just want to follow along (or contribute to the project) then we’d love for you to join our Discord!

–The PhotonVision Team

38 Likes

If this is your first time playing with computer vision in FRC, we’ve got you covered with complete pipeline tuning docs, which cover everything from HSV filtering and target grouping to camera calibration.


12 Likes

Those FPS numbers are spicy. Do you have pipeline latency measurements as well, and how does that compare to the camera-sensor-to-start-of-pipeline latency?

Quick Pipeline Latency samples I just ran on a Pi3b and a v1 camera:

Resolution 3D (solvePNP) 2D
320 x 240 4ms 2ms
640 x 480 Untested 5ms
960 x 720 Untested 10ms
1920 x 1080 25ms 21ms

Still in the works: doing a full, end-to-end “photons to NT target” latency checkout.

5 Likes

Gloworm image available here: Release PhotonVision v2021.1.3 (Gloworm 023434d) · gloworm-vision/pi-gen · GitHub

4 Likes