Thanks to the insanely hard work of the following Discord users (+ more), Photon has a MVP using YOLOV5s-based object detection on 2024 NOTEs on Orange Pi 5! This would not have been possible without everyone pulling together and cooperating on writing and debugging embedded code and figuring out how to train models from broken instructions.
The model above was trained on images from a collection of Roboflow NOTE datasets. A particular shoutout to nisala and the folks over at https://getbaseline.app/ who stepped up unsolicited to donate compute resources for model training and conversion. Our progress over even the last 12 hours wouldn’t have been possible without them, as well as the following Discord users:
alex_idk
moto moto
js & asid61 & craig for testing code
All code is public and licensed under the GNU GPL V3 like normal, and all models we train will be released likewise. Artifacts are published to our Maven server by the JNI repo and consumed by the main photon repo in that pull request.
Photon code: Add RKNN / Object Detection Pipeline by mdurrani808 · Pull Request #1144 · PhotonVision/photonvision · GitHub
RKNN JNI code: GitHub - PhotonVision/rknn_jni: Java wrapper around rknn converted yolov5 model
Model conversion is still something of a dark art known only by alex_idk at this point, but docs on that process are in flight. Stuff probably isn’t quite ready for general consumption, but as always drop by the discord and say hi! We are always happy to get more testers to find bugs for us.