Here’s a Jupyter notebook to train YOLO11 models and convert them to RKNN so they can be used by coprocessors with Rockchips, it is intended to be used with an Orange Pi 5.
This notebook is based on LaviRZ’s and LevyIshai’s YOLOv8 training notebook. I’m currently a high school sophomore in my second year with FRC Team 503, Frog Force, and as I’ve been working with vision, their work with implementing YOLO in FRC has really helped me learn a lot, especially since I started learning about computer vision last year.
In summary, this takes a Roboflow Universe dataset, trains a YOLO11 object detection model from scratch, and then converts it to RKNN so that it can be used on a Rockchip processor’s NPU. To use the notebook, just make a copy of it in Google Colab. Then you just have to find a YOLO11 dataset in Roboflow and paste the download code in the specified code cell. I would use a dataset with a fewer number of classes and larger amount of images. Then you can adjust certain arguments for training the model and exporting it. It’s only written to be used in Google Colab with a GPU. Colab has pretty short time limits for use with the GPU so make sure to finish training your model quickly.
One problem I noticed when initially using LaviRZ’s code was that he included code to fix directories in the data.yaml file of the Roboflow dataset that only worked with his dataset’s specific directory errors. I adjusted the code to work with any data.yaml file as long as it has train, val, and test image directories in it. This is something to watch out for if you have errors with training. Also, when downloading RKNN Toolkit to convert the model, the notebook didn’t account for losing local variables after the runtime has to be restarted, so I added code for that.
I also added a lot more yap in the notebook which explains a lot about how all of the code works, and includes a lot of background information for people who are new to computer vision and Python. When I was learning how all of this stuff worked, those resources would have been really helpful. This model also includes a link to a notebook that can run YOLO11 models on videos.
To actually deploy this model, I would recommend using LaviRZ’s and Levyishai’s RKNN fork of PhotonVision found here: High FPS object detection with PhotonVision fork - v1.0!. You need an Orange Pi 5, an SD card flashed with the PhotonVision fork’s image, an ethernet cord & power (or a robot). This model hasn’t been tested with this fork so I don’t really know if this will work or not. Try running this notebook and deploying this model, it would be really great to see teams use this model in the coming Reefscape season!
Also I apologize if I didn’t follow proper citations with the resources I used and linked:)
