Our current neural network looks pretty promising. It can handle most lighting conditions and only starts to get iffy in direct sunlight. This is running on a Raspberry Pi 4 with the Coral USB accelerator and Microsoft HD-3000 webcam. We can achieve roughly 30 fps with 75% CPU usage on one core.
We trained on all 5000 images provided by FIRST for 500 epochs. However, we did not use the provided training scripts and instead opted to write our own. It was based on the demo object detection scripts provided by Coral and modified to use our dataset.
I still have some stuff to polish up, but I’m planning to release our training image and dataset. If you know how Docker works lower down, the image is on Docker Hub.
We actually utilize a system called zebracaptcha for when the students sign in and out of the lab. They have to identify the correct field element - we’ve found that it helps cut down on student abuse of the sign in system.
As many as there are of the balls… fuel… boulders… power cells… whatever word I can use to call them without being yelled at by the mods… again.
Resolution is good… I haven’t heard we need anything better. We are using the images and not the provided data or training code. A white paper at the end of the season on what we are doing is probable… as always.
Following this up with actual data, for whatever reason the blue high goal is underrepresented in the images we’ve been able to find. The weird thing is that there are like 50% more labeled low blue goals than high goals. The left red loading bay graphics are also low compared to the center and right graphics. Red loading bay middle graphics are a bit lower than expected as well, so it looks like there’s just a lack of the right kind of images from that particular DS wall.
We haven’t been 100% thorough in labeling every image, but even looking to fill in those gaps those are the ones which stand out as being unlucky (I’m assuming there’s not a conspiracy against them ).
Resolution was fine - and honestly, we probably should have downsampled the really large field images released by FIRST before using them.
Only nit I’d have would be to make the aspect ratio match what teams would typically get from a web cam - 16:9 in particular. I’m not sure it is hurting us, but if possible seems like it might be better?
Are you still running on the FRC OS provided by WPILIB, and using the script from WPILIB? We’ve done something similar, where we trained the model on our own using an alternate method, but we’re having significant issues with FPS. ~3 frames per second.
All of the code is heavily based on the object detection example provided by Google Coral and modified to use our dataset. We are also using the Google Coral USB accelerator, rather than just CPU which reduces our inference time from ~200ms to ~6ms. Are you just using the CPU on the Raspberry Pi or something else?
When you’re using neural networks how do you make sure that the speed is not slow? For me, my computer is doing .14 frames a second. I’m currently attempting to detect balls by sending each frame and analyzing the contour and the color using opencv. Thanks!
).