I’m convinced machine learning for object recognition is the next big thing in FRC.
Does anyone have any current solutions for running machine learning on board? Just using a raspberry pi seems to net very poor performance. I know there are accelerators like Coral, but seems like you can’t get those.
Inference is still pretty slow on the edge if you don’t have dedicated hardware (GPU/NPU). The coral is great but like you said, it’s not really in stock.
Taking snapshots and doing inference on a single image might suffice in some situations but actively tracking a game piece would be difficult.
Disclaimer: I work for NXP
NXP has the i.MX 8M Plus which has an NPU and can do inference quite fast (simple object detection models can run inference at >100hz). There’s not a lot of commercially available products with the 8M Plus but I found this one on the web that looks like a nice solution: CuBox M - Tiny Multimedia Powerhouse | SolidRun
I’m not sure what the software support looks like for this one, typically with our products companies will leave it up to the user to create a Linux distribution for the board.
On my team, we have a board coming soon with this chip that is targeted for mobile robotics. It would be a nice set up for FRC - it runs Ubuntu 20.04/22.04, so you get ROS support. If some teams wanted to try it I could possibly put together some example applications and documentation. But I’m not sure we will be able to produce enough boards.
There are several teams attempting this. Searching CD will reveal that rPI4/Coral, rPI4/IntelNCSx, and Jetson Nano are the primary edge hardware solutions being evaluated. The ones that have tried / are trying this obviously acquired their TPU/GPU hardware during some earlier era of a functioning supply chain.
Not sure if you are already trying this and have gotten to the point that an rPI is working but with too low of a framerate for real-time robot use or if you are just reading that. If you haven’t really gotten into this yet, you might want to work on training and rPI deployment to gain experience and keep an eye on the market for one or more of the accelerators to come back in stock.
It will be interesting to see when or if ML object recognition becomes a big thing in FRC. Traditional vision processing may be more accessible, reliable, and robust for human-assist tasks in teleop for a while. Autonomous might be the first place where ML starts to offer a net advantage.
I don’t know a whole lot about this stuff, and I’ve never tried it in an FRC application, but I’ve been pretty impressed with the HuskyLens. Unfortunately I think the firmware is closed, but this crafty little device shows you the potential of wonderful things to come in this field. HuskyLens - AI Machine Vision Sensor | DroneBot Workshop
I’ve recently started to look into this, and I’m experiencing the same stock issues with Coral (which doesn’t even ship to Canada ) and Jetson Nano. I’m wondering if the Arduino Portena with a vision shield would be powerful enough to run inference? Probably not, but I’ll be taking a deeper look into this.