I’ve been working on developing system that integrates computer-vision based machine learning using the OpenCV AI Kit for FRC applications. I’m excited to finally share the results of my research publicly.
The full paper describing my research can be found here.
The dataset I used for training can be found on Supervisely.
Additional demonstration videos can be found on my YouTube playlist.
Huge thanks to @GabeHart17, @marshall, and the rest of team 900 for making their data public.
Also thanks to @AustinSchuh for sharing their implementation of SIFT.
This is a really great paper. Very thought provoking, and I must go back and read it again. It really does seem like machine learning is the new swerve drive, and it is thanks to efforts like this.
Although there are some ML posts on CD lately, I would be careful before crowning vision ML as the new swerve.
Remember that swerve gained it’s popularity thanks to success on the field + being a COTS item.
The next comment should be @marshall suggesting fully autonomous robot.
But there’s lack of data to model the environment (full scores breakdown from FMS please!; and some other stuff like proper robot model and game pieces/robots locations on the field).
Probably getting full scores breakdown is the easiest step (and actually would be useful for teams in many other areas). Do you know what prevents it from happening?
What I’m referring to is something like this: https://www.cs.colostate.edu/~anderson/res/rl/. You use repeated trials with carrot-and-stick methods and variations on gradient descent to converge on an optimal control policy. It was kind of tongue-in-cheek because for a lot of constrained optimization problems, you can get better results quicker for a lot less work with dynamic programming (if your system isn’t super high-dimensional, and you can model it well). That Quikplan trajectory optimization thing FRC 604 posted a while ago is an example.
The reasons I’ve heard are pretty common in longer-lived software companies in general: technical debt, and company politics/culture preventing cleanup of the former.
I wouldn’t worry about that. One nice thing about ML models is they can be packaged and reused. It’s much less of an investment to use than swerve which requires large amounts of infrastructure and tuning.
That’s not to say it’s easy. But a plug and play system akin to the limelight is viable in this space.
Some people seem to be interpreting my post as a negative comment, so allow me to clarify. Far from it! The comparison to swerve was meant as a compliment. This is high-end technology that will trickle down to raise the floor for a lot of teams.