I was wonder what other teams were getting for FPS rates with on-board (on the bot) vision processing.
We have been able to do 30 FPS @ 640x480 per camera. That includes identifying targets, custom HUD, and distance calculations to target all on the bot. The vision processing is using two cameras at the same time. Both camera feeds are combined into a 640x240 image that is sent over the radio. We could push bigger images over for the output, but the limitation is the radio bandwidth during competition.
On the Jetson TX1, we can manage 30-45 fps at 320x240 for contour based vision. On our other coprocessor (AMD A6-5200) we can get around 100 fps for said contour vision, although that’s not what we use it for.
While we didn’t actually get it to work in the end, our vision processing on the roboRIO got about 10-15 fps. The processing algorithm was complete, we just didn’t finish writing the robot’s response to the vision.
Two things mainly. I didn’t want to bother with doing CUDA acceleration for our imagehomography, so I just tossed it on the AMD board. Maybe I should’ve in retrospect, considering we only got about one frame per second, hah (the SIFT algorithm is very slow turns out). Main thing we use it for, however, is our neural network based robot detection. We actually have a GTX 1050 Ti onboard the robot for neural network acceleration connected to the AMD board (no NVIDIA drivers for ARM platforms like the Jetson, so we had to use an x86 CPU, hence the A6-5200). With the GPU, we get around 24 FPS, and without, about one frame every fifteen seconds, so it’s pretty helpful.
That’s really cool. I missed your talk in Houston, any chance you guys recorded it? I definitely want to mess around with some deep learning this offseason.
