Apriltags with a fast B/W camera

We have been experimenting with Apriltags using a Raspberry Pi 4. Initially we were just using the standard 5MP Raspberry Pi camera but found that when the camera was moving the motion blurring of the image it make it difficult to detect the tags and, even if we did detect them, the position and rotation vectors were compromised.

We found a fast black and white camera that we decided to try. This particular camera is able to capture images at 144 FPS at a resolution of 1280x800. It also has an adjustable shutter speed that can be set to anything between 9us to 7.4ms. Using this camera we were able to eliminate the motion blurring.

With this camera and the official implementation of the Apriltags code we can get a frame rate 75 FPS with a Decimate of 6 and a frame rate of 50 FPS with a Decimate of 4 at a resolution of 1280x800.

We also have experimented with alternate code to decode the tags and, although it is not quite as robust as the official implementation, we are seeing frame rates up to 140 FPS at 1280x800.

If you are interested in what we have done, you can find a complete write up along with links to all of the code we are using at http://programming.team2102.org/apriltags.

8 Likes

Awesome work! Mind sharing more about what you had to do to get the OV9281 working? We’ve had some trouble with it as well.

Also, did you replace the lens on the OV9281? The one you linked has a wideangle fisheye lens, not sure if you calibrated specifically for the fisheye lens or not. If you replaced the lens, a link would be appreciated, I had a hard time finding one that was affordable.

Well, they provided an install file but unfortunately it only would work for two specific versions of the Pi kernel. I had to poke around a bit to find one that worked and I believe the one that I used was 2022-01-28-raspios-bullseye-armhf-lite. However the site that I linked to above has a link where you can download a pi image that is already configured for both the camera and OpenCV which is needed to get the translation and rotation vectors from the image.

As for the lens, yes it is a wide angle lens and that is the one that the code that I posted is calibrated for. I do have a 6mm lens which we may switch to depending on how the tags are used on the field next year.

Amazon has a large selection of these lenses. For example, here is a link to the 6mm one that I have (but have not yet tried). https://www.amazon.com/dp/B081YXRKH7?psc=1&ref=ppx_yo2ov_dt_b_product_details

This is some excellent work. Thank you for sharing your results and code! Reducing motion blur is going to be very important for good detection results.

What kind of range are you getting for tag detection? It looks like your test setup has them about 100 inches from your camera and facing front. My understanding of what the decimate setting is doing is that it is reducing the resolution of the image before it gets run through the april tag detection, which seems like you would be trading off distance for frame rate? How does your setup perform at greater distances or with the tags set at an angle?

With the current wide angle lens that comes with the camera, the maximum distance with a Decimate of 4 is around 140 inches. However the lens is replaceable. We have a 6mm lens that we have not tried yet which will allow for a much larger distance. Other lenses are also possible. It remains to be seen, based on the new game, whether seeing a long distance is more important than a wide angle of view.

As for the behavior at angles, if you use the standard Apriltags implementation, I expect it will behave as it normally does. The faster implementation is not quite as robust at the current time but is worth exploring further.

1 Like

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.