Visual Fiducials in Future FRC Games

Hi everyone,

As an active robotics researcher (senior robotics PhD student at Oregon State University) I’m a bit surprised by the continued use of retro-reflective vision targets rather than visual fiducials in FRC. The current vision targets FRC uses is the retro-reflective tape which lends itself well to color based processing algorithms. However, for a variety of reasons color based vision processing is not particularly robust and most current robotics in research and industry would use visual fiducials, like AprilTags, instead. After a few searches on Chief Delphi there seems to be some interest, but limited discussion. So the goal of this post is to start on a conversation about whether switching to visual fiducials instead of the current retro-reflective tape is a good idea and to garner support for an official request to FRC to include visual fiducials for vision targets in future FRC games.

Pros:

  • Very robust processing using open source software, such as AprilTags (AprilTag). You can even download apps for your phone that can efficiently and robustly track these targets.
  • Use of stereo camera’s for very precise depth measurements with visual fiducials is easier than with color processing.
  • Easier for teams to construct the targets, no needing special reflective tape, just a printer and paper is required.
  • Visual fiducials matches what is used in industry and academic settings for tracking targets.
  • No more blinding vision lights required on robots. :slight_smile:
  • Edit: Forgot to mention accurate pose estimation of visual fidicuals.

Cons:

  • Writing your own software to track fiduicials is fairly complicated for a high school student. Whereas color based vision processing is much easier to describe to a high school student and have them be able to write. In my experience though, most naive color based methods are very time consuming to tune, and only work in certain lighting.
  • Current knowledge and infrastructure on tracking retro-reflective tape is well known in the FRC community. For example, the Limelights are used by large number of teams quite effectively.
  • WPILib doesn’t currently have any software to track fiducials, but could easily be added in the future.

So, my question to the community is what are your thoughts? Is the use of retro-reflective targets worth simpler vision tracking algorithms for teams that develop their own algorithms? Would you just like to see both retro-reflective targets and visual fiducials in the future?

Thanks for your input!

30 Likes

Could distance of the target along with camera resolution be a concern? Easier pose estimation of ARUCO and April tags is a huge pro in my opinion. It would also be nice if FIRST switched away from transparent field walls and field elements to make SLAM more feasible.

6 Likes

Pose. :wink:

Edit: sniped.

2 Likes

i could see a game include both retro-reflective tape & AprilTags. perhaps retro-reflective tape to line up to a target and for pose estimation (like it was commonly used in this years game) but then AprilTags for game piece or landmark recognition? I imagine in the future that FIRST will capitalize on autonomous control.

2 Likes

I can absolutely tell you that nothing is likely to happen with this until someone produces a compelling demo involving an actual FRC robot…. Sadly.

6 Likes

Yep, and from a team perspective I’d like to see detection working on LL hardware at a good framerate. It might be possible using the Python pipeline feature, and then upstreamed? Please don’t make me buy hundreds of dollars of new hardware :slight_smile:

2 Likes

Agreed, easier pose estimation is a big pro. I will explicitly list that in my pros.

The distance vs resolution is a good thought. For april tags using the formula from this webpage Determining the Ideal Resolution for AprilTag Detection | LEMUR, the number is a bit worse than I expected 3 ish meters for 10cm tag at 720 horizontal pixels and a standard fov for a webcam. That seemed a bit conservative though.

3 Likes

Ah, that’s good to know. Out of curiosity, what do you think FIRST would consider a compelling demo?

2 Likes

I imagine a pose calculation that’s “good enough” and some sort of goal tracking without LEDs.

4 Likes

Construct a vision Tetra per 2005 rules, and perform designed experiments comparing effectiveness of several methods to get it capped in auton.

2 Likes

Well…

5 Likes

Yeah…. I know you know.

Sadly, both you and us share a commonality in that just because we can do it, it is assumed others cannot.

5 Likes

I fully expect that the PhotonVision project would expand to include fiducials. It’s a software-based CV solution that runs on LL hardware as a supported platform (though you won’t need the lights for fiducials)

3 Likes

I tried running Photon with a team this season, ran into too many issues and had to revert to an LL image for competition. That’s another topic though. Either way, I’ll believe in a turnkey solution once I see it.

1 Like

My experience with AprilTags/various fiducial markers is limited to one internship project, but I do agree that they are the next step for FRC vision targets. AprilTags are robust, but I think they’d need some further engineering, specific to FRC, to accommodate the combination of occlusion, distance from target, image resolution, and velocity of the robot.

I think using them for landmark registration and clocking would be really cool and appropriate. Maze game?

1 Like

So, dumb question: What’s to keep a team from taping a fiducial to the inside of their driver station and using it to inform their robot’s location-finding during auto?

5 Likes

Nothing - 1768 did that for lining up to their feeder ramp in 2015.

7 Likes

OK ok - how about we meet half way. April tags stenciled out of retroreflective tape?

11 Likes

Not needed and a lot of work. Easier to just outline it with a box of retro tape if we want both.

12 Likes

Just to expand on the off-hand comments we’re making:

Proper fiducials would be amazing. In 2020, we made use of the graphics around the target to get better vision returns than we could readily get from the retroreflective tape alone (see the 2020 971 reveal thread). Using that worked pretty well, but was a bit irritating because the images consisted of multiple pieces, so we had to recalibrate at each event, and couldn’t make as much use of existing libraries as you could with well-defined markers.

This year, there were no good logos/visuals/whetever near the goal that are visible from large parts of the field, so we had to go back to doing retroreflective tape and so had to build an LED board and write the code for processing the tape-based targets. This has proven more finnicky and just as, if not more, complex than the code required to use SIFT (for the same level of results).

Also, I haven’t gone back and looked at it recently, but @Peter_Mitrano and some other seniors at WPI did some exploration of options for localization in FRC back in 2018 and also looked at ArUco markers, so it’s not just teams whose numbers start with 9 who have been able to do this stuff. See https://digital.wpi.edu/show/wp988m53f

6 Likes