Backboard angle using vision tracking

[DallonF]

I've been doing a ton of work on vision tracking and so far it's been going pretty well, but the main issue is that we don't want to shoot at the vision target - we want to shoot at the center of the hoop! We have the math working to adjust for that, but we need the angle of the backboard relative to the robot

We were thinking of using the apparent aspect ratio of the target to determine the angle (the target is 4:3, so if it appears to be 1:1, we should know we're at an angle). The problem with this is that A. we have no idea what direction we're facing the target at, and B. due to fisheye on the Axis 206 camera we're using, even when we're directly facing the target, if the target is toward the sides of the screen, it reports a ~20 angle.

Does anyone have any ideas on this?