We are looking to up our vision for this season, Has anyone had any success working with those rpi cam fisheye lenses? 180 degrees sound promising though I’m not sure whether it will work well or not with retroreflective tapes.
2 Likes
They can work, however the high distortion they cause to the image violates the pinehole camera model which stops most vision processing algorithms from working well - including PhotonVision’s strategies.
Even with a simple “find the big bright thing in the image” algorithm, the same target will look quite a bit smaller if it’s off to the side, so you’ll have to pick your thresholds for what counts as “big” carefully.
Per the example linked - another strategy is to undistort the image first.
4 Likes
This is purely my opinion. When on 2073, we used the following rule of thumb. Vision tracking used a max 90 deg camera. Live view maxed out at 120 deg.
Distortion and object apparent size become issues beyond those values.
There are transformations that can be applied to project parts of the image into something more sane that can kinda work. They are not computationally simple. Converting a fisheye image to panoramic, spherical and perspective projection contains some examples.
If you were asking me - I’d point the camera straight up, and focus on a donut of interest (all donuts are of interest…) such that the X coordinates map from 0 degrees to 359.9 degrees.
Disclaimer - Owl Labs Meeting Owl uses such an approach - they are a former employer. I didn’t work on the vision system there.
4 Likes
Such a system/use case might be better used by the OP to direct a non-fisheye camera towards the place of interest, as opposed to using the fisheye as a direct image source.
As mentioned, while it is possible to get a ‘flat’ image from a wide-angle camera, it’s not trivial and definitely has its limitations. This may be more than the OP should be trying to bite off and chew, BUT one never knows what genius lurks out there.
@Just_A_Bored_Fella While you are told it might be sub-optimal, if finances permit give it a try. I 100% guarantee you will learn several very important things about camera vision, and with that knowledge you will be in a place to provide significant help to your team.
With a guarantee like that, what have you got to lose??
1 Like
Eh, if you’ve got enough processing oomph it can work fine - you’re just processing and throwing away a huge chunk of the image. No real need to lug around two camera systems.