There is definitely some barrel distortion, and there exist algorithms to try and account for it.
That said, if you have images getting processed at 15, 30, 60 frames per second*, it doesn't much matter - as you turn closer to the center of the object, you will receive new azimuth information which should correct for prior errors.
(*Our current prototype tracking code is running at 150 frames per second...can anyone beat that?
It's a shame the camera can't go past 60 )