[Jared Russell]

I have had to solve analogous problems many times in my academic and professional lives. The first problem is getting reliable and robust features to track between images (for a sparse optical flow solution). I do not have much experience using RGB cameras under water, but I have looked at the bottom of a pool plenty of times and would expect the dancing lights and shadows to make this a real challenge. This makes getting robust sparse features difficult, and using dense optical flow (where every pixel is a feature) basically impossible.

Do you have representative imagery of what the cameras would be capturing? If so I might be able to help identify feature extraction algorithms that would work. Terms to Google would be "SIFT", "SURF", and "Harris corners". OpenCV-based implementations of all of these are easy to find. Optical mice "cheat" and use a light source at an oblique angle to cause minor imperfections in a surface to turn into reliable features. You most likely will not have that option unless you are hugging the pool bottom.

Once you obtain features that you can track between multiple images, estimating the robot's motion is fairly straightforward. There are a few different techniques you can use, but in general the more assumptions you can make (e.g. I am at constant depth with a zero pitch and roll), the fewer degrees of freedom you need to worry about, and therefore the fewer feature correspondences you need to accurately measure your position. But step one is finding the right features.

As you've already identified, a DVL is by far the best way to do this underwater (currently). Failing that, if there is no water current (or a known water current), you could buy a water speed sensor and combine that with compass/gyros to do dead reckoning.