Optical Flow Tracking, Help!

[falconmaster]

Hello all,
We have a problem. We have a underwater robot that has two cameras. One front camera and one downward facing camera. We want to use the downward facing camera to use the bottom of the pool at the transdec facility in Point Loma, San Diego, at the naval base, to track the motion of our AUV. The pool bottom is patchy and algae covered so there are patterns that can be seen. Assuming we are a a given height, above the pool bottom, we want to use the same type of program that a mouse uses to track a table top for us to basically have our auv be the mouse in the pool and for us to able to tell what direction and how fast we are traveling. Does anybody out there know how to help us, or know somebody else out there that can help us? Thanks! This is for the ROBOSUB event for next year, so we have some time to develop this and we really want to. The teams with the big bucks are using Doppler Velocity Loggers which cost 20k and uses acoustic Doppler shifting to track the velocity of the AUV. We feel that we can achieve a similar capability using our downward camera and the "mouse code"

[techhelpbb]

Originally Posted by falconmaster

Assuming we are a a given height, above the pool bottom, we want to use the same type of program that a mouse uses to track a table top for us to basically have our auv be the mouse in the pool and for us to able to tell what direction and how fast we are traveling.

I am unclear on what you are entirely asking.
Are you saying this device will be kept at a given fixed height in the pool or that this device will need to use the bottom of the pool to put itself at a certain height from the bottom of the pool?

If your intent is to put yourself a fixed height from the bottom of the pool you would be much better off using sonar not video for that purpose.
The depth measurement is much easier to achieve with sonar (a Doppler technology).
In this case though you are moving perpendicular to the plane of measurement so it will be much easier and cheaper.

As far as tracking the pattern on the floor for X/Y movement the technology used by optical mice is DIC / DDIT and it is derived in part from military technology.

What is the resolution of your camera and what do you have to process the image?
Start with Moire, MatLab and Mathematica.

[falconmaster]

We want to use the bottom the pool to track our movements and then compare it to a virtual map of the pool.

1. The pool floor has a patter of markings caused by algae patches and the pool floor.

2. We are assuming a given height for the time being because different distances from the pool floor will result in different velocities. We can add that to the formula after we are able to track our velocities. Different heights will require different adjustments to the pattern tracking algorithm. We can determine this by experimentation

3. We know a mouse uses this type of algorithm to move the arrow on a computer screen, this is the code we are trying to get our hands on. We think if we have this, we will be able to use it to accomplish tracking goal.

4. Basically it is comparing one frame of video to another frame of video and measuring the differences between the two images and calculating the direction, the degree of difference is compared to the known frame rate and this will give us speed.

5. We will use this data of movement to locate where we are in a virtual pool that we will build in our computer. We do know where we start so we should know where we are if this works.

Does this help? Now can you help? Or Do you know someone who can help us get this code?

Thanks in advance!!!!!!

[techhelpbb]

Originally Posted by falconmaster

Does this help? Now can you help? Or Do you know someone who can help us get this code?

Thanks in advance!!!!!!

I can provide you some help though it has been a few years since I did this kind of work.

Do you want something that does that like Greg offered but that works on a PC and works out of the box?
Do you want an actual source code example?

If you look back in this topic I already suggested: Moire, MatLab and Mathematica all of which have existing work to demonstrate the mathematics.

I really will need more information about your cameras and illumination to give you specific advice it makes a huge difference in the result.

This should help you quite a bit and quite directly:
http://robots.stanford.edu/cs223b05/...tical_flow.pdf

What else can I offer you?

[Greg McKaskle]

Not meaning to be the LV guy, but it is the hammer in my hand ...

If you have LV installed, examples/vision/Motion Estimation shows two similar approaches. They have images in a directory and overlay the vectors of each particle to show how the different HS and LK implementations compare.

If you have images or video to test with, I may be able to validate the approach, even if you later use a different implementation of the algorithm.

Attached screenshot shows the examples. Top one had the particles rotating on a turntable -- obvious from the vectors.

Bottom is some sort of spark/flame with particles moving at different speeds and less applicable I'd assume.

You may also want to look at the other vision SW that came in the FIRST kit, I don't remember the name, but had screen steps to document how to use it and it is pretty cool.

Greg McKaskle

[falconmaster]

Originally Posted by Greg McKaskle

Not meaning to be the LV guy, but it is the hammer in my hand ...

If you have LV installed, examples/vision/Motion Estimation shows two similar approaches. They have images in a directory and overlay the vectors of each particle to show how the different HS and LK implementations compare.

If you have images or video to test with, I may be able to validate the approach, even if you later use a different implementation of the algorithm.

Attached screenshot shows the examples. Top one had the particles rotating on a turntable -- obvious from the vectors.

Bottom is some sort of spark/flame with particles moving at different speeds and less applicable I'd assume.

You may also want to look at the other vision SW that came in the FIRST kit, I don't remember the name, but had screen steps to document how to use it and it is pretty cool.

Greg McKaskle

I will have to get my programmers get on this, when can we set up a conference call?

[Greg McKaskle]

Sent via PM.

Greg McKaskle

[falconmaster]

My original intent here was to find people that know how to do this and to see if there was any code that we could use to make this work on our windows 7 based Ivy Bridge processor running C sharp and A-Forge vision libraries. I am trying to light a fire under my programmers butts! One is a Freshman, but he is sharp the other a junior who needs some pushing sometime. I just want it done! If we can do this, there are many other AUV teams out there that could bennefit that are being out teched by the big guns in the competition like Cornel and Florida because they have DVLs and I feel that a vision based tracking system will work in this particular environment. It will give the rest of us a better chance of being in the top three or possibly winning. Here is the event website
Robosub
http://www.auvsifoundation.org/found...tions/robosub/
We finished 9th out of 32 teams, our best showing yet, but we aren't finished yet!
Here is our AUV team website
https://sites.google.com/site/falcon...csauvteam/home

[techhelpbb]

Originally Posted by falconmaster

My original intent here was to find people that know how to do this and to see if there was any code that we could use to make this work on our windows 7 based Ivy Bridge processor running C sharp and A-Forge vision libraries. I am trying to light a fire under my programmers butts! One is a Freshman, but he is sharp the other a junior who needs some pushing sometime. I just want it done! If we can do this, there are many other AUV teams out there that could bennefit that are being out teched by the big guns in the competition like Cornel and Florida because they have DVLs and I feel that a vision based tracking system will work in this particular environment. It will give the rest of us a better chance of being in the top three or possibly winning. Here is the event website
Robosub
http://www.auvsifoundation.org/found...tions/robosub/
We finished 9th out of 32 teams, our best showing yet, but we aren't finished yet!
Here is our AUV team website
https://sites.google.com/site/falcon...csauvteam/home

Here's what you said you'd be using:
"but are converting to C++ and Open CV vision libraries."

If you read the end of the link from Stanford I provided there is working source code that tells you all you need to really know to make this happen.
http://robots.stanford.edu/cs223b05/...tical_flow.pdf

I note again image quality is your issue here. Particulate in the water that is moving will mess with your tracking.

I will also caution you to remove your contact numbers from this topic before something reads it and starts calling you all the time.
I've PMed you my contact information.

[falconmaster]

Thanks, I removed them. I think people don't prank call as much now that cell phones have numbers show up, harder to hide.....anyway I took them off thanks!

[techhelpbb]

Originally Posted by falconmaster

Thanks, I removed them. I think people don't prank call as much now that cell phones have numbers show up, harder to hide.....anyway I took them off thanks!

So now we finally have the FIRST water game we've all joked about for so long.



I see from the link you are using Guppy firewire cameras.

[falconmaster]

Originally Posted by techhelpbb

So now we finally have the FIRST water game we've all joked about for so long.



I see from the link you are using Guppy firewire cameras.

Hey, we are ready! Bring on the water!

[falconmaster]

You posted

"This should help you quite a bit and quite directly:
http://robots.stanford.edu/cs223b05/...tical_flow.pdf"

good stuff in here, will show my programmers! Thanks!

[Gdeaver]

I've looked at the team website and the competition web site. What exactly are you having a problem with? From the call for mouse odometry, I would infer that you are having problems navigating to the various stations. Is this it? Do you have a well stabilized platform? Can you navigate a straight line using the PNI compass and gyro board? What sensor fussion algorithm are you using. Can you maintain constant depth with the pressure sensor? You have 10 DOF with the sensors you should have a good AHRS solution. So you are looking to supplement this with a X and Y distance measurement. Be aware that with only 1 mouse odometry input there are multiple solutions. 1 with no body rotation (a line) and one with body rotation. In 3 D of course. Thus the need to fuse with the AHRS. The curved bottom of the pool and variations in depth and platform orientation from level will mess with accuracy. Looking at the diagram of the pool 1st gen cruise missile comes to mind. Dead recon to a way point, detect way point object and orientation with vision. Rotate and dead recon heading to next way point etc. So what have you achieved and what needs improvement?

[falconmaster]

Originally Posted by techhelpbb

I can provide you some help though it has been a few years since I did this kind of work.

Do you want something that does that like Greg offered but that works on a PC and works out of the box?
Do you want an actual source code example?

If you look back in this topic I already suggested: Moire, MatLab and Mathematica all of which have existing work to demonstrate the mathematics.

I really will need more information about your cameras and illumination to give you specific advice it makes a huge difference in the result.

This should help you quite a bit and quite directly:
http://robots.stanford.edu/cs223b05/...tical_flow.pdf

What else can I offer you?

I will have to get my programmers to get in on this, when can we set up a conference call?