[NotInControl]

Originally Posted by faust1706

... then I do a linear transformation between the camera's coordinates (3d coordinates with camera being the origin) to the field coordinates (where the bottom left corner is the origin)...

How do you do this linear transformation from local image coordinates to global coordinates used in your path planning, if you do not have either a constant object of known dimension in view at all times, or you have a fixed camera distance with fixed focal length? What is the equation you use that ignores using either of those paramters?

Originally Posted by faust1706

... (See also: http://www.chiefdelphi.com/media/photos/38819 this is a pose estimation. It knows where we are in 3 dimensions with respect to the center of the top hoop, as well as how rotated we are in pitch roll and yaw).

Correct me if I am wrong, but is this not localization, by keeping a known object of fixed dimension in frame, and determining the scaling factor of your image based on that? Then determining your location based on the assumed distance, the object is from camera, and the angle between the center of the frame and center of the object?

If this is the case, then once the goals are out of frame, you can no longer determine where you are in the world, correct? How do you plan to do something similar just based on pixel value, without having either a fixed camera distance, or a fixed object of known dimension in the frame?

I don't know of a method that only uses pixel location without knowing the distance to the object, or keeping a fixed dimension object in frame to determine the scaling value. You need one of those to calculate global position. Also, depending on your camera and field of view, the reason I keep bringing up skew is because going from local to global coordinates is not linear, the edges of the frame will have a different (skewed) distance per pixel where the center of the image will have another. As long as you are more focused on the center of the image, you can use the small angle approximation in order to linearize distance per pixel.

Keep us posted on the project.

Regards,
Kevin