Good job! It takes a lot of work to do just this, believe it or not.
If you want to take it one step further, you can figure out the angle the camera(robot) needs to move to make the center of the object the center of the image.
Simply solve this proportion
Image width/Fov_x= distance from the center of the image/degrees
be careful though, the origin of the image in opencv is the top left, so you have to do a linear remapping of the point so the origin is the center of the screen!
proof of concept: In 2013 we used this x rotation value to line up with the 3 pt goal autonomously using a pid as shown slightly in this video:
http://youtu.be/VhE10dNBxXs?t=27s
Those micro adjustments were not done by our driver, but rather by simply pushing a button on the controller to line up.
This is just food for thought. Take it as you will. (You can also calculate distance to your target if your camera height and angle stay constant, that involves y rotation
)