pic: Year of Depth Cameras?

[JohnGilb]

The Kinect does some unexpected depth processing that you may want to be aware of (or at least, it did in the past, and I don't think it's changed recently).

When you ask for the depth value of a specific pixel (and this is hard for me to put into words) it:

returns the distance to the closest point of the plane that:
-intersects the object in that pixel
AND
-is normal to the viewing angle of the Kinect.

This is easily observed by putting an object directly ahead of the Kinect, say, at 6 feet. Then, slide the object left and right. The Kinect will report that the object is still 6 feet away, even though as you slide the object to the left or the right, the absolute distance to the object increases!

This is very helpful for games, by the way - since the Kinect was originally intended to watch the human body, they wanted to report constant distance values for the depth of the player body even if the player body is not directly in front of the sensor.

However, if you're trying to find objects and navigate to them, you should be aware of this particular quirk. This means as you rotate the sensor, the reported distance of objects will change, even though their real-world distance has remained constant.