M$ Kinect controlling robots!

[Tom_Bottiglieri]

The depth measurement (a time-of-flight LIDAR array cleverly built into the CMOS camera) is responsible for much of the capability of the Kinect (using depth to segment you from your surroundings is much faster and more robust than doing it with color/intensity via the RGB camera

I’d like to see how the IR time of flight measuring works in noisy environments (like say, outside or under stage lights). I’ll have to get my hands on some hardware pretty soon.

Also, if the frame rate is decent enough, you may be able to spin this thing on a vertical axis for Velodyne type readings. We did this with single plane LIDARs (this project: MIT CSAIL Autonomous Forklift), and the results were pretty good.

The cool thing about the depth sensor here is most CV algorithms (edge detectors, feature finders), should just work.

[Vikesrock]

I’d like to see how the IR time of flight measuring works in noisy environments (like say, outside or under stage lights). I’ll have to get my hands on some hardware pretty soon.

Also, if the frame rate is decent enough, you may be able to spin this thing on a vertical axis for Velodyne type readings. We did this with single plane LIDARs (this project: MIT CSAIL Autonomous Forklift), and the results were pretty good.

The cool thing about the depth sensor here is most CV algorithms (edge detectors, feature finders), should just work.

Frame rate is 30 fps according to the technical specs that I have seen floating around.

I’m also very curious as to how it would perform outside. The Microsoft documentation specifically mentions making sure there is no direct sunlight on you or the Kinect when you are using it so there is probably at least some performance degradation.

[basicxman]

Who says it has to be on the robot?
Why not use it as it was designed, and have your drivers be the controllers?

That’s exactly what I was saying; I realize that it is probably not FIRST-competition reliable but would still be cool in other environments.

[JesseK]

Who says it has to be on the robot?
Why not use it as it was designed, and have your drivers be the controllers?

That’s exactly what I was saying; I realize that it is probably not FIRST-competition reliable but would still be cool in other environments.

The Kinect sense has a practical ranging limit of 1.2–3.5 metres (3.9–11 ft) distance

Standing far enough back from the controls for Kinect to properly sensor a driver means that driver has to stand at the very rear of the driver’s station. Additionally, both the Driver Coach and the other Driver have to make sure they do not interfere with Kinect’s imaging. Finally, every couple of years the human players are also confined to the driver’s station; thus any of the HP’s movement would have to keep from interfering with the Kinect as well.

By all means file it under ‘neat’ and ‘innovative’ – so long as it’s not within a competition robot driver controls context.

[Mark_McLeod]

Using it on the driver station would also seem to be in technical violation of the Autonomous rules, where you would be able to use the controls from behind the autonomous driver line.

[davidthefat]

OpenKinect/libfreenect

Thats the working link.

Now how would we change the motors to legal ones?

[NickE]

Using it on the driver station would also seem to be in technical violation of the Autonomous rules, where you would be able to use the controls from behind the autonomous driver line.

In theory, the coach could also control the robot, which could be an issue.

[Vikesrock]

https://github.com/OpenKinect/libfreenect

Thats the working link.

Now how would we change the motors to legal ones?

The two options I would look at would be ditching the motors altogether or attaching some type of servo powered pan-tilt mount externally. I haven’t yet gotten the chance to pull one apart, but the images I’ve seen seem to suggest that it’s packed pretty tight so fitting a servo inside the base may be difficult or impossible.

[davidthefat]

Actually I ditch the idea of kinect all together, going back to my old idea which seems like the best way for me

[jason_zielke]

The depth measurement (a time-of-flight LIDAR array cleverly built into the CMOS camera) is responsible for much of the capability of the Kinect

Did you find an explanation somewhere that lead you to believe the Kinetic uses IR time of flight? That approach is generally very expensive but has been done in the approach you are suggesting in the SwissRanger . . . http://www.acroname.com/robotics/parts/R330-SR4000-10M-ETH.html

However, given the price point of the Kinect, I am guessing they are using a triangulation approach similar to that described here: http://en.wikipedia.org/wiki/3D_scanner#Triangulation

This approach is also often referred to as Structured Laser Light Ranging (try a google search - lots of research in this area).

In either case, I believe the FRC rules would have to change to allow the device to be used on the robot because it uses an “exposed laser”. This rule is likely in place for eye safety reasons, but I would love the opportunity for our team to be able to use a ranging device like this on the robot. It opens up all kinds of advanced opportunities for robot intelligence and would put students in the thick of developing cutting edge software.

[Bot190]

I sincerely doubt that the rule you cited would be broken by the Kinect… I have to wonder if you actually looked at the product and its intended use.

I can see it breaking the rules about controlling the robot during autonomous though.

It seems like it would be quite hard to separate the driver from people in the background too.

[davidthefat]

If you are aware, the only potential problems with the Kinect is the illegal motors, may be teh custom circuitry might be illegal.

[kamocat]

Remember that the motors are only illegal if you power them.
If you use them as decoration, they’re perfectly legal.
The Kinect is a COTS part, available for less than $400.

[davidthefat]

Remember that the motors are only illegal if you power them.
If you use them as decoration, they’re perfectly legal.
The Kinect is a COTS part, available for less than $400.

Exactly, its wasted money, it only costs about $57 actually manufacture one, which has teh motors, cameras, mics and the casing and ect, which is unneeded. Thats what I meant about over priced. All you need is the cameras, so I am just making my own system

[Vikesrock]

Exactly, its wasted money, it only costs about $57 actually manufacture one, which has teh motors, cameras, mics and the casing and ect, which is unneeded. Thats what I meant about over priced. All you need is the cameras, so I am just making my own system

The BOM number being thrown around from TechInsights is an estimate based only on the cost of components. It does not include manufacturing costs, recoup on R&D, retail margins, and any other costs. These types of costs will be included in any COTS solution.

The Kinect works very differently than a pair of cameras set up for stereo vision. It uses an IR projector to illuminate the scene with a dot pattern, the reflections are then observed using an IR Camera. The data from the sensor is then fed into a chip made by a company called PrimeSense that processes that data into the depth image.

I also wanted to address something you posted in the other thread about the Kinect, suggesting that similar things have been around in robotics for a while. I am not aware of any prior solutions that provided depth information with no processing at this resolution (in all 3 dimensions) for anywhere near the $150 price point. If you do know of such a device I would love a link to further info.

[davidthefat]

The BOM number being thrown around from TechInsights is an estimate based only on the cost of components. It does not include manufacturing costs, recoup on R&D, retail margins, and any other costs. These types of costs will be included in any COTS solution.

The Kinect works very differently than a pair of cameras set up for stereo vision. It uses an IR projector to illuminate the scene with a dot pattern, the reflections are then observed using an IR Camera. The data from the sensor is then fed into a chip made by a company called PrimeSense that processes that data into the depth image.

I also wanted to address something you posted in the other thread about the Kinect, suggesting that similar things have been around in robotics for a while. I am not aware of any prior solutions that provided depth information with no processing at this resolution (in all 3 dimensions) for anywhere near the $150 price point. If you do know of such a device I would love a link to further info.

I do not want to argue, I said the technology is nothing new, never said the said technology was cheap. The fundamental ideas have been used for quite a long time, in fact, its in the book: http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=10138. The projection of an image is projected onto the object which the camera picks up and processes. Nothing new.

http://www.google.com/url?sa=t&source=web&cd=1&sqi=2&ved=0CBcQFjAA&url=http%3A%2F%2Frobotics.stanford.edu%2F~ang%2Fpapers%2Ficra09-3dSensingMobileManipulation.pdf&rct=j&q=high-accuracy%203d%20sensing%20for%20mobile%20manipulation%20improving%20object%20detection%20and%20door%20opening&ei=Ru3hTOrcL4G-sAOpg7juCg&usg=AFQjCNGKdwQQm3gdK3uR3Vz3Gg48er68eg&sig2=gLHUQIJbvnlZkw3eizNXvg&cad=rja

[Vikesrock]

I do not want to argue, I said the technology is nothing new, never said the said technology was cheap. The fundamental ideas have been used for quite a long time, in fact, its in the book: http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=10138. The projection of an image is projected onto the object which the camera picks up and processes. Nothing new.

http://www.google.com/url?sa=t&source=web&cd=1&sqi=2&ved=0CBcQFjAA&url=http%3A%2F%2Frobotics.stanford.edu%2F~ang%2Fpapers%2Ficra09-3dSensingMobileManipulation.pdf&rct=j&q=high-accuracy%203d%20sensing%20for%20mobile%20manipulation%20improving%20object%20detection%20and%20door%20opening&ei=Ru3hTOrcL4G-sAOpg7juCg&usg=AFQjCNGKdwQQm3gdK3uR3Vz3Gg48er68eg&sig2=gLHUQIJbvnlZkw3eizNXvg&cad=rja

Ok, that is a valid point. The basics of the approach are nothing new.

Most prior implementations of this approach either used visible light which has the downside of being, well, visible as well as a number of others or used lasers which are expensive.

[davidthefat]

Ok, that is a valid point. The basics of the approach are nothing new.

Most prior implementations of this approach either used visible light which has the downside of being, well, visible as well as a number of others or used lasers which are expensive.

Now I had the idea of using an IR laser when I first initially investigated the idea of 3d imaging, but I found out that they are EXPENSIVE. But I read that the laser in a CD/DVD reader is an IR laser, I would have to find out. Using a champagne cup’s “neck” works fine as a prism. IDK I might consider it again

[Jared_Russell]

Did you find an explanation somewhere that lead you to believe the Kinetic uses IR time of flight?

Yes.

How Motion Detection Works in Xbox Kinect

[jason_zielke]

Yes.

http://www.wired.com/gadgetlab/2010/11/tonights-release-xbox-kinect-how-does-it-work/

Thanks for the link. There is a comment at the end of the article that had the same doubts I did so I dug into the answer a little further.

From the FAQ section (http://www.primesense.com/?p=535) of the PrimeSense website (core technology behind the Kinect):

“The PrimeSensor™ technology is based on PrimeSense’s patent pending Light Coding™ technology. PrimeSense™ is not infringing any depth measurement patents such as patents related to Time-of-Flight or Structured Light – these are different technologies, which have not yet proven viable for mass consumer market.”

Looks like my theory about structured light was wrong as well.

As it relates to FRC rules on lasers, this was also on the PrimeSense website:

“The PrimeSensor™ Reference Design incorporates a class 1 laser device and has been certified by independent laboratories for class 1 compliancy in accordance with IEC 60825-1.”

Although it is an eye safe laser, it is still a laser that is exposed outside the device (unlike in a laser ring gyro where the laser light never leaves the device). This would make it illegal per last seasons rules.