Kinect is Legal for Both Drive Station and Field Use (Here’s why)

[winglerw28]

I feel like this is more cool-factor than anything else, and in my opinion isn't worth hassling FIRST and wasting time over. Sure you could use a Kinect, but you could also just program a camera or two more efficiently.

[davidthefat]

Originally Posted by MaxKinect

This is a very good question.

We will program our drive station to use both a XBOX controller and the Kinect. (Not at the same time) Based on years of using a joystick (Xbox controller) we know how it works. The Kinect interface will be a totally new adventure. Our goal will be to make a natural interface in which anyone can walk up and drive our robot. This may not be possible in 5 weeks, it may never be possible.



The kinect on the field may not be a good choice for this game. We will find out by trying both a standard camera and the Kinect (depth sensor).

As for the cRio precious clock cycle, if we off load the visual processing to a laptop and use C++ in the cRio there will be lots of unused cRio cycles.

Yes, the laptop would get the image input and process it, but you would then have to transfer that data to the cRio and then the cRio has to move the motors and ect. I honestly do not think one has to invest a $100 for a kinect but rather just invest $20 on a pair of low end webcams.

[Ryan Gordon]

Originally Posted by MaxKinect

Because the Kinect controller uses a pattern match on a non-repeating caustic pattern it is very resistant to interference. When there is a problem, the area of the problem is marked with a 'unknown' depth code (2047).

I think there will be more interference then you expect. Current tests have revealed that two Kinect's pointed at the same area at a 90 degree angle show little interference but six Kinect's pointed at any varying angle, possibly all at each other will render it useless. Just think if you could see the infrared pattern with 6 Kinect's pointed at the same area (which you can with an infrared camera or infrared goggles) all you will see is a big infrared blob, not a pattern. There's no way the computer could make sense of something like that and give you enough information to make your algorithms work.

It would be pretty easy for defender robots to use the Kinect or similar allowed-lasers to mess with it if they wanted. And not to be pessimistic but even people from the audience or other team members could use items to interfere and no one could know because its invisible to the eye. Even things like camera auto-focusing that release IR wavelengths could interfere to a certain degree.

I think it's great that there's people out there that are pushing to use this but I simply don't think that the technology is mature enough to use easily because it posses too many potential uncertainties in different situations for any algorithm you may throw at it. So far everything it's been used for (including Microsoft) and has actually worked reasonably well has been done in very very controlled environments.

Just my 2 cents.

[Grim Tuesday]

Sure, its easy for me to stand in the stands and hold up a vision target, rendering anyones autoscoring system useless. But am I going to? No.

[PayneTrain]

Best of luck to what you are trying to accomplish, but I think most can use a joystick to drive as well. I think using joysticks can provide more predictable, and therefore more effective and safer results.

[Billfred]

Just one point to bring up:

The Kinect has a motorized pivot on the base. How do you pass the FRC motor rules, short of cracking the case and removing it?

[GaryVoshol]

The GDC has spoken - Not Legal: http://forums.usfirst.org/showthread.php?t=16240

[Tristan Lall]

Originally Posted by GaryVoshol

The GDC has spoken - Not Legal: http://forums.usfirst.org/showthread.php?t=16240

On the robot, at least. (<R02> refers to "[i]tems specifically prohibited from use on the ROBOT".)

Arguably, you could enclose the laser completely to avoid the <R02> violation. (The laser is integral; the device causing it not to be exposed to the surroundings may not have to be integral, depending on the interpretation of the rule. However, as a practical matter, I can anticipate the Q&A response....) As for the motorized base, since there is significant signal processing and I/O on the Kinect, it might be considered a "COTS computing device" for the purposes of <R45>.


On the topic of the laser rule, while it's relatively easy to enforce—except for the sometimes-fluid definition of "exposed"—it's somewhat too conservative. These are obviously eye-safe lasers—the Kinect has a Class 1 rating (for an exposed device, eye-safe on a continuous basis).

The rule ought to allow any number of unmodified Class 1 lasers for competition use, on the robot and operator console, subject to a gameplay rule about creating distractions. There should be a separate venue rule stating that if you're in possession of a class 3 or higher laser, or in possession of a laser modified so as to invalidate its rating, or using any laser whatsoever in a vexatious way, you'll be thrown out of the building.

[Vikesrock]

Originally Posted by Billfred

Just one point to bring up:

The Kinect has a motorized pivot on the base. How do you pass the FRC motor rules, short of cracking the case and removing it?

It isn't all that bad to do this once you have the right tools. We have two at work that we have removed the entire base from.

[PrinceTyke]

I'd just like to point out that, while it wouldn't necessarily be better, it would certainly be cooler to control it with Kinect.

[Vikesrock]

Originally Posted by Vikesrock

It isn't all that bad to do this once you have the right tools. We have two at work that we have removed the entire base from.

And now here's a video of those Kinect's mounted to a robot to perform autonomous navigation.

http://www.youtube.com/watch?v=kn93BS44Das

The processing overhead for the Kinect's is much lower than any stereo vision implementations I have heard of. This entire system runs on a pair of dual core 1.66GHZ Intel Atom motherboards, mounted on the robot and running off an unfiltered 12V DC in from the battery. It should be noted that the motors are MUCH smaller than FRC and the battery is much bigger so voltage dip is nowhere near what is seen on an FRC robot.

[Al Skierkiewicz]

<R02> ROBOT parts shall not be made from hazardous materials, be unsafe, or cause an unsafe condition. Items specifically prohibited from use on the ROBOT include (but are not limited to):

C. Any devices or decorations specifically intended to jam or interfere with the remote sensing capabilities of another robot,including vision systems, acoustic range finders, sonars, infra-red proximity detectors, etc.(e.g. including imagery on your robot that, to a reasonably astute observer, mimics the VISION TARGET)
D. Exposed lasers of any type (COTS devices with completely enclosed integral lasers, such as a laser ring gyro, are permitted)

This kind of says it all doesn't it?

[Vikesrock]

Originally Posted by Al Skierkiewicz

This kind of says it all doesn't it?

Absolutely. I was not intending my post here to be a suggestion that anyone try using the Kinect on their FRC robot, as the GDC has explicitly ruled on it in the Q&A in addition to the rule you cited. I just thought some people here may be interested in what can be done with the device on a robot.

[MaxKinect]

Absolutely. I was not intending my post here to be a suggestion that anyone try using the Kinect on their FRC robot, as the GDC has explicitly ruled on it in the Q&A in addition to the rule you cited. I just thought some people here may be interested in what can be done with the device on a robot.

This is a very good point. Kinect is a very safe sensor technology. The fact that is uses a single frequency infrared light source makes it a laser device and therefore "this year" (note: these words were used by the GDC to answer to question) not allowed for field use.

The Martians (494) and More Martians (70) are still working with the Kinect and hope that other teams will join us in the adventure of discovery.

[Tom Line]

Originally Posted by davidthefat

Yes, the laptop would get the image input and process it, but you would then have to transfer that data to the cRio and then the cRio has to move the motors and ect. I honestly do not think one has to invest a $100 for a kinect but rather just invest $20 on a pair of low end webcams.

David, I would be very interested in seeing you develop a system out of two webcams that provides the kinect's capabilities. I suspect the engineers at any of the large console suppliers like Sony, Microsoft, and Nintendo would be equally as interested.

Max, as I understand it the IR lasers extend out in a spread pattern. The provide full depth information at thousands of points and combine that with full color video. The Kinect does not just take 2 dimensional pictures of things. It provides distances, widths, heights, and all at a speed that can record high speed movement in real time and reconcile that with a full color picture. This system could be used in FIRST to emulate the Lidar used on so many DARPA vehicles for full field navigation.

Indeed, David, this system would be ideal for a fully autonomous robot. How's that going, by the way?

The software and hardware has already been developed by thousands of engineers over a number of years, using technology by many different countries. It works in ANY ambient light condition because of the IR - even in nearly pitch black. It's accurate from around 1.5 feet to 20 feet - 1/3 of one of our fields.

Max, that's really pretty exciting stuff. The year you put one of these on the robots and use it in autonomous is the year you win the innovation award available at every competition you attend. Awesome. I'm sorry to see FIRST rule it out this year, though I can't see that rule in place for long.