usb connections and Ethernet connections
usb -> kinect
Ethernet -> crio sending basic commands like aim left or right
what operating system you might ask
puppy linux or tiny core for their capability of loading all data from a flashdrive to ram so if a cold shutdown occurs it doesnt mess up your harddisk
with mono in your linux machine you can start up an exe file developed in C# on your programming computer and transfer it from the bin file
have the exe file start up when your linux box starts send commands over Ethernet saying aim left aim right
complexity 8/10
benefits over regular camera 2/10
bragging rights 10/10
the reason i posted this online is because i want it to be done but it appears our team has decided not to do it.
Does this sound like a good method and what do you guys think
The Kinect uses patterned light and a camera to produce a depth map. If environmental factors don’t interfere with it, it gives similar data to very expensive LIDAR sensors. If used appropriately, this is great to use to simplify the image processing and make independent measurements that can improve the interpretation of the vision code.
Of course this is only helpful if you make it reliable and solve all of the issues listed in the previous posts.
If you can use mySQL, the comms between the crio and kinect will be relatively simple.
You cannot see the retro tape, but you can see the border tape. I’m currently using depth / color intersection and a couple of std dev passes to find the backboard in less than ideal lighting situations.
Depending on your algorithm this may actually be a good thing. The openCV libraries are a bit overkill for this years challenge.
the benefits over a regular camera are 7/10 in my book. see my notes below.
I was able to get a perfect vision tracking system working last year by the end of week one. This thing tracked to the target every time. It was gorgeous. Then I got to competition and realized that all my assumptions were wrong. Lighting, reflections, and angles all changed on the real field. Unless you have a legitimate field mock up with arena lighting, get ready for a world of hurt when you get to competition, and realize you get little to no time to calibrate to the field, let alone correct your assumptions that were wrong.
The kinect helps to design a system with a much better tolerance. I would recommend testing your system in a very dimly lit room at night, and a very bright room during the day. If your system works in both, you are much more likely to have a system that will work at competition. A system with this flexibility is very difficult to achieve with just a camera and a light source. Especially from 12’ away with all of the other reflections and crowd colors on the legitimate FIRST playing field.
The kinect gives me some key advantages. First off I can filter out the crowd and drivers from view. Second it helps me to separate the backboard from any other playing field object by taking the intersection of depth and color.
I don’t know of any teams that were successful last year with the retro reflective tape. I believe a few will be this year, however I strongly believe that teams would be much more successful with the kinect, because there is a strong sdk backing it, and it is less susceptible to environmental variables.
You guys talk of nice database solutions and full O/S systems. But if your autonomous mode has to depend on an O/S that takes fourteen seconds to boot, then 20-30 seconds for PostGRE, mySQL, or some other crap DB to fully initialize to a usable state, then don’t expect it to work in a timely manner. I deliver these type of systems as part of my projects for work, and quite honestly they never go on systems that go from cold boot to full use in under several minutes. Even teams who are very organized sometimes get stuck with a match or two where they’re rushed and have 30 seconds to set the robot down and get behind the wall.
My advice – hack the API and wrestle it into doing what you need it to do (point clouds). There’s the real creds. Don’t do it in the context of ‘sticking it to Microsoft’; that’s tacky and unproductive. Do it because it’s quite literally the best technological way to do what you want to do.
Of course, this assumes you have a full field setup so you can adjust the point cloud to something that’s remotely useful and non-noisy when facing a wall under sometimes blinding light.
One.
Why go through all this mess.
Now if I’m wrong then so be it.
There are existing arduino implementations of using the kinect as a sensor.
SO how did they do it?
What my electrical team has suggested is a Arduino USB shield wired to the I2C on the digital sidecar the USB host can be programmed to decode the kinect.data. into serial data.
Look up the OpenKinect project and there is a quadcopter programmed to detect obstacles by using the kinect.
The reason I choose puppy linux was because it has the capability to boot in 30 seconds and can load its whole operating system into ram.
We thought about using windows but its big and clunky to boot and we all know if we simply cold shutdown a windows computer it will want to check the disk for errors
The arduino was one of my first thoughts but I wasnt sure if it could take it, but i also saw a team using the beaglebone and had successful results
It is a little off topic, but I’d like to hear more about this perfect system that broke when it got to the real field. What did it process? Did it use the retroreflective tape? Why did it break?
I think that sort if information is very valuable to the teams trying to do a vision system. A list of tests that likely make your code fail is often more valuable than anything else you can be given.
Personally, I would want to see the Kinect solution work outside on a cloudy day and with fluorescent and other lights behind the DS wall and shining into the camera. Those are the environmental factors that will be challenging when using the Kinect. We can’t see IR, so we don’t really know IR pollution until we use a special camera. Personally, I’d love to see teams succeed with Kinect and with camera, and with IMUs. Just make auto exciting to watch.
^This.
If people would take a look at a scene containing retroreflective tape with the IR Feed from the Kinect, I think they would instantly understand why the Kinect is awesome.
IR pollution can be an issue. But since we are looking for rectangles, it is easy to filter out other objects since most light sources aren’t rectangles
Another huge advantage over a single camera system is being able to get a depth value for every pixel in the image. You can glean a LOT of informaion from this and you need no reference object in the scene.
http://www.roborealm.com/
Contact them with your team name, location, website, and number.
The mentor of Team 443 will email you back about how they have already been using the Kinect ON their robot.
The biggest mistake we made was mounting the camera low. Essentially last year there was no reason to have a bottom peg with reflective tape, because you would also get the reflection off the aluminum plates of the driver stations. Of course we had built our tracking system looking at a black plate that didn’t reflect as much as the FIRST field. Had we mounted the camera at the top of the robot we may have had a different experience, but we also would have been blinding our drivers. This year you wont be blinding your drivers, but you may be blinding the other drivers, which may or may not be against the rules.
So the first piece of advice I could give is mount your camera as high as you can.
Second would be look at this frame from the field over and over
One question I have is how does the half inch smoked poly look to a kinect or a camera. The camera may pick up the supports of the baskets and the player station framing.
Finally even though you are using your own light source, the further out you get, the more impact ambient light will have. I would test you vision system from every distance you intend to use it at.
If it can’t be calibrated in 5 minutes, you’ll probably have a tough time calibrating it at competition.
It a team posts an easy enough way to do this with detailed instructions for C++ (I’m good with computers but not someone who could figure this stuff out on my own). I would be glad to give it a try since I got a Kinect at my house and we can’t get a new camera right now…
The FIRST field is indeed shiny. It has aluminum diamond plate all along the wall, and lexan up above. Lexan is both transparent and reflective. All polished surfaces, like glass, lexan, aluminum, and painted surfaces will reflect to some degree, and that means that overhead lights, and lights from the robots will show up where a reflection point (where the camera and light reflect about the surface normal). Furthermore, the lexan doesn’t prevent lights from shining through the back surface, so lights in the stands, windows and other light sources can shine into the camera through the driver wall of lexan. And there are also the t-shirts, hats, and other gear worn by the drivers that are showing through the lexan. It all leads to an image that is quite difficult for simple processing to deal with.
This is why I wouldn’t recommend looking just for color. If you combine color and shape info you will be far more robust. This will almost always reject glare, even from the diamond plate. If the shape info is robust enough, you don’t even need to use color, just brightness.
The camera will most definitely pick up the hoop, net, and supports. Camera placement is indeed important in part due to the hoop and net blocking the reflective tape. As shown on the targets in the paper, the lower edge will be the first to be impacted.
As for blinding drivers, I don’t think the LEDs need to be very bright. Certainly they aren’t as bright as the BFL.
Microsoft has rules prohibiting using the kinect on non Win7 devices, our team is using a PICO-ITX setup for our image processing but we are not planning on using the kinect because of microsofts rules about usage of the kinect…here is a link to the onboard computer we are using wich is much smalle than the ones mentioned http://www.viaembedded.com/en/products/boards/productDetail.jsp?productLine=1&id=1310&tabs=1
if the kinect uses a density of how many dots per square inches having 2 kinects would double the amount of dots and effectively making the kinect think the distance is shorter than what it really is
There’s been about three or four threads about this before this one. The consensus is that it is generally a bad idea. In fact, the shoehorning of the Kinect into this year’s competition was less than tactful, and as far as I can tell has only succeeded in upsetting people due to its uselessness.
Do you really want to stick it on the robot? Well… Fine, but you probably won’t get much more data from it than you would by just using a camera. You already have distance, based on the starting position. Alignment you would get from the normal camera.
You’ll need to spend money to do this. In one of the other threads, someone mentioned a USB-ethernet converter/interface. That would probably be the best option, but it was about $125 or so. Another would be the Panda Board, at closer to $200, but you would need to program it, it is an embedded computer.
Teams should stop getting hung up on the idea of using the Kinect as some kind of uber-sensor. The data will simply not help that much. I would keep it for another time and use it in an off-season project.
