Video: Driving Miss Daisy...with a Kinect

[Jared Russell]

http://forums.usfirst.org/showthread.php?p=52782#post52782

:ahh:

[Travis Hoffman]

I shudder to click this link, anticipating the carnage that lies ahead.

.....the horror! Wait, that was a trailer for "The Thing".

Thanks for sharing.

I presume we are seeing fixed-speed commands - not any analog "go faster....I said faster.....FASTER DANGIT!....Wait!....Slow down!....Slower!....Not the Grandma in the front row!.......STOOOOOOOOOP!" stuff. Right?

Maybe that should be the benchmark for ever using this on a competition field to control a drivetrain. Put a grandma covered in copious amounts of pool noodles and protective gear at one end of the field. Have the robot drive at her at full speed, then turn away at the last moment, do 10 complete circles around her in each direction, and return. If grandma remains upright and/or doesn't run away screaming in terror, then roll with it.

You may substitute any member of the GDC for Grandma, if you like. :-P

I think it would be really cool to see the sensor control an arm to hang tubes - seems like a more natural fit. Can't wait to see the vid for that!

[Jared Russell]

They are analog commands, but we capped the maximum output at 20% (so it's only analog until your arms are 20% above or below horizontal). Functionally very similar since the motors don't begin to move until ~10-15% throttle has been applied.

[BrendanB]

Very cool!

How is it practically? Lag time, user error, multiple people/interference, etc?

[Jared Russell]

Very cool!

How is it practically? Lag time, user error, multiple people/interference, etc?

You can get a sense of lag time from the video (both best case and worst case - Netbeans was just finishing opening up on the netbook as we began this trial and was hogging some CPU cycles; by the middle of the video you will notice latency was in the half a second range).

User error was a bit of a factor. The "stop it right now!" command is basically folding your arms, but you'll notice me reminding the student at one point not to simply drop her arms to her side - that is full reverse! With a few minutes of practice it becomes more natural.

We plan on doing some stress testing over coming weeks to examine the "multiple people/interference" issue, as well as to see if we can further reduce the lag with a more powerful computer.

[Jared Russell]

I think it would be really cool to see the sensor control an arm to hang tubes - seems like a more natural fit. Can't wait to see the vid for that!

(You Ninja-edited while I was typing my response)

It's coming. I agree that that will be a much more natural fit.

[Travis Hoffman]

Very cool!

How is it practically? Lag time, user error, multiple people/interference, etc?

I'd definitely want to see a simulation of the competition experience with background movement, etc. to see how it "rejects" everything but the actual movements of the person controlling it. Don't forget to add the neon shirts, facepaint, glitter, and metallic wigs for a genuine experience. Oh, and flowery foam headgear. :)

[Cyberphil]

Awesome stuff! 341 never disappoints.

When did you guys receive this, and how many hours did you spend making it work? I am very curious to see the ratio here.

[Andrew Lawrence]

At this point in testing, do you see it as a viable control system for a competition? Or at least for anything in the competition?

[nightpool]

Quite cool. Seems a little awkward to be just waving your arms about though. Have you looked into anything else? See for example this Youtube video (http://www.youtube.com/watch?v=WlKcgtABcqk), which shows a hybrid control solution, with a Xbox controller combined with the Kinect in a FPS-like environment. He seems a little stiff at the beginning, but watch towards the end; he becomes a lot more natural with his movements and it looks quite fluid.



Also, does anyone know if you can embed youtube videos in posts? That'd be really cool.

[plnyyanks]

This is really cool. I'm gonna have a hard time waiting 84 days until I get my own kinect to play with. In the meantime, keep up the good work!

Also, does anyone know if you can embed youtube videos in posts? That'd be really cool.
Seconded. Brandon, if you get the time, this would be great.

[Jared Russell]

When did you guys receive this, and how many hours did you spend making it work? I am very curious to see the ratio here.

We got the Kinect a little over a week ago. I installed the drivers and updated the software a couple days ago, and last night was the first time we took the Kinect out of the box. In about a half an hour of concerted testing we were able to go from a brand new, blank Java project to what you see in the video.

[Travis Hoffman]

Quite cool. Seems a little awkward to be just waving your arms about though.

It's better in "airplane mode" than sticking your hands in front of and behind you. Certain upward-swept arm angles in front of the operator would lead to some rather upset uninformed spectators, I'd imagine. :rolleyes:

This is a good point though - seems like this control method requires a lot of room, which isn't typically in ample supply behind the glass.

[rcmolloy]

As soon and you guys mentioned this earlier Jared, I knew the team was going to actually do it. Props to you and 341 for showing the actual community what the Kinect will most likely be used for during the season.

[pfreivald]

It seems to me that joystick (or Xbox controller or whatever) operation for the drive train would be much more intuitive, but that Kinect could be really, really cool for operating some kind of arm or lifting device!

[Nick Lawrence]

Impressive.

-Nick

[MagiChau]

Thanks for the demonstration. Does the default software for controlling the robot drive map the current arm position for the motor speed? I know the arms act like virtual joysticks but I do not know if the program is detecting "movement" or "position" of the arm.

[wilsonmw04]

One of my students noticed this: your "reverse" command is your hands at your side. That position seems to be a natural rest state for most folks. That seems like it could be rather dangerous if you relaxed without the robot being disabled.

I don't want to take anything away from how impressed my team is with this. Just food for thought.

[Ninja_Bait]

What I want to see next is someone controlling swerve or mechanum by using one arm as a direction control and one arm as a speed control (imagine a clock or semaphore-like control system)

[Arjun Namineni]

Did you guys place a certain type of background/wall cover behind your driver? If so, what was it?

[Jared Russell]

Thanks for the demonstration. Does the default software for controlling the robot drive map the current arm position for the motor speed? I know the arms act like virtual joysticks but I do not know if the program is detecting "movement" or "position" of the arm.

It is detecting the position of the arm as well, but because we are limiting the drive speed, you can't really tell. The video that 294 just posted shows off this feature much more clearly.

[Retired Starman]

Sorry, but I just have to give my first impression--which is---

They finally found a way to get nerds to do calisthenics!

Like I said, Sorry!

[Brandon Zalinsky]

Very cool! I'd be interested to see if anyone uses the Kinect fully in the 2012 season.

[FTC4397]

At this point in testing, do you see it as a viable control system for a competition? Or at least for anything in the competition?

Can Team #0341 post a link to this video?

[OZ_341]

Can Team #0341 post a link to this video?

Here is the link
http://www.youtube.com/watch?v=6OFzdeCfbp4

[FTC4397]

Here is the link
http://www.youtube.com/watch?v=6OFzdeCfbp4

Thank You

[JesseK]

Kinect has some impressive possibilities for controlling a high DoF arm (think 1731 in 2007, from the "Behind the Design" book) ... impressive, until the driver sneezes. Be careful with that thing! Thanks for the video.

Iteration 2 (IMO) should weed out background noise so that the Kinect doesn't accidentally translate the coach's jumping up and down as a dance the robot should perform.

[Mike Rizzo]

Now drive it doing the Macarena

[Jared Russell]

Here is video of our demonstration of the Kinect at Ramp Riot. It took two tries, but we successfully scored a tube using the Kinect to both drive and operate our arm.

http://www.youtube.com/watch?v=8qSt1ZUmtjI

[Chris is me]

That spam is brilliant. Jared truly is a superb actor!

[James Rutheford]

I might not use the Kinect in competition, just because of the time it takes the robot to respond, but I would use this for recruiting purposes :D This is simply amazing!

[OZ_341]

I might not use the Kinect in competition, just because of the time it takes the robot to respond, but I would use this for recruiting purposes :D This is simply amazing!

What Jared found is that changing laptops can greatly improve your response time.
We have no idea how they will use the Kinect or if they will use the Kinect in the 2012 game.
However my guess would be that the GDC will try to make it "compelling" to use the Kinect if it's part of the game. In other words they may give you the option, but make the task very hard to achieve with straight autonomous. Again this is all guess work.

[giantmidget31]

This looks really awesome!

My only concern is e-stop. If you have the kinect on the driver station table, and you are standing as far away from it as you are in the video, you will have to walk a considerable distance to get to the e-stop. One solution I am thinking of is a wearable e-stop button.

Great work!