Go to Post I hope I can do something as awesome as Curiosity when I'm older.. - rachelholladay [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rating: Thread Rating: 2 votes, 5.00 average. Display Modes
  #1   Spotlight this post!  
Unread 04-10-2013, 10:18
ekapalka's Avatar
ekapalka ekapalka is offline
Registered User
FRC #3216
 
Join Date: Dec 2012
Location: Bermuda
Posts: 277
ekapalka has a spectacular aura aboutekapalka has a spectacular aura about
Using a Kinect as a primary camera for drivers

Hello! I was wondering how we would go about streaming images from a Kinect which is plugged into an on-board computer to the driver station. I've been looking at TCP (using C++, because we're using C++ OpenCV) for sending both targeting data (just three floating point numbers) and video, but I'm not entirely sure where to begin. What have other teams done? Do they use 3rd party TCP/UDP/___ libraries or do they write their own? I guess if all else fails we can use the Axis Camera for the driver and simply send targeting data to the cRIO via NetworkTables (or the C++ equivalent). It's bugging me. All the vision processing I've learned so far has just been on my personal computer, because I know that actually sending that information to the cRIO is going to be a challenge of it's own. Any thoughts on the matter would be greatly appreciated!
  #2   Spotlight this post!  
Unread 04-10-2013, 13:59
Invictus3593's Avatar
Invictus3593 Invictus3593 is offline
time you like wasting is not wasted
FRC #3593 (Team Invictus)
Team Role: Leadership
 
Join Date: Jan 2013
Rookie Year: 2010
Location: Tulsa, OK
Posts: 318
Invictus3593 is just really niceInvictus3593 is just really niceInvictus3593 is just really niceInvictus3593 is just really nice
Re: Using a Kinect as a primary camera for drivers

I've been working on C# code to get kinect depth camera and color camera and I've gotten success in recieving the feeds in an onoard computer, but I'm having trouble with networking as well.

I know TCP is the protocol used by the axis camera and the port it uses is 80.If you were going to use just the kinect and no axis camera, you could use port 80 for the data. If you needed another port, look in the manual for guidelines on which ones you can use and select one that is bidirectional, same as 80.

If you can't write your own TCP libraries, i'd definitely find a third-party client to work with.

Hopefully this helps!
__________________
Per Audacia Ad Astra
  #3   Spotlight this post!  
Unread 04-10-2013, 15:01
curtTheGreat curtTheGreat is offline
Registered User
FRC #0930
 
Join Date: Jan 2013
Location: Wisconsin
Posts: 1
curtTheGreat is an unknown quantity at this point
Re: Using a Kinect as a primary camera for drivers

Last year when we used a Kinect sensor, we ran into some troubles with a network based solution. That being said we chose to use the serial port on our CRio. Going serial allowed us to get an uninterrupted signal from our onboard computer to the CRio.
  #4   Spotlight this post!  
Unread 04-10-2013, 15:26
Domtech's Avatar
Domtech Domtech is offline
I Do Everything
AKA: Dominic
FRC #2412 (Robototes)
Team Role: Mentor
 
Join Date: Feb 2010
Rookie Year: 2004
Location: Bellevue WA
Posts: 46
Domtech is an unknown quantity at this point
Re: Using a Kinect as a primary camera for drivers

I played around with using the kinect as a driver camera last year, and I managed to get it working with just some of the shelf freeware:

1. KinectCam, a driver to make the kinect work as a USB webcam.

2. yawcam, a small aplication to host the camera feed over the network.

It worked quit well, but at about a second of lag with the classmate on the robot transmitting over wifi, it was too slow to be useful, and we didn't have any weight allowance left.

Cool part is that KinectCam was able to place a BMP crosshair at a certain "depth" into the image, which was great for aiming.
  #5   Spotlight this post!  
Unread 04-10-2013, 22:26
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Using a Kinect as a primary camera for drivers

There are numerous threads asking how to communicate between devices on the robot.

I've attached an image to show how the networking can be accomplished in LV.

The TCP and UDP icons are in the Data Communication/Protocols directory. When you don't need to receive all data, and are sending smallish amounts, I prefer to use UDP. It is really simple to use. You need to identify how to encode the data. You can use json, xml, or flattened binary. Just have the read side match and have it check for timeout errors.

TCP is similar -- but different. Large amounts of data, above a few Kbytes will typically use TCP instead of UDP, and if you are sending a protocol where all elements must be processed in order, TCP is your friend.

Greg McKaskle
Attached Thumbnails
Click image for larger version

Name:	Clipboard 1.png
Views:	128
Size:	106.9 KB
ID:	15263  
  #6   Spotlight this post!  
Unread 04-10-2013, 23:18
ekapalka's Avatar
ekapalka ekapalka is offline
Registered User
FRC #3216
 
Join Date: Dec 2012
Location: Bermuda
Posts: 277
ekapalka has a spectacular aura aboutekapalka has a spectacular aura about
Re: Using a Kinect as a primary camera for drivers

Quote:
Originally Posted by Greg McKaskle View Post
I've attached an image to show how the networking can be accomplished in LV.
[small, elegant LabView code]
sigh... I really hope it turns out to be like that in C++... or even Java. Actually it wouldn't be too much of a learning stretch (formatting wise) to use java.
  #7   Spotlight this post!  
Unread 05-10-2013, 21:03
yash101 yash101 is offline
Curiosity | I have too much of it!
AKA: null
no team
 
Join Date: Oct 2012
Rookie Year: 2012
Location: devnull
Posts: 1,191
yash101 is an unknown quantity at this point
Re: Using a Kinect as a primary camera for drivers

Why not use the axis camera for the targeting? I think that it may suffice. Also, I believe that there is a framework, openKinect that may be what you are looking for. Also, personally, do you have anyone who I can talk to about openCV? I am trying to learn it.
Thanx, Dev
  #8   Spotlight this post!  
Unread 06-10-2013, 20:46
billylo's Avatar
billylo billylo is offline
Registered User
FRC #0610 (Coyotes)
Team Role: Mentor
 
Join Date: Mar 2012
Rookie Year: 2012
Location: Toronto
Posts: 161
billylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant future
Re: Using a Kinect as a primary camera for drivers

610 explored the use of Kinect and an onboard netbook (running RoboRealm) during the build season. Here are a few highlights:

1. Infra-red beam is strong enough for it to guide our auto-aiming routines (full court and under the pyramid.) The programmers had it working beautifully during practice, and were pretty proud that our robot can auto-aim and shoot in the dark (full court!)

2. Unfortunately, in real competition, the other sources of infra-red lights in the Verizon Arena (BAE) rendered it unusable. The team reverted back to using it as a camera only for driving and manual aiming. We later removed the Kinect and Netbook to make room for our disc-pickup enhancement during Waterloo.

3. Overall, Kinect/Netbook/RoboRealm combo was very promising until we got into the BAE arena. The team may try it again if the new Kinect that comes with XBox ONE is more capable in filtering. RoboRealm is great and we received great support from the product guys.

Feel free to PM me if you need more details.
  #9   Spotlight this post!  
Unread 06-10-2013, 21:30
yash101 yash101 is offline
Curiosity | I have too much of it!
AKA: null
no team
 
Join Date: Oct 2012
Rookie Year: 2012
Location: devnull
Posts: 1,191
yash101 is an unknown quantity at this point
Talking Re: Using a Kinect as a primary camera for drivers

Quote:
Originally Posted by Domtech View Post
I played around with using the kinect as a driver camera last year, and I managed to get it working with just some of the shelf freeware:

1. KinectCam, a driver to make the kinect work as a USB webcam.

2. yawcam, a small aplication to host the camera feed over the network.

It worked quit well, but at about a second of lag with the classmate on the robot transmitting over wifi, it was too slow to be useful, and we didn't have any weight allowance left.

Cool part is that KinectCam was able to place a BMP crosshair at a certain "depth" into the image, which was great for aiming.
This post just solves half the problems that I am having with OpenCV and hacking the kinect
  #10   Spotlight this post!  
Unread 06-10-2013, 21:33
yash101 yash101 is offline
Curiosity | I have too much of it!
AKA: null
no team
 
Join Date: Oct 2012
Rookie Year: 2012
Location: devnull
Posts: 1,191
yash101 is an unknown quantity at this point
Re: Using a Kinect as a primary camera for drivers

According to the post by Domtech, I believe that you can use an RPi with the software that he listed and do the processing on the driver station, probably a more accessible and powerful computer.
  #11   Spotlight this post!  
Unread 07-10-2013, 18:25
billylo's Avatar
billylo billylo is offline
Registered User
FRC #0610 (Coyotes)
Team Role: Mentor
 
Join Date: Mar 2012
Rookie Year: 2012
Location: Toronto
Posts: 161
billylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant futurebillylo has a brilliant future
Re: Using a Kinect as a primary camera for drivers

If you are interested in using Kinect with RoboRealm, here is a good tutorial on the 2013 game.
  #12   Spotlight this post!  
Unread 08-10-2013, 00:50
yash101 yash101 is offline
Curiosity | I have too much of it!
AKA: null
no team
 
Join Date: Oct 2012
Rookie Year: 2012
Location: devnull
Posts: 1,191
yash101 is an unknown quantity at this point
Re: Using a Kinect as a primary camera for drivers

That's Nice. Thank You. I have been trying to find vision tracking resources for many years and now I think that I am close.
  #13   Spotlight this post!  
Unread 08-10-2013, 21:12
ekapalka's Avatar
ekapalka ekapalka is offline
Registered User
FRC #3216
 
Join Date: Dec 2012
Location: Bermuda
Posts: 277
ekapalka has a spectacular aura aboutekapalka has a spectacular aura about
Re: Using a Kinect as a primary camera for drivers

Quote:
Originally Posted by Invictus3593 View Post
I know TCP is the protocol used by the axis camera and the port it uses is 80.

[...] If you can't write your own TCP libraries, i'd definitely find a third-party client to work with.
Thanks! That really does help. I've been looking all over, and I've gotten the impression MJPEG just passed java by, and wasn't really picked up and developed by the java community (which makes finding useful resources a pain...). I've found great libraries for streaming straight JPG images via UDP, but I have a feeling that the DriverStation is pretty locked down and resistant to change. I've also found a few promising and "robust" third party TCP libraries, but I'm almost certain we would end up having to write our own program to encode the series of JPEG images to MJPEG video (I can't find info on this ANYWHERE). Are there any other options for streaming video through port 80 to the driver station (without creating a custom dashboard or something like that)? Thanks!
  #14   Spotlight this post!  
Unread 08-10-2013, 21:16
yash101 yash101 is offline
Curiosity | I have too much of it!
AKA: null
no team
 
Join Date: Oct 2012
Rookie Year: 2012
Location: devnull
Posts: 1,191
yash101 is an unknown quantity at this point
Question Re: Using a Kinect as a primary camera for drivers

Since I am familiar with PHP, can I have nginx and php serve a file to the cRIO? How would I do this?
  #15   Spotlight this post!  
Unread 09-10-2013, 17:00
yara92's Avatar
yara92 yara92 is offline
M.Fawdah Mechanical engineering
AKA: Mohamed
FRC #1946 (Mechka Monster)
Team Role: RoboCoach
 
Join Date: Jan 2007
Rookie Year: 2006
Location: Israel
Posts: 236
yara92 will become famous soon enoughyara92 will become famous soon enough
Re: Using a Kinect as a primary camera for drivers

I found this
[FRC 2012] Basic Kinect Tutorial
[FRC 2013] Kinect Tutorial
Robotic Arm Control Using Kinect Camera
Using the Kinect with LabVIEW and the Upgraded Microsoft .NET API
Kinect use with Code Laboratories drivers
https://decibel.ni.com/content/docs/DOC-15655
__________________
TEAM 1946-Tamra
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 03:36.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi