Go to Post [I]'Twas the night before ship day, And all through the shop, Not a creature was stirring, Not even[/I] -- Naah, they'll never believe that. - GaryVoshol [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 31-01-2017, 13:39
sgk525 sgk525 is offline
Registered User
FRC #5465
 
Join Date: Jan 2016
Location: Chandler, AZ
Posts: 4
sgk525 is an unknown quantity at this point
Clarity on Smart Dashboard Camera Mechanism

I have developed a straight forward vision code on the Raspberry Pi that grabs a frame, applies masking/Canny Edge Detection, and contour fitting that will send a udp string of distance measurements to the roborio. The main issue I am having is sending an image from the pi to the SmartDashboard. I have been reading about mjpeg streaming, and touched on the mjpeg streamer by robotpy, although am a little confused on how the smartdashboard will grab the frames. I tweaked a stream i found on-line for python3, found here:

https://github.com/SachinKonan/Windo.../httpserver.py

I was wondering how it could be changed to interface with the Smart Dashboard.
Reply With Quote
  #2   Spotlight this post!  
Unread 31-01-2017, 16:19
Peter Johnson Peter Johnson is offline
WPILib Developer
FRC #0294 (Beach Cities Robotics)
Team Role: Mentor
 
Join Date: Jan 2010
Rookie Year: 2008
Location: Redondo Beach, CA
Posts: 268
Peter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud of
Re: Clarity on Smart Dashboard Camera Mechanism

Quote:
Originally Posted by sgk525 View Post
I have developed a straight forward vision code on the Raspberry Pi that grabs a frame, applies masking/Canny Edge Detection, and contour fitting that will send a udp string of distance measurements to the roborio. The main issue I am having is sending an image from the pi to the SmartDashboard. I have been reading about mjpeg streaming, and touched on the mjpeg streamer by robotpy, although am a little confused on how the smartdashboard will grab the frames. I tweaked a stream i found on-line for python3, found here:

https://github.com/SachinKonan/Windo.../httpserver.py

I was wondering how it could be changed to interface with the Smart Dashboard.
That code as-is should work fine with SmartDashboard, as it appears to be serving up just a standard MJPEG over HTTP stream. For competition you'll want to change the port number from 9090 to one of the ones allowed through the field radio. You'll need to use the mjpg streamer widget rather than the CameraServer widget, and put in the URL manually (e.g. http://10.x.y.z:9090/stream.mjpg), as the CameraServer widget uses information from NetworkTables to find the camera stream URL, and that won't be present in this instance (unless you put it there).

Note this code is inefficient in that it's decompressing and recompressing the image from the camera. The Raspberry Pi likely has enough horsepower to do this, particularly at lower resolutions, but I wanted to note this fact because you'll see more CPU use of this approach versus others for this reason.
__________________
Author of cscore - WPILib CameraServer for 2017+
Author of ntcore - WPILib NetworkTables for 2016+
Creator of RobotPy - Python for FRC

2010 FRC World Champions (294, 67, 177)
2007 FTC World Champions (30, 74, 23)
2001 FRC National Champions (71, 294, 125, 365, 279)

Last edited by Peter Johnson : 31-01-2017 at 16:22.
Reply With Quote
  #3   Spotlight this post!  
Unread 31-01-2017, 18:16
sgk525 sgk525 is offline
Registered User
FRC #5465
 
Join Date: Jan 2016
Location: Chandler, AZ
Posts: 4
sgk525 is an unknown quantity at this point
Re: Clarity on Smart Dashboard Camera Mechanism

Thanks, so what will basically happen is I will start an MPEG server on my raspberry pi that is updating frames and sending to the the url you specified, the roborio's background CameraServer class will grab these frames according to the url (http://10.x.y.z:9090/stream.mjpg), and will then send the images to the smartdashboard on the computer?

Also, what might be a faster way of uploading these frames to byte format, because even on my desktop, I am noticing significant lag.
Reply With Quote
  #4   Spotlight this post!  
Unread 31-01-2017, 18:27
Hjelstrom's Avatar
Hjelstrom Hjelstrom is offline
Mentor
FRC #0987 (High Rollers)
Team Role: Mentor
 
Join Date: Mar 2008
Rookie Year: 2005
Location: Las Vegas
Posts: 148
Hjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond reputeHjelstrom has a reputation beyond repute
Re: Clarity on Smart Dashboard Camera Mechanism

Quote:
Originally Posted by Peter Johnson View Post
That code as-is should work fine with SmartDashboard, as it appears to be serving up just a standard MJPEG over HTTP stream. For competition you'll want to change the port number from 9090 to one of the ones allowed through the field radio. You'll need to use the mjpg streamer widget rather than the CameraServer widget, and put in the URL manually (e.g. http://10.x.y.z:9090/stream.mjpg), as the CameraServer widget uses information from NetworkTables to find the camera stream URL, and that won't be present in this instance (unless you put it there).

Note this code is inefficient in that it's decompressing and recompressing the image from the camera. The Raspberry Pi likely has enough horsepower to do this, particularly at lower resolutions, but I wanted to note this fact because you'll see more CPU use of this approach versus others for this reason.
Its pretty useful to edit the images before you send them. You can draw the OpenCV contours on them, put data on them, etc.
Reply With Quote
  #5   Spotlight this post!  
Unread 31-01-2017, 18:49
Peter Johnson Peter Johnson is offline
WPILib Developer
FRC #0294 (Beach Cities Robotics)
Team Role: Mentor
 
Join Date: Jan 2010
Rookie Year: 2008
Location: Redondo Beach, CA
Posts: 268
Peter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud of
Re: Clarity on Smart Dashboard Camera Mechanism

Quote:
Originally Posted by sgk525 View Post
Thanks, so what will basically happen is I will start an MPEG server on my raspberry pi that is updating frames and sending to the the url you specified, the roborio's background CameraServer class will grab these frames according to the url (http://10.x.y.z:9090/stream.mjpg), and will then send the images to the smartdashboard on the computer?

Also, what might be a faster way of uploading these frames to byte format, because even on my desktop, I am noticing significant lag.
No, the SmartDashboard will connect directly to the Raspberry Pi, bypassing the roboRio entirely.

OpenCV does not provide a way to avoid the decompress+recompress. If you aren't actually editing the images being sent to the dashboard, you can use the cscore library that we use on the roboRio to both send the original camera JPEG via HTTP in the lowest overhead way possible and make it available for OpenCV processing. There's a Java tutorial for this here: http://wpilib.screenstepslive.com/s/...essing-in-java. The RobotPy folks are working on a cscore wrapper that should make this possible to do in Python, but there's not an existing tutorial that I know of.
__________________
Author of cscore - WPILib CameraServer for 2017+
Author of ntcore - WPILib NetworkTables for 2016+
Creator of RobotPy - Python for FRC

2010 FRC World Champions (294, 67, 177)
2007 FTC World Champions (30, 74, 23)
2001 FRC National Champions (71, 294, 125, 365, 279)
Reply With Quote
  #6   Spotlight this post!  
Unread 31-01-2017, 19:11
sgk525 sgk525 is offline
Registered User
FRC #5465
 
Join Date: Jan 2016
Location: Chandler, AZ
Posts: 4
sgk525 is an unknown quantity at this point
Re: Clarity on Smart Dashboard Camera Mechanism

I think I understand now. So I will replace line 34 of my server code with:

self.wfile.write('<img src="http://10.x.y.z:9090/stream.mjpg" height="240px" width="320px"/>')

and thread the server_serve_forever() statement, so that the pi can simultaneously process and set the images in a global variable, perhaps a queue.

So just for clarification, there isn't much I will have to do on my end other than set up a mjpeg stream with an appropriate send port and ip address. The smart dashboard will do the rest by finding the url and grabbing the images.
Reply With Quote
  #7   Spotlight this post!  
Unread 31-01-2017, 19:27
Peter Johnson Peter Johnson is offline
WPILib Developer
FRC #0294 (Beach Cities Robotics)
Team Role: Mentor
 
Join Date: Jan 2010
Rookie Year: 2008
Location: Redondo Beach, CA
Posts: 268
Peter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud of
Re: Clarity on Smart Dashboard Camera Mechanism

Quote:
Originally Posted by sgk525 View Post
I think I understand now. So I will replace line 34 of my server code with:

self.wfile.write('<img src="http://10.x.y.z:9090/stream.mjpg" height="240px" width="320px"/>')

and thread the server_serve_forever() statement, so that the pi can simultaneously process and set the images in a global variable, perhaps a queue.

So just for clarification, there isn't much I will have to do on my end other than set up a mjpeg stream with an appropriate send port and ip address. The smart dashboard will do the rest by finding the url and grabbing the images.
To be clear, while the format of the URL is what's in the src= portion above, SmartDashboard doesn't use or hit that line of code at all. SmartDashboard will try to connect to whatever URL you configure in its settings. The important line in your code is line 49, which sets up the HTTP server on a particular port.
__________________
Author of cscore - WPILib CameraServer for 2017+
Author of ntcore - WPILib NetworkTables for 2016+
Creator of RobotPy - Python for FRC

2010 FRC World Champions (294, 67, 177)
2007 FTC World Champions (30, 74, 23)
2001 FRC National Champions (71, 294, 125, 365, 279)
Reply With Quote
  #8   Spotlight this post!  
Unread 31-01-2017, 21:53
sgk525 sgk525 is offline
Registered User
FRC #5465
 
Join Date: Jan 2016
Location: Chandler, AZ
Posts: 4
sgk525 is an unknown quantity at this point
Re: Clarity on Smart Dashboard Camera Mechanism

So I have been looking at the smartdashboard camera viewer properties and it asks for the "camera's ip address or mdns name". Here I can feed the URL: "http://10.x.y.z:9090/stream.mjpg" and should it successfully make a connection to the Dash? Should the x, y, and z have integer values? Is it possible to set a mdns name for the http web server?

Also the the ports that are allowed for team use ranges from "5800-5810"
Reply With Quote
  #9   Spotlight this post!  
Unread 01-02-2017, 00:40
Joe Ross's Avatar Unsung FIRST Hero
Joe Ross Joe Ross is offline
Registered User
FRC #0330 (Beachbots)
Team Role: Engineer
 
Join Date: Jun 2001
Rookie Year: 1997
Location: Los Angeles, CA
Posts: 8,600
Joe Ross has a reputation beyond reputeJoe Ross has a reputation beyond reputeJoe Ross has a reputation beyond reputeJoe Ross has a reputation beyond reputeJoe Ross has a reputation beyond reputeJoe Ross has a reputation beyond reputeJoe Ross has a reputation beyond reputeJoe Ross has a reputation beyond reputeJoe Ross has a reputation beyond reputeJoe Ross has a reputation beyond reputeJoe Ross has a reputation beyond repute
Re: Clarity on Smart Dashboard Camera Mechanism

Quote:
Originally Posted by sgk525 View Post
So I have been looking at the smartdashboard camera viewer properties and it asks for the "camera's ip address or mdns name". Here I can feed the URL: "http://10.x.y.z:9090/stream.mjpg" and should it successfully make a connection to the Dash? Should the x, y, and z have integer values? Is it possible to set a mdns name for the http web server?

Also the the ports that are allowed for team use ranges from "5800-5810"
Have you updated to the 2017 plugins? It sounds like you have an old smartdashboard.
Reply With Quote
  #10   Spotlight this post!  
Unread 01-02-2017, 03:21
Peter Johnson Peter Johnson is offline
WPILib Developer
FRC #0294 (Beach Cities Robotics)
Team Role: Mentor
 
Join Date: Jan 2010
Rookie Year: 2008
Location: Redondo Beach, CA
Posts: 268
Peter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud ofPeter Johnson has much to be proud of
Re: Clarity on Smart Dashboard Camera Mechanism

Quote:
Originally Posted by sgk525 View Post
So I have been looking at the smartdashboard camera viewer properties and it asks for the "camera's ip address or mdns name". Here I can feed the URL: "http://10.x.y.z:9090/stream.mjpg" and should it successfully make a connection to the Dash? Should the x, y, and z have integer values? Is it possible to set a mdns name for the http web server?

Also the the ports that are allowed for team use ranges from "5800-5810"
I would not recommend using mDNS with coprocessors. Go with fixed IPs in the 10.team.number.xx range. Yes, for IP addresses, x, y, and z are integers in the range 0-255, but for use on the field, x and y should be derived from the team number (e.g. x=2, y=94 for team 294), and z should not be 0, 255, or 1-5.

The ports allowed for team use are listed in R56. Ports 5800-5810 are one option; another is port 80.
__________________
Author of cscore - WPILib CameraServer for 2017+
Author of ntcore - WPILib NetworkTables for 2016+
Creator of RobotPy - Python for FRC

2010 FRC World Champions (294, 67, 177)
2007 FTC World Champions (30, 74, 23)
2001 FRC National Champions (71, 294, 125, 365, 279)
Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 01:14.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi