Go to Post FIRST robotics: Not just for nerds. - Koko Ed [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rate Thread Display Modes
  #16   Spotlight this post!  
Unread 20-12-2013, 22:30
gluxon's Avatar
gluxon gluxon is offline
\n
AKA: Brandon Cheng
FRC #0178 (The 2nd Law Enforcers)
Team Role: Leadership
 
Join Date: Apr 2012
Rookie Year: 2011
Location: Connecticut
Posts: 65
gluxon has a spectacular aura aboutgluxon has a spectacular aura aboutgluxon has a spectacular aura about
Re: Fixing opencv lag

Request a video stream from VAPIX with the "fps" HTTP parameter. Details are on page 12 of the VAPIX API doc -> http://www.axis.com/files/manuals/va...37_en_1307.pdf

And if it helps, there's an incomplete Node.js implementation of VAPIX that supports a video stream with the slowed down fps you need. https://github.com/gluxon/node-vapix

This answers your question, although I feel like the problem will still occur, just over a longer period of time with this change.

Edit: I would actually go with what virtuald said in his post for the solution. Sounds like a more complete way of handling things.

Last edited by gluxon : 20-12-2013 at 22:33.
  #17   Spotlight this post!  
Unread 21-12-2013, 12:06
virtuald's Avatar
virtuald virtuald is offline
RobotPy Guy
AKA: Dustin Spicuzza
FRC #1418 (), FRC #1973, FRC #4796, FRC #6367 ()
Team Role: Mentor
 
Join Date: Dec 2008
Rookie Year: 2003
Location: Boston, MA
Posts: 1,102
virtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant future
Re: Fixing opencv lag

Quote:
Originally Posted by billbo911 View Post
I would definitely like to hear more about this approach.
I am no expert on the efficient use of OpenCV, and what you are proposing just might be the key to get rid of the minor delay we are seeing in our images. We can process 15-20 fps, but there is a small, but noticeable lag in our images.

We are using a USB camera and a PCDuino, but otherwise it is mostly the same.
Our implementation is available on my website if you want to look at it. Look for kwarqs2013.py.
__________________
Maintainer of RobotPy - Python for FRC
Creator of pyfrc (Robot Simulator + utilities for Python) and pynetworktables/pynetworktables2js (NetworkTables for Python & Javascript)

2017 Season: Teams #1973, #4796, #6369
Team #1418 (remote mentor): Newton Quarterfinalists, 2016 Chesapeake District Champion, 2x Innovation in Control award, 2x district event winner
Team #1418: 2015 DC Regional Innovation In Control Award, #2 seed; 2014 VA Industrial Design Award; 2014 Finalists in DC & VA
Team #2423: 2012 & 2013 Boston Regional Innovation in Control Award


Resources: FIRSTWiki (relaunched!) | My Software Stuff
  #18   Spotlight this post!  
Unread 21-12-2013, 12:11
virtuald's Avatar
virtuald virtuald is offline
RobotPy Guy
AKA: Dustin Spicuzza
FRC #1418 (), FRC #1973, FRC #4796, FRC #6367 ()
Team Role: Mentor
 
Join Date: Dec 2008
Rookie Year: 2003
Location: Boston, MA
Posts: 1,102
virtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant future
Re: Fixing opencv lag

Quote:
Originally Posted by sst.thad View Post
As for grabbing the image directly, openCV has no way of just opening a jpg over the network.
Not true. If you have ffmpeg support enabled (which it is by default in most opencv builds), you can retrieve the mjpeg stream directly from the camera via FFMPEG. Something similar to the following (error checking omitted):

Code:
vc = cv2.VideoCapture()
vc.open('http://%s/mjpg/video.mjpg' % camera_host)

h = vc.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH)
w = vc.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT)

capture_buffer = np.empty(shape=(h, w, 3), dtype=np.uint8)

while True:
    retval, img = vc.read(capture_buffer)
__________________
Maintainer of RobotPy - Python for FRC
Creator of pyfrc (Robot Simulator + utilities for Python) and pynetworktables/pynetworktables2js (NetworkTables for Python & Javascript)

2017 Season: Teams #1973, #4796, #6369
Team #1418 (remote mentor): Newton Quarterfinalists, 2016 Chesapeake District Champion, 2x Innovation in Control award, 2x district event winner
Team #1418: 2015 DC Regional Innovation In Control Award, #2 seed; 2014 VA Industrial Design Award; 2014 Finalists in DC & VA
Team #2423: 2012 & 2013 Boston Regional Innovation in Control Award


Resources: FIRSTWiki (relaunched!) | My Software Stuff
  #19   Spotlight this post!  
Unread 21-12-2013, 12:43
magnets's Avatar
magnets magnets is offline
Registered User
no team
 
Join Date: Jun 2013
Rookie Year: 2012
Location: United States
Posts: 748
magnets has a reputation beyond reputemagnets has a reputation beyond reputemagnets has a reputation beyond reputemagnets has a reputation beyond reputemagnets has a reputation beyond reputemagnets has a reputation beyond reputemagnets has a reputation beyond reputemagnets has a reputation beyond reputemagnets has a reputation beyond reputemagnets has a reputation beyond reputemagnets has a reputation beyond repute
Re: Fixing opencv lag

Quote:
Originally Posted by sst.thad View Post
The reason we are on a beaglebone instead of the dashboard is i have heard stories of FTA's requiring dashboards to get shut off, and we don't want to be stuck if that happens. So we want it to be onboard.
This.

In 2012, I had to tell a student that the vision software they worked on was not allowed because the FTA wouldn't let us use it. Then, we were stupid enough to try vision again in 2013, thinking that the "bandwidth limit" (which doesn't work) would prevent the FTA from banning the dashboard, but no, the FTA said we weren't allowed to use our vision stuff yet again, but this time, because a robot on the field had stopped working "because we sabotaged their network connection". What really happened? The other teams battery fell out. Not our problem, but we still couldn't use vision. Vision is not a guaranteed ability for robots. Nor are dashboards, or any amount of communication.
  #20   Spotlight this post!  
Unread 21-12-2013, 13:42
billbo911's Avatar
billbo911 billbo911 is offline
I prefer you give a perfect effort.
AKA: That's "Mr. Bill"
FRC #2073 (EagleForce)
Team Role: Mentor
 
Join Date: Mar 2005
Rookie Year: 2005
Location: Elk Grove, Ca.
Posts: 2,384
billbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond repute
Re: Fixing opencv lag

Quote:
Originally Posted by magnets View Post
... Vision is not a guaranteed ability for robots. Nor are dashboards, or any amount of communication.
I think the crux of this thread points out the error in your statement.

Yes, it is really a shame when a feature that has been pushed by FIRST is taken away, for any reason, and that feature is part of the major design a team has undertaken. The point of this thread, on the other hand, is how to avoid this possibility entirely.

Here is why this thread addresses this. If the vision processing is done by a separate on-board system, Beaglebone in this case and PCDuino in ours, it removes the wireless network entirely. All acquisition of images, processing and communication of target information remains on the robot. Honestly, for much less than $200, any team can do this! It just takes time to learn how, and the information is already readily available.
__________________
CalGames 2009 Autonomous Champion Award winner
Sacramento 2010 Creativity in Design winner, Sacramento 2010 Quarter finalist
2011 Sacramento Finalist, 2011 Madtown Engineering Inspiration Award.
2012 Sacramento Semi-Finals, 2012 Sacramento Innovation in Control Award, 2012 SVR Judges Award.
2012 CalGames Autonomous Challenge Award winner ($$$).
2014 2X Rockwell Automation: Innovation in Control Award (CVR and SAC). Curie Division Gracious Professionalism Award.
2014 Capital City Classic Winner AND Runner Up. Madtown Throwdown: Runner up.
2015 Innovation in Control Award, Sacramento.
2016 Chezy Champs Finalist, 2016 MTTD Finalist
  #21   Spotlight this post!  
Unread 21-12-2013, 14:39
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Re: Fixing opencv lag

Quote:
Originally Posted by virtuald View Post
Not true. If you have ffmpeg support enabled (which it is by default in most opencv builds), you can retrieve the mjpeg stream directly from the camera via FFMPEG. Something similar to the following (error checking omitted):

Code:
vc = cv2.VideoCapture()
vc.open('http://%s/mjpg/video.mjpg' % camera_host)

h = vc.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH)
w = vc.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT)

capture_buffer = np.empty(shape=(h, w, 3), dtype=np.uint8)

while True:
    retval, img = vc.read(capture_buffer)
Yeah I know I can recieve the MJPEG stream like this, and that is what I am doing. The camera still pushes frames into the stream at 30 FPS. If the vc.read does not occur faster then 30 times a second, The buffer will start to backlog and lag. In fact your code was what I started from, and I was doing exactly that. If you put at 50ms delay at the end of your processing function you will see that it starts to lag.

When people were talking about requesting images, they are talking about not opening the stream, but just directly requesting a single image, and working with that, so the camera does not push a buffer. Thats the part that does not work natively in openCV.
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 03:21.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi