Go to Post Uncle Sam wants you to watch NERD Vision. - artdutra04 [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 11-12-2015, 20:04
jojoguy10's Avatar
jojoguy10 jojoguy10 is offline
Programming/Controls Mentor
AKA: Joe Kelly
FRC #0997 (Spartan Robotics) and FRC #2990 (Hotwire Robotics)
Team Role: Mentor
 
Join Date: Jan 2013
Rookie Year: 2010
Location: Corvallis, OR
Posts: 298
jojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant future
Vision Processing - Questions

Hello everyone,

We want to start experimenting with Vision Processing with the Raspberry Pi, OpenCV, and a USB camera.

We use Labview to program our robot, but I know C++ enough to work with some OpenCV libraries. I have a couple of questions that I couldn't find the answers to in my search:
  1. How can I send image data from the Raspberry Pi to the RoboRIO (which is running Labview)?
  2. Is the Raspberry Pi even a good choice?
  3. Is OpenCV a better choice than the Labview vision vi?

Thanks,
Joe Kelly
__________________

2013 and 2014 - Student (FRC#2990), Programmer (LabVIEW)
2015 and 2016 - Mentor (FRC#2990), Programming (LabVIEW)
2017-Current - Mentor (FRC#997), Controls/Programming (Java)
2015-Current - A/V Volunteer at Oregon events
  #2   Spotlight this post!  
Unread 11-12-2015, 20:39
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Re: Vision Processing - Questions

Quote:
Originally Posted by jojoguy10 View Post
Hello everyone,

We want to start experimenting with Vision Processing with the Raspberry Pi, OpenCV, and a USB camera.

We use Labview to program our robot, but I know C++ enough to work with some OpenCV libraries. I have a couple of questions that I couldn't find the answers to in my search:
  1. How can I send image data from the Raspberry Pi to the RoboRIO (which is running Labview)?
  2. Is the Raspberry Pi even a good choice?
  3. Is OpenCV a better choice than the Labview vision vi?

Thanks,
Joe Kelly
You can use NetworkTables to send image data from the Pi to the RoboRIO. You would have to compile the ntcore library for Raspberry Pi, which isn't too hard to do, and then just link to that. You can also build a custom communication interface, which shouldn't be too hard, but NetworkTables is the easiest.

As long as you are using a Pi 2, you should be good. In fall 2013 I did alot of camera testing, and got about 9 fps out of a Pi 1, 16 out of a BeagleBone black, and a Pi 2 should be much quicker then the BBB, so it should get in the 25 or so range.

I've always found that OpenCV is much easier to work with then NIVision. In addition, since its not running on the RoboRIO you can have the RoboRIO only doing robot code. Having the processor on the robot is better then on the driver station IMO because I've seen FTAs request that teams turn off their dashboards to ease up field traffic, and if your processor is on board even if they ask you to do that you don't lose tracking.
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
  #3   Spotlight this post!  
Unread 11-12-2015, 21:56
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Vision Processing - Questions

I'll suggest that you to run the existing vision examples and try the vision processing on the roboRIO. The roboRIO is many times faster than the PI2 and doesn't require you to coordinate several processors, different implementations of network tables, etc. Once you have some experience with that, you can run a similar experiment on the PI2.

Also, keep in mind that fps is not necessarily a good measure of how well a vision system is working. Most of the time, it is a good idea to use the camera as a slow sensor and close the loop on trajectory or orientation using a different sensor.

Greg McKaskle
  #4   Spotlight this post!  
Unread 12-12-2015, 03:31
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Re: Vision Processing - Questions

Quote:
Originally Posted by Greg McKaskle View Post

Also, keep in mind that fps is not necessarily a good measure of how well a vision system is working. Most of the time, it is a good idea to use the camera as a slow sensor and close the loop on trajectory or orientation using a different sensor.
True, as long as you can process faster then the camera sends images. We couldn't get the axis camera to send less then 15 FPS, so it would actually start lagging and getting further and further behind as more images came in. And we couldn't figure out a way to make opencv flush the buffer up to the last image.

Also how is the RoboRIO (dual core ~700 Mhz Armv7) much faster then a Pi 2 (quad core 1000 Mhz Armv7)?
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
  #5   Spotlight this post!  
Unread 12-12-2015, 07:32
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Vision Processing - Questions

The camera drivers or IP cameras may indeed send more images than you asked for. The LabVIEW WPILib implementation consumes all of those images and hands the use code the latest when they ask for it. If you want to process 10 per second, it takes in 15 and hands out ten. This avoids introducing lag. I'm not sure if WPILib does this for other languages.

As for the processor comparison. I used the wikipedia page for "instructions per second" and looked up Cortex A9. I divided by two to adjust for the processor speed. Then I looked up PI 2 and adjusted for the number of cores I thought it had. I'll be honest, I don't have a PI and thought it had two cores and didn't verify my assumption. Even with that math, the ratio is less than 2 to 1, so I shouldn't have said "many times faster".

I still think that the OP will be well served by looking at the examples and trying things on a single computer, single language, and doing the control using additional sensors. If they then feel the need to elaborate the system with two of everything, they are better prepared for the journey.

Greg McKaskle
  #6   Spotlight this post!  
Unread 12-12-2015, 10:35
GeeTwo's Avatar
GeeTwo GeeTwo is offline
Technical Director
AKA: Gus Michel II
FRC #3946 (Tiger Robotics)
Team Role: Mentor
 
Join Date: Jan 2014
Rookie Year: 2013
Location: Slidell, LA
Posts: 3,723
GeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond repute
Re: Vision Processing - Questions

I recommend never sending the full images to the 'RIO at all. Use the pi (open CV is a good way to go) to process the images down to the key pieces of info that you need and send those to the roboRIO. If you want a visual for the driver, you can get CV to generate "schematics" of the full images and send these reduced images to the driver station using much less bandwidth (and/or higher frame rate) than if you send full images. The Canny edge detector is a great way to reduce an image to its key elements.
__________________

If you can't find time to do it right, how are you going to find time to do it over?
If you don't pass it on, it never happened.
Robots are great, but inspiration is the reason we're here.
Friends don't let friends use master links.
  #7   Spotlight this post!  
Unread 12-12-2015, 11:50
jojoguy10's Avatar
jojoguy10 jojoguy10 is offline
Programming/Controls Mentor
AKA: Joe Kelly
FRC #0997 (Spartan Robotics) and FRC #2990 (Hotwire Robotics)
Team Role: Mentor
 
Join Date: Jan 2013
Rookie Year: 2010
Location: Corvallis, OR
Posts: 298
jojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant future
Re: Vision Processing - Questions

Quote:
Originally Posted by Thad House View Post
You can use NetworkTables to send image data from the Pi to the RoboRIO. You would have to compile the ntcore library for Raspberry Pi, which isn't too hard to do, and then just link to that. You can also build a custom communication interface, which shouldn't be too hard, but NetworkTables is the easiest.
Is there a quick "getting started" guide for setting up network tables in C++ (for OpenCV)? I know where the blocks are in Labview, but I've never done them in C++


Quote:
Originally Posted by Thad House View Post
As long as you are using a Pi 2, you should be good. In fall 2013 I did alot of camera testing, and got about 9 fps out of a Pi 1, 16 out of a BeagleBone black, and a Pi 2 should be much quicker then the BBB, so it should get in the 25 or so range.
Awesome! I'll look into the Pi2 more specifically.

Quote:
Originally Posted by Thad House View Post
I've always found that OpenCV is much easier to work with then NIVision. In addition, since its not running on the RoboRIO you can have the RoboRIO only doing robot code. Having the processor on the robot is better then on the driver station IMO because I've seen FTAs request that teams turn off their dashboards to ease up field traffic, and if your processor is on board even if they ask you to do that you don't lose tracking.
Right, I have heard running vision processing code on the RoboRIO will slow it down. That's why I was wondering about how to transfer the image data from the board to the RoboRIO.

Quote:
I'll suggest that you to run the existing vision examples and try the vision processing on the roboRIO. The roboRIO is many times faster than the PI2 and doesn't require you to coordinate several processors, different implementations of network tables, etc. Once you have some experience with that, you can run a similar experiment on the PI2.
We have run a few examples already. We thought the results were slow and laggy. So, we wanted to try to find a faster and more reliable solution.

Thanks everyone!
__________________

2013 and 2014 - Student (FRC#2990), Programmer (LabVIEW)
2015 and 2016 - Mentor (FRC#2990), Programming (LabVIEW)
2017-Current - Mentor (FRC#997), Controls/Programming (Java)
2015-Current - A/V Volunteer at Oregon events
  #8   Spotlight this post!  
Unread 12-12-2015, 12:01
marshall's Avatar
marshall marshall is offline
My pants are louder than yours.
FRC #0900 (The Zebracorns)
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2003
Location: North Carolina
Posts: 1,337
marshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond repute
Re: Vision Processing - Questions

Quote:
Originally Posted by Greg McKaskle View Post
I still think that the OP will be well served by looking at the examples and trying things on a single computer, single language, and doing the control using additional sensors. If they then feel the need to elaborate the system with two of everything, they are better prepared for the journey.

Greg McKaskle
For what it's worth, Greg's suggestion is a very good one. Try the examples before running off on the secondary system quest...

However, if you are undeterred and wish to forge ahead and you want to send data between two systems then I would look at Network Tables (which has been heavily revised for this year) and I would possibly look at ROS. I2C and serial are also possibilities. You could also just send raw streams but there are bandwidth limitations and port restrictions to keep in mind (Read the rules).

Our team (900) has been doing a lot with vision processing. Last year we included an Nvidia Jetson TK1 on our robot that used a webcam and OpenCV to process the data. We used NetworkTables to send the data between the Jetson and the RoboRIO. We also proudly program our RoboRIO in LabVIEW.

EDIT: One more point to make. You don't always NEED to process video. Sometimes a single image or a set of images will work just fine. Single frames are minuscule in comparison to video streams and a lot faster to process. For instance, this auto aim was done using single frame captures: https://www.youtube.com/watch?v=QT2OmzrAhPI
__________________
"La mejor salsa del mundo es la hambre" - Miguel de Cervantes
"The future is unwritten" - Joe Strummer
"Simplify, then add lightness" - Colin Chapman

Last edited by marshall : 12-12-2015 at 12:24.
  #9   Spotlight this post!  
Unread 12-12-2015, 14:13
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Vision Processing - Questions

Do you remember which examples? And what image size and results?

Greg McKaskle
  #10   Spotlight this post!  
Unread 12-12-2015, 17:04
jojoguy10's Avatar
jojoguy10 jojoguy10 is offline
Programming/Controls Mentor
AKA: Joe Kelly
FRC #0997 (Spartan Robotics) and FRC #2990 (Hotwire Robotics)
Team Role: Mentor
 
Join Date: Jan 2013
Rookie Year: 2010
Location: Corvallis, OR
Posts: 298
jojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant futurejojoguy10 has a brilliant future
Re: Vision Processing - Questions

Quote:
Originally Posted by Greg McKaskle View Post
Do you remember which examples? And what image size and results?
We just played around a bit with the NIViosion examples in Labview (Tutorial #8 in the "Tutorials" tab in Labview). I don't really remember the specifics, but I just remember we were having trouble calibrating for the reflections of the plexiglass backs at competitions. I know a second board might not be the way to fix that, but I would also like to just tinker and experiment with a second board anyway. :-)
__________________

2013 and 2014 - Student (FRC#2990), Programmer (LabVIEW)
2015 and 2016 - Mentor (FRC#2990), Programming (LabVIEW)
2017-Current - Mentor (FRC#997), Controls/Programming (Java)
2015-Current - A/V Volunteer at Oregon events
  #11   Spotlight this post!  
Unread 12-12-2015, 17:58
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Vision Processing - Questions

FRC is a great place to experiment. If you have issues, PI or roboRIO, please post info and ask questions. Lots of folks will learn from it. And good luck.

Greg McKaskle
  #12   Spotlight this post!  
Unread 12-12-2015, 21:08
AlexanderTheOK AlexanderTheOK is offline
Guy
no team
 
Join Date: Jan 2014
Rookie Year: 2012
Location: Los Angeles
Posts: 147
AlexanderTheOK is just really niceAlexanderTheOK is just really niceAlexanderTheOK is just really niceAlexanderTheOK is just really nice
Re: Vision Processing - Questions

we were getting 15 fps from 2 separate cameras at 640p on the odroid u3 in 2014 (were obviously not used much after the little FMS issue was found). I believe the company that makes them has moved on to make cheaper and more powerful alternatives now.

You really can get a lot more out of doing vision processing when running it on an independent board. Not only can you get better framerates, but you can test your code without using the roborio, and with a proper monitor.
  #13   Spotlight this post!  
Unread 15-12-2015, 06:46
weaversam8 weaversam8 is offline
Team Captain
AKA: Sam Weaver
FRC #4534 (Wired Wizards)
Team Role: Leadership
 
Join Date: Jan 2014
Rookie Year: 2014
Location: Wilmington, North Carolina
Posts: 137
weaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to behold
We used a Jetson TK1 instead of a Pi last year since the code running on the Pi was a bit too slow.

While other have been talking about network tables, I would simply recommend serial. We used both serial ports for different purposes last year, remember that there is one on the case of the RIO and one in the MXP port.
__________________
ORB Haow!

Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 03:17.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi