![]() |
OpenCV for RoboRio - Release
2 Attachment(s)
OpenCV For the RoboRio - Release
A few hours before kickoff and were finally able to release this. We have been working very hard to provide a working OpenCV solution for everyone that not only supports the Axis IP camera we all have, but also the Microsoft HD 3000 USB webcam. Our github page has everything to help you get starting using OpenCV, including our 2014 vision code which we used throughout the 2014 season for hot target detection. The example supports reading images from a USB camera, Axis IP camera, or from a file, by simply changing command line arguments We have successfully ran this code on The RoboRio, and on a beaglebone black, so you can pick your poison. We also have this running on the Tegra TK1, and will release a how-to on that very soon. The OpenCV version we are providing is version 2.4.10 which we then patched to support the various camera settings of the MS HD 3000 camera. It will also support other USB cameras although we havent tested it. And of course it supports the Axis IP cameras. The OpenCV libraries have been cross compiled using GCC 4.6.3 and has VFP_V3 and NEON support. Some of the options it was compiled with: TBB, FFMPEG, GSTREAMER, JAVA Bindings, JPEG, VL42, GTK Java bindings means you can use this in Java as well, although we never tested it, so please report back if you do use it in Java. Our github page has everything to help you get set up using C++ (Even though we program our Robot in Java, we program our Vision in C++). Please let me know if you have any questions, we are here to help. Link to Release: https://github.com/Team2168/2168_Vision_Example The Program will take an input image like input.jpg (provided by the camera) and produce an output like that of output.jpg. Regards, Kevin Happy New Year, and Happy New Season And as always, if this is what you were waiting for, please provide you thoughts or any feedback you may have |
Re: OpenCV for RoboRio - Release
Looks very cool, we'll definitely try the java bindings out. How's the performance on the RoboRIO?
|
Re: OpenCV for RoboRio - Release
not too bad actually... we started doing more controlled vision performance tests a few weeks back, but then that got pushed behind some other priorities.
Our original tests were documented here: http://khengineering.github.io/RoboR...on/cameratest/ and still hold some valid data, however, the new openCV libraries we just produced are a bit more streamlined and should run a little bit faster. It is too early to give a good recommendation on the Rio's performance, but we have been able to get 320x240 at 15fps without any lag using the axis camera under 2014 game scenarios, and that was more than what we needed for last years game. Everything we have done so far shows that the IP camera outperforms the USB HD 3000 Camera on the Rio, but I don't want to official state that as of yet until we do more controlled observations. We should have a new set of comparative data by the end of week 1 for both IP and USB cameras. The main page where that information will be held is at http://khengineering.github.io/RoboRio/faq/vision/ |
Re: OpenCV for RoboRio - Release
Awesome work. I've been planning to attempt to compile OpenCV on the roboRIO itself once we finally get one, but this seems like a better way.
|
Re: OpenCV for RoboRio - Release
Wow. Looks really well done and documented. We'll definitely be using this as a resource this season. I'm assuming there will be some vision challenge this year given the Microsoft camera beta testing.
Good work. |
Re: OpenCV for RoboRio - Release
Good to know it works! We've got numpy packaged for the RoboRIO in opkg format, so OpenCV will probably next on the list of things to package.
|
Re: OpenCV for RoboRio - Release
Thanks Kevin! This is really valuable resource you have created.
While our whole team is brainstorming and prototyping I've been "secretly" toying around with the control system. What I've achieved so far is: I cloned the 2168_Vision_Example project from GitHub, successfully built it in Eclipse with the arm-frc-linux-gnueabi toolchain. Then transferred the libraries and the binary onto roboRIO and successfully executed it in there. All the how-to steps are clearly documented in the project's README.md file. Thank you very much, Kevin! Next will be to modify the 2168's code until it breaks and then start prototyping something my own :D |
Re: OpenCV for RoboRio - Release
Any way to do some processing with opencv and then display the results in the dashboard? (Processed image stream)
PS. We are using java. |
Re: OpenCV for RoboRio - Release
I just tried this out today using the Java libraries and I got it to work.
After installing OpenCV to the roboRIO via the instructions on GitHub, I had to do some more work to make it work in Java. First I had to add: Code:
<var name="classpath" value="${classpath}:/path/to/opencv-2410.jar" />To load the native library in the robot code I had to add this line: Code:
System.load("/usr/local/lib/lib_OpenCV/java/opencv_java2410.so");I was able to capture and stream an image from the Microsoft Lifecam HD3000, but it is at a very low resolution and I can't seem to change it. The same code works fine in Linux on my laptop. I'll see if I can figure more out tomorrow. |
Re: OpenCV for RoboRio - Release
I'm currently running two versions of openCV in eclipse, the official 2.4.10 version and this one, and I've installed the java version (for both) as descrbied here. For some reason I'm getting an error with your version that I'm not getting with the 2.4.10 version from opencv.
It appears that this librabry only runs in 32 bit java and not 64. Is there any way to fix this without having to reinstall 32 bit java? |
Re: OpenCV for RoboRio - Release
Quote:
If you are having trouble changing the camera settings, try using the patch we added to OpenCV, we modified some of the v4l calls and exposed a new constructor to the videoCap class. You can work off of the C++ example here: which will allow you to set FPS, image size, and some quality settings. https://github.com/Team2168/2168_Vis....cpp#L633-L664 Let me know if this works for you or not. |
Re: OpenCV for RoboRio - Release
Quote:
The OpenCV libraries we provide are for Arm based processors only so they will only run on a RoboRio, Beaglebone etc or anything running ArmV7 soft eabi. They will not run on an Intel based processor. Please provide more details of where you are trying to run the program (causing the error), and what you have done thus far getting up to that point so that we can try to help figure it out. Thanks for trying this out. |
Re: OpenCV for RoboRio - Release
So, this happened. Not sure what went wrong. :I
We followed the instructions, and the Rio_Beagle directory was successfully created. However, the main.o that was inside was corrupted to kingdom-come. |
Re: OpenCV for RoboRio - Release
Quote:
|
Re: OpenCV for RoboRio - Release
2 Attachment(s)
Team 2481 is trying to get this working with our C++ project. Our project settings are what was generated by the FRC C++ plugin.
I'll admit that we don't have a lot of expertise when it comes to modifying the build/compiler/linker configuration. We followed the instructions in the readme. The sample project compiles but when when we try to add the libraries to our project we aren't able to compile. I'm including the build console and our .cproject file. Any assistance would be greatly appreciated. Thanks, Team 2481 |
Re: OpenCV for RoboRio - Release
Quote:
Main.o is an object file and is not human readable... it is not corrupt. The file named 2168_Vision in that folder is the binary that you can transfer to the arm7 embedded device and run it. I dont see any obvious problems in your posted images. If you have any probs, please repprt back. Regards, Kevin |
Re: OpenCV for RoboRio - Release
Quote:
Take a look at the build settings in the Sample Project we supplied, and modify the WPI build settings to include the OpenCV specific changes, such as adding the OpenCV _Libraries path to your WPILib project, and adding the -rpath command to the linker misc settings. The build settings in the Sample Project should be all that you need to get this up and running in your own project. |
Re: OpenCV for RoboRio - Release
Quote:
In the WPILib.h file on line 8 you will find the following. Code:
#define REALCode:
enumThe best I can tell the #define REAL in WPILib is only used in examples provided for the simulator to distinguish between simulation and real mode so it shouldn't cause a problem to remove it. Thanks, Team 2481 |
Re: OpenCV for RoboRio - Release
Quote:
As for what I've done thus far, I've already stated it, I've installed it like the official version and run some test code on it that works fine with the main version. I believe the version that's up for download just hasn't been properly packaged. |
Re: OpenCV for RoboRio - Release
Quote:
|
Re: OpenCV for RoboRio - Release
tl;dr of this thread (copied from http://www.reddit.com/r/FRC/comments...or_roborio_on/) Credit goes to u/ironmig
|
Re: OpenCV for RoboRio - Release
I have also confirmed that the Java version works if you make the fixes to the build.xml. Great work, Team 2168!
|
Re: OpenCV for RoboRio - Release
My apologies for not being used to linux development nor C++ (Windows + Java almost exclusively for the past 10 years) but the first time I tried to build, it gave me an error that it couldn't find libc++.so.6 (and another one that I don't remember). Now, when I try to build, I get this message
"Info: Nothing to build for 2168_Vision_Example" I thought it might be because the builder uses a diff scanner and it's already built, but I tried to modify the source code and it still refused to build. E/ deleting the Rio-Beagle folder re-gave me the first error Code:
c:/frc/bin/../lib/gcc/arm-frc-linux-gnueabi/4.9.1/../../../../arm-frc-linux-gnueabi/bin/ld.exe: warning: libstdc++.so.6, needed by D:\Miscellaneous Documents\CDT Code\2168_Vision_Example\_Libraries\softFP\lib_OpenCV/libopencv_highgui.so, not found (try using -rpath or -rpath-link) |
Re: OpenCV for RoboRio - Release
Has there been any developments with the Tegra TK1 as suggested in the original post?
|
Re: OpenCV for RoboRio - Release
One thing I do not understand is the whole downloading process. Okays I have the library's set up on the robRIO (I think). Anyways here are my questions.
I ask the second question is because the provided example I eclipse does not provide an option to deploy to the roboRIO, only an option to run as a local application or something like that. |
Re: OpenCV for RoboRio - Release
Quote:
Quote:
Edit the source code, and it will build again. Quote:
Quote:
To download the code all you need to do is Transfer the binary using FTP, winSCP, or any other method you like. Follow the instructions on the readme to get this running as a standalone project. This project is not apart of WPILib, it is standalone, which is why there is no deploy. It can be deployed to any Arm7 device, including a beagle bone without any changes. |
Re: OpenCV for RoboRio - Release
Quote:
So I'm trying to do the same in C++. I'd like to capture/process the stream on the roborio, but then send the processed image back to the driver station. As far as I can tell, you need to get a JPEG image and send that back. The capture gives me a cv::Mat object back, I manipulate it, but then how to I send that cv::Mat data back to the driver station? Is it already in JPEG format? or do I need to convert it. And where do I access a JPEG stream within the cv::Mat ? I know, I know, this is a CV question, but this is the first time working with CV. edit - found my answer... Just need to convert the Mat to an array using cv::imencode() and send that back to the DS. |
Re: OpenCV for RoboRio - Release
Greetings!
I attempted to get OpenCV working with Java. However, once I pushed the Java code to the roboRio, I get the following error on the driver station: Code:
ERROR Unhandled exception: java.lang.UnsatisfiedLinkError: no opencv_java2410 in java.library.path at [java.lang.ClassLoader.loadLibrary(ClassLoader.java:1857), java.lang.Runtime.loadLibrary0(Runtime.java:870), java.lang.System.loadLibrary(System.java:1119), org.team1515.pastabot.Robot.robotInit(Robot.java:41), edu.wpi.first.wpilibj.IterativeRobot.startCompetition(IterativeRobot.java:76), edu.wpi.first.wpilibj.RobotBase.main(RobotBase.java:234)]Code:
System.load("/usr/local/lib/lib_OpenCV/java/opencv_java2410.so"); |
Re: OpenCV for RoboRio - Release
Quote:
You can put the System.load(PATH); line inside of a static initialization block in the main robot class. It should look something like this, but with CV_PATH replaced by the path you installed to. It can be inserted above your robotInit() method. Code:
static { |
Re: OpenCV for RoboRio - Release
Quote:
|
Re: OpenCV for RoboRio - Release
To encode the JPEG, i have:
unsigned int SPAMVision::_encodeJpeg(std::vector<uchar> & dst, cv::Mat src, int quality) { std::vector<int> param = std::vector<int>(2); param[0] = CV_IMWRITE_JPEG_QUALITY; param[1] = quality; cv::imencode(".jpg", src, dst, param); return dst.size(); } And I use this in something very similar to the CameraServer class. |
Re: OpenCV for RoboRio - Release
I have a new problem...
I'm trying to do this on a beaglebone black. I thought it should work out of the box, but... when I run an executable built in eclipse, I get: error while loading shared libraries: libstdc++.so.6.0.20: cannot open shared object file: No such file or directory ldd shows libstdc++.so.6 isn't found, as well as some others. In fact, these are the missing ones (from ldd) libffi.so.6 => not found libstdc++.so.6 => not found libstdc++.so.6.0.20 => not found libz.so.1 => not found This will be a problem in that the beaglebone only has libstdc++.so.6.0.17. Does that mean I need to recompile opencv for this to remove that dependency? |
Re: OpenCV for RoboRio - Release
hello!
Looks like an awesome resource. I'm having an issue though. When I try and compile it it gives errors on all the includes, even string and anything else from the c std library. Do you have any solutions for this? Thanks again! |
Re: OpenCV for RoboRio - Release
Quote:
Details: http://www.chiefdelphi.com/forums/sh...8&postcount=18 |
Re: OpenCV for RoboRio - Release
Hello there! We are up to the point where we can grab image data from the camera and process it using OpenCV. However, we can't figure out how to get that jpeg data on the smartdashboard via CameraServer.
Here is what we have: Code:
std::string videoStreamAddress = "http://169.254.149.22/mjpg/video.mjpg";Thanks, Team 2053 |
Re: OpenCV for RoboRio - Release
2 Attachment(s)
The CameraServer is designed to work with the the NI Vision libraries and I don't know how to make it work with OpenCV. I wrote an OpenCV mjpg server a while ago (not for the roboRIO), that you might be able to make work. It requires Boost, which might be difficult to get on the roboRIO. Maybe it will help you with something though.
|
| All times are GMT -5. The time now is 01:57. |
Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi