OpenCV for RoboRio - Release

[NotInControl]

OpenCV For the RoboRio - Release

A few hours before kickoff and were finally able to release this. We have been working very hard to provide a working OpenCV
solution for everyone that not only supports the Axis IP camera we all have, but also the Microsoft HD 3000 USB webcam.

Our github page has everything to help you get starting using OpenCV, including our 2014 vision code which we used throughout
the 2014 season for hot target detection. The example supports reading images from a USB camera, Axis IP camera, or from
a file, by simply changing command line arguments

We have successfully ran this code on The RoboRio, and on a beaglebone black, so you can pick your poison. We also have this running
on the Tegra TK1, and will release a how-to on that very soon.

The OpenCV version we are providing is version 2.4.10 which we then patched to support the various camera settings of the
MS HD 3000 camera. It will also support other USB cameras although we havent tested it. And of course it supports the Axis
IP cameras.

The OpenCV libraries have been cross compiled using GCC 4.6.3 and has VFP_V3 and NEON support.

Some of the options it was compiled with:
TBB, FFMPEG, GSTREAMER, JAVA Bindings, JPEG, VL42, GTK

Java bindings means you can use this in Java as well, although we never tested it, so please report back if you do use it in Java.

Our github page has everything to help you get set up using C++ (Even though we program our Robot in Java, we program our Vision in C++).

Please let me know if you have any questions, we are here to help.

Link to Release: https://github.com/Team2168/2168_Vision_Example

The Program will take an input image like input.jpg (provided by the camera) and produce an output like that of output.jpg.

Regards,
Kevin

Happy New Year, and Happy New Season

And as always, if this is what you were waiting for, please provide you thoughts or any feedback you may have

[Bryce Paputa]

Looks very cool, we'll definitely try the java bindings out. How's the performance on the RoboRIO?

[NotInControl]

not too bad actually... we started doing more controlled vision performance tests a few weeks back, but then that got pushed behind some other priorities.

Our original tests were documented here: http://khengineering.github.io/RoboR...on/cameratest/

and still hold some valid data, however, the new openCV libraries we just produced are a bit more streamlined and should run a little bit faster.

It is too early to give a good recommendation on the Rio's performance, but we have been able to get 320x240 at 15fps without any lag using the axis camera under 2014 game scenarios, and that was more than what we needed for last years game.

Everything we have done so far shows that the IP camera outperforms the USB HD 3000 Camera on the Rio, but I don't want to official state that as of yet until we do more controlled observations. We should have a new set of comparative data by the end of week 1 for both IP and USB cameras.

The main page where that information will be held is at http://khengineering.github.io/RoboRio/faq/vision/

[Jared]

Awesome work. I've been planning to attempt to compile OpenCV on the roboRIO itself once we finally get one, but this seems like a better way.

[Jefferson]

Wow. Looks really well done and documented. We'll definitely be using this as a resource this season. I'm assuming there will be some vision challenge this year given the Microsoft camera beta testing.

Good work.

[virtuald]

Good to know it works! We've got numpy packaged for the RoboRIO in opkg format, so OpenCV will probably next on the list of things to package.

[mikhail]

Thanks Kevin! This is really valuable resource you have created.

While our whole team is brainstorming and prototyping I've been "secretly" toying around with the control system. What I've achieved so far is: I cloned the 2168_Vision_Example project from GitHub, successfully built it in Eclipse with the arm-frc-linux-gnueabi toolchain. Then transferred the libraries and the binary onto roboRIO and successfully executed it in there. All the how-to steps are clearly documented in the project's README.md file. Thank you very much, Kevin!

Next will be to modify the 2168's code until it breaks and then start prototyping something my own

[Wazzaps]

Any way to do some processing with opencv and then display the results in the dashboard? (Processed image stream)

PS. We are using java.

[Ben Wolsieffer]

I just tried this out today using the Java libraries and I got it to work.

After installing OpenCV to the roboRIO via the instructions on GitHub, I had to do some more work to make it work in Java.

First I had to add:

<var name="classpath" value="${classpath}:/path/to/opencv-2410.jar" />

to the end of the build.xml in the robot code Eclipse project (add it just before the closing </project> tag). This includes the OpenCV classes in the jar file that is uploaded to the roboRIO.

To load the native library in the robot code I had to add this line:

System.load("/usr/local/lib/lib_OpenCV/java/opencv_java2410.so");

Put this somewhere where it will get executed before any OpenCV code (ie. static initializer). The path will obviously be different if you installed OpenCV somewhere else on the roboRIO.

I was able to capture and stream an image from the Microsoft Lifecam HD3000, but it is at a very low resolution and I can't seem to change it. The same code works fine in Linux on my laptop. I'll see if I can figure more out tomorrow.

[NotInControl]

Originally Posted by lopsided98

I just tried this out today using the Java libraries and I got it to work.

After installing OpenCV to the roboRIO via the instructions on GitHub, I had to do some more work to make it work in Java.

First I had to add:

<var name="classpath" value="${classpath}:/path/to/opencv-2410.jar" />

to the end of the build.xml in the robot code Eclipse project (add it just before the closing </project> tag). This includes the OpenCV classes in the jar file that is uploaded to the roboRIO.

To load the native library in the robot code I had to add this line:

System.load("/usr/local/lib/lib_OpenCV/java/opencv_java2410.so");

Put this somewhere where it will get executed before any OpenCV code (ie. static initializer). The path will obviously be different if you installed OpenCV somewhere else on the roboRIO.

I was able to capture and stream an image from the Microsoft Lifecam HD3000, but it is at a very low resolution and I can't seem to change it. The same code works fine in Linux on my laptop. I'll see if I can figure more out tomorrow.

This is awesome news. I haven't had the time to test out Java myself (it is 1 item on a long to-do list of mine, but glad to hear it is working for you.

If you are having trouble changing the camera settings, try using the patch we added to OpenCV, we modified some of the v4l calls and exposed a new constructor to the videoCap class.

You can work off of the C++ example here: which will allow you to set FPS, image size, and some quality settings.

https://github.com/Team2168/2168_Vis....cpp#L633-L664

Let me know if this works for you or not.

[mikhail]

Originally Posted by Wazzaps

Any way to do some processing with opencv and then display the results in the dashboard? (Processed image stream)

PS. We are using java.

I wonder the same but in C++. What would be the best way to do something similar to imshow() but on the driver station's dashboard?

[faust1706]

tl;dr of this thread (copied from http://www.reddit.com/r/FRC/comments...or_roborio_on/) Credit goes to u/ironmig

1. Install dependencies
   sudo apt-get install build-essential
   sudo apt-get install cmake
   sudo apt-get install gcc-arm-linux-gnueabi
   sudo apt-get install g++-arm-linux-gnueabi
2. Enter directory that will hold opencv
   cd ~/Documents
3. Clone OpenCV from git
   git clone https://github.com/Itseez/opencv.git
4. Checkout to a recent OpenCV release (change 2.4.10.1 to a recent release)
   cd opencv
   git checkout tags/2.4.10.1
5. Set toolchain to correct version of g++ (change line in file or run this complicated one-liner
   cd platforms/linux
   x=$(dpkg -s g++-arm-linux-gnueabi | grep "Version" | grep -o "..."); sed -i 's/set(GCC_COMPILER_VERSION "."/set(GCC_COMPILER_VERSION \"'$x'\" CACHE STRING "GCC Compiler version"/g' arm-gnueabi.toolchain.cmake
6. Prepare file for build
   mkdir build_roborio
   cd build_roborio
   cmake -DSOFTFP=ON -DCMAKE_TOOLCHAIN_FILE=../arm-gnueabi.toolchain.cmake ../../..
7. Build project (this will take a while)
   make

Disclaimer* I haven't actually tried this.

[sebas]

I have also confirmed that the Java version works if you make the fixes to the build.xml. Great work, Team 2168!

[Arhowk]

My apologies for not being used to linux development nor C++ (Windows + Java almost exclusively for the past 10 years) but the first time I tried to build, it gave me an error that it couldn't find libc++.so.6 (and another one that I don't remember). Now, when I try to build, I get this message

"Info: Nothing to build for 2168_Vision_Example"

I thought it might be because the builder uses a diff scanner and it's already built, but I tried to modify the source code and it still refused to build.

E/ deleting the Rio-Beagle folder re-gave me the first error

c:/frc/bin/../lib/gcc/arm-frc-linux-gnueabi/4.9.1/../../../../arm-frc-linux-gnueabi/bin/ld.exe: warning: libstdc++.so.6, needed by D:\Miscellaneous Documents\CDT Code\2168_Vision_Example\_Libraries\softFP\lib_OpenCV/libopencv_highgui.so, not found (try using -rpath or -rpath-link)
c:/frc/bin/../lib/gcc/arm-frc-linux-gnueabi/4.9.1/../../../../arm-frc-linux-gnueabi/bin/ld.exe: warning: libffi.so.6, needed by D:/Miscellaneous Documents/CDT Code/2168_Vision_Example/_Libraries/softFP/lib_OpenCV/libgio-2.0.so.0, not found (try using -rpath or -rpath-link)

should I just ignore it?

[frechetn]

Has there been any developments with the Tegra TK1 as suggested in the original post?