OpenCV for RoboRio - Release

[NotInControl]

Originally Posted by JagCode

So, this happened. Not sure what went wrong. :I

We followed the instructions, and the Rio_Beagle directory was successfully created. However, the main.o that was inside was corrupted to kingdom-come.

Nothing went wrong actually. It looks like you successfully compiled the code. The two warnings you get are known warnings which everyone on windows will receive and do not affect code output.

Main.o is an object file and is not human readable... it is not corrupt.

The file named 2168_Vision in that folder is the binary that you can transfer to the arm7 embedded device and run it.

I dont see any obvious problems in your posted images.

If you have any probs, please repprt back.

Regards,
Kevin

[NotInControl]

Originally Posted by kylelanman

Team 2481 is trying to get this working with our C++ project. Our project settings are what was generated by the FRC C++ plugin.

I'll admit that we don't have a lot of expertise when it comes to modifying the build/compiler/linker configuration.

We followed the instructions in the readme. The sample project compiles but when when we try to add the libraries to our project we aren't able to compile. I'm including the build console and our .cproject file.

Any assistance would be greatly appreciated.

Thanks,

Team 2481

The WPILib project is not set up properly. The linker is not directed to the OpenCv libraries supplied by the Sample project _Libraries folder. You will need to manually add that yourself if you are starting from a WPI template Robot project.

Take a look at the build settings in the Sample Project we supplied, and modify the WPI build settings to include the OpenCV specific changes, such as adding the OpenCV _Libraries path to your WPILib project, and adding the -rpath command to the linker misc settings.

The build settings in the Sample Project should be all that you need to get this up and running in your own project.

[kylelanman]

Originally Posted by NotInControl

The WPILib project is not set up properly. The linker is not directed to the OpenCv libraries supplied by the Sample project _Libraries folder. You will need to manually add that yourself if you are starting from a WPI template Robot project.

Take a look at the build settings in the Sample Project we supplied, and modify the WPI build settings to include the OpenCV specific changes, such as adding the OpenCV _Libraries path to your WPILib project, and adding the -rpath command to the linker misc settings.

The build settings in the Sample Project should be all that you need to get this up and running in your own project.

We double checked all the settings again and did not find any problems. We did however find the problem and I'm nearly certain other people will run into this.

In the WPILib.h file on line 8 you will find the following.

#define REAL

In the core.hpp file on line 4132 you will find the following enum definition.

enum
    {
        NONE=0, //!< empty node
        INT=1, //!< an integer
        REAL=2, //!< floating-point number
        FLOAT=REAL, //!< synonym or REAL
        STR=3, //!< text string in UTF-8 encoding
        STRING=STR, //!< synonym for STR
        REF=4, //!< integer of size size_t. Typically used for storing complex dynamic structures where some elements reference the others
        SEQ=5, //!< sequence
        MAP=6, //!< mapping
        TYPE_MASK=7,
        FLOW=8, //!< compact representation of a sequence or mapping. Used only by YAML writer
        USER=16, //!< a registered object (e.g. a matrix)
        EMPTY=32, //!< empty structure (sequence or mapping)
        NAMED=64 //!< the node has a name (i.e. it is element of a mapping)
    };

The two definitions of REAL conflict. We removed the #define in WPILib.h and were able to compile.

The best I can tell the #define REAL in WPILib is only used in examples provided for the simulator to distinguish between simulation and real mode so it shouldn't cause a problem to remove it.

Thanks,

Team 2481

[Corvo]

Originally Posted by NotInControl

Are you trying to run this on your desktop or on a Arm based processor?

The OpenCV libraries we provide are for Arm based processors only so they will only run on a RoboRio, Beaglebone etc or anything running ArmV7 soft eabi. They will not run on an Intel based processor.

Please provide more details of where you are trying to run the program (causing the error), and what you have done thus far getting up to that point so that we can try to help figure it out.

Thanks for trying this out.

Currently I'm running this on my laptop (desktop) but have plans to migrate over to an Odroid (Arm processor) for competition.

As for what I've done thus far, I've already stated it, I've installed it like the official version and run some test code on it that works fine with the main version. I believe the version that's up for download just hasn't been properly packaged.

[mikhail]

Originally Posted by Wazzaps

Any way to do some processing with opencv and then display the results in the dashboard? (Processed image stream)

PS. We are using java.

I wonder the same but in C++. What would be the best way to do something similar to imshow() but on the driver station's dashboard?

[faust1706]

tl;dr of this thread (copied from http://www.reddit.com/r/FRC/comments...or_roborio_on/) Credit goes to u/ironmig

1. Install dependencies
   sudo apt-get install build-essential
   sudo apt-get install cmake
   sudo apt-get install gcc-arm-linux-gnueabi
   sudo apt-get install g++-arm-linux-gnueabi
2. Enter directory that will hold opencv
   cd ~/Documents
3. Clone OpenCV from git
   git clone https://github.com/Itseez/opencv.git
4. Checkout to a recent OpenCV release (change 2.4.10.1 to a recent release)
   cd opencv
   git checkout tags/2.4.10.1
5. Set toolchain to correct version of g++ (change line in file or run this complicated one-liner
   cd platforms/linux
   x=$(dpkg -s g++-arm-linux-gnueabi | grep "Version" | grep -o "..."); sed -i 's/set(GCC_COMPILER_VERSION "."/set(GCC_COMPILER_VERSION \"'$x'\" CACHE STRING "GCC Compiler version"/g' arm-gnueabi.toolchain.cmake
6. Prepare file for build
   mkdir build_roborio
   cd build_roborio
   cmake -DSOFTFP=ON -DCMAKE_TOOLCHAIN_FILE=../arm-gnueabi.toolchain.cmake ../../..
7. Build project (this will take a while)
   make

Disclaimer* I haven't actually tried this.

[sebas]

I have also confirmed that the Java version works if you make the fixes to the build.xml. Great work, Team 2168!

[Arhowk]

My apologies for not being used to linux development nor C++ (Windows + Java almost exclusively for the past 10 years) but the first time I tried to build, it gave me an error that it couldn't find libc++.so.6 (and another one that I don't remember). Now, when I try to build, I get this message

"Info: Nothing to build for 2168_Vision_Example"

I thought it might be because the builder uses a diff scanner and it's already built, but I tried to modify the source code and it still refused to build.

E/ deleting the Rio-Beagle folder re-gave me the first error

c:/frc/bin/../lib/gcc/arm-frc-linux-gnueabi/4.9.1/../../../../arm-frc-linux-gnueabi/bin/ld.exe: warning: libstdc++.so.6, needed by D:\Miscellaneous Documents\CDT Code\2168_Vision_Example\_Libraries\softFP\lib_OpenCV/libopencv_highgui.so, not found (try using -rpath or -rpath-link)
c:/frc/bin/../lib/gcc/arm-frc-linux-gnueabi/4.9.1/../../../../arm-frc-linux-gnueabi/bin/ld.exe: warning: libffi.so.6, needed by D:/Miscellaneous Documents/CDT Code/2168_Vision_Example/_Libraries/softFP/lib_OpenCV/libgio-2.0.so.0, not found (try using -rpath or -rpath-link)

should I just ignore it?

[frechetn]

Has there been any developments with the Tegra TK1 as suggested in the original post?

[JohnM]

One thing I do not understand is the whole downloading process. Okays I have the library's set up on the robRIO (I think). Anyways here are my questions.

1. Will all of openCV's methods for C++ fork on this?
2. To download code to the robot you have to do the FTP tranfer of the binary? You can't just deploy the code to the robot

I ask the second question is because the provided example I eclipse does not provide an option to deploy to the roboRIO, only an option to run as a local application or something like that.

[NotInControl]

Originally Posted by sebas

I have also confirmed that the Java version works if you make the fixes to the build.xml. Great work, Team 2168!

Awesome!. I will add that to the Github readme so others can follow it as well.

Originally Posted by Arhowk

My apologies for not being used to linux development nor C++ (Windows + Java almost exclusively for the past 10 years) but the first time I tried to build, it gave me an error that it couldn't find libc++.so.6 (and another one that I don't remember). Now, when I try to build, I get this message

"Info: Nothing to build for 2168_Vision_Example"

I thought it might be because the builder uses a diff scanner and it's already built, but I tried to modify the source code and it still refused to build.

E/ deleting the Rio-Beagle folder re-gave me the first error

c:/frc/bin/../lib/gcc/arm-frc-linux-gnueabi/4.9.1/../../../../arm-frc-linux-gnueabi/bin/ld.exe: warning: libstdc++.so.6, needed by D:\Miscellaneous Documents\CDT Code\2168_Vision_Example\_Libraries\softFP\lib_OpenCV/libopencv_highgui.so, not found (try using -rpath or -rpath-link)
c:/frc/bin/../lib/gcc/arm-frc-linux-gnueabi/4.9.1/../../../../arm-frc-linux-gnueabi/bin/ld.exe: warning: libffi.so.6, needed by D:/Miscellaneous Documents/CDT Code/2168_Vision_Example/_Libraries/softFP/lib_OpenCV/libgio-2.0.so.0, not found (try using -rpath or -rpath-link)

should I just ignore it?

Did you install the WPILib C++ tools? Can you build a WPI C++ project? The message nothing to build means that nothing in the source code changed from the last time you built the program and created the 2168_Vision binary.

Edit the source code, and it will build again.

Originally Posted by frechetn

Has there been any developments with the Tegra TK1 as suggested in the original post?

Yes there has, I need to upload the code. These libraries won't won't on the Tegra because the Tegra used hardFP eabi. I will upload them as soon as I can.

Originally Posted by JohnM

One thing I do not understand is the whole downloading process. Okays I have the library's set up on the robRIO (I think). Anyways here are my questions.

1. Will all of openCV's methods for C++ fork on this?
2. To download code to the robot you have to do the FTP tranfer of the binary? You can't just deploy the code to the robot

I ask the second question is because the provided example I eclipse does not provide an option to deploy to the roboRIO, only an option to run as a local application or something like that.

This is a full OpenCV package so all methods are included, we also added a few modules so you get more than just the basic OpenCV. We also patched OpenCV so it supports the Microsoft HD 3000 camera.

To download the code all you need to do is Transfer the binary using FTP, winSCP, or any other method you like. Follow the instructions on the readme to get this running as a standalone project.

This project is not apart of WPILib, it is standalone, which is why there is no deploy. It can be deployed to any Arm7 device, including a beagle bone without any changes.

[brk]

Originally Posted by mikhail

I wonder the same but in C++. What would be the best way to do something similar to imshow() but on the driver station's dashboard?

So I'm trying to do the same in C++.
I'd like to capture/process the stream on the roborio, but then send the processed image back to the driver station.

As far as I can tell, you need to get a JPEG image and send that back. The capture gives me a cv::Mat object back, I manipulate it, but then how to I send that cv::Mat data back to the driver station? Is it already in JPEG format? or do I need to convert it. And where do I access a JPEG stream within the cv::Mat ? I know, I know, this is a CV question, but this is the first time working with CV.


edit - found my answer...
Just need to convert the Mat to an array using cv::imencode() and send that back to the DS.

[DaPasta]

Greetings!

I attempted to get OpenCV working with Java. However, once I pushed the Java code to the roboRio, I get the following error on the driver station:

ERROR Unhandled exception: java.lang.UnsatisfiedLinkError: no opencv_java2410 in java.library.path at [java.lang.ClassLoader.loadLibrary(ClassLoader.java:1857), java.lang.Runtime.loadLibrary0(Runtime.java:870), java.lang.System.loadLibrary(System.java:1119), org.team1515.pastabot.Robot.robotInit(Robot.java:41), edu.wpi.first.wpilibj.IterativeRobot.startCompetition(IterativeRobot.java:76), edu.wpi.first.wpilibj.RobotBase.main(RobotBase.java:234)]

I also added the following line in the roboInit method per instructions based on a previous post:

System.load("/usr/local/lib/lib_OpenCV/java/opencv_java2410.so");

I double checked multiple times that this path was correct.

[Jared]

Originally Posted by DaPasta

Greetings!

I attempted to get OpenCV working with Java. However, once I pushed the Java code to the roboRio, I get the following error on the driver station:

ERROR Unhandled exception: java.lang.UnsatisfiedLinkError: no opencv_java2410 in java.library.path at [java.lang.ClassLoader.loadLibrary(ClassLoader.java:1857), java.lang.Runtime.loadLibrary0(Runtime.java:870), java.lang.System.loadLibrary(System.java:1119), org.team1515.pastabot.Robot.robotInit(Robot.java:41), edu.wpi.first.wpilibj.IterativeRobot.startCompetition(IterativeRobot.java:76), edu.wpi.first.wpilibj.RobotBase.main(RobotBase.java:234)]

I also added the following line in the roboInit method per instructions based on a previous post:

System.load("/usr/local/lib/lib_OpenCV/java/opencv_java2410.so");

I double checked multiple times that this path was correct.

This may be due to the robotInit() method being called a short period of time after the robot code actually starts.

You can put the System.load(PATH); line inside of a static initialization block in the main robot class.

It should look something like this, but with CV_PATH replaced by the path you installed to. It can be inserted above your robotInit() method.

static { 
     System.load(CV_PATH);
}

[mikhail]

Originally Posted by brk

edit - found my answer...
Just need to convert the Mat to an array using cv::imencode() and send that back to the DS.

A small code snippet would be very appreciated. What parameters do you use for cv::imencode? Do you use CameraServer class to send the image to the DS? Thank you.