JavaCV Help

[thblackhelmetma]

Hello fine members of Chief Delphi!

Team 670 is trying to use OpenCV for vision tracking of the goals, and for various reasons, we exclusively code in Java. Therefore, we are using JavaCV in order to do our tracking.

Our algorithm works fine when we use images and run it directly on our PC. However, when we upload the code to the roboRIO, we get an UnsatisfiedLinkError (Stack Trace Below). This error pops up whenever we call any method from any class in the OpenCV package.

ERROR Unhandled exception: java.lang.UnsatisfiedLinkError: no jniopencv_core in java.library.path at
[java.lang.ClassLoader.loadLibrary(ClassLoader.java:1857),
java.lang.Runtime.loadLibrary0(Runtime.java:870),
java.lang.System.loadLibrary(System.java:1119),
org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:597),
org.bytedeco.javacpp.Loader.load(Loader.java:438),
org.bytedeco.javacpp.Loader.load(Loader.java:381),
org.bytedeco.javacpp.opencv_core.<clinit>(opencv_core.java:10),
java.lang.Class.forName0(Native Method),
java.lang.Class.forName(Class.java:340),
org.bytedeco.javacpp.Loader.load(Loader.java:413),
org.bytedeco.javacpp.Loader.load(Loader.java:381),
org.bytedeco.javacpp.opencv_imgcodecs.<clinit>(opencv_imgcodecs.java:13),
org.usfirst.frc.team670.robot.Robot.process(Robot.java:85),
org.usfirst.frc.team670.robot.Robot.operatorControl(Robot.java:68),
edu.wpi.first.wpilibj.SampleRobot.startCompetition(SampleRobot.java:159),
edu.wpi.first.wpilibj.RobotBase.main(RobotBase.java:242)]
WARNING: Robots don't quit!

I have verified our build.properties and build.xml files many times, and they appear to be correct, but I will include them here anyway.

build.properties

# Project specific information
package=org.usfirst.frc.team670.robot
robot.class=${package}.Robot
userLibs = ${user.home}/wpilib/user/lib/javacpp.jar:${user.home}/wpilib/user/lib/opencv.jar:${user.home}/wpilib/user/lib/javacv.jar

build.xml

<project name="FRC Deployment" default="deploy">
    <property file="${user.home}/wpilib/wpilib.properties"/>
    <property file="build.properties"/>
    <property file="${user.home}/wpilib/java/${version}/ant/build.properties"/>
    <property name="classpath" value="${opencv.jar}:${javacpp.jar}:${javacv.jar}" />
    <import file="${wpilib.ant.dir}/build.xml"/>
</project>

So far, I have tried adding various native libraries to all of the different jars, in order to try and appease the UnsatisfiedLinkError, adding various dll files, and many other solutions recommended by the Internet. Nothing has worked thus far.

Could anyone help with this problem?

[ThomasClark]

JavaCV doesn't support the roboRIO out-of-the-box. You could grab the build that GRIP uses and save yourself the trouble of compiling it yourself.

[Ozuru]

Running vision processing code on your roboRIO itself is generally a bad idea. You should simply intercept the images being sent and perform the analyses on your driver's station.

[thblackhelmetma]

Originally Posted by Ozuru

Running vision processing code on your roboRIO itself is generally a bad idea. You should simply intercept the images being sent and perform the analyses on your driver's station.

Thanks for your input. Is there any particular way to intercept the images when using a USB camera?

[Ozuru]

Originally Posted by thblackhelmetma

Thanks for your input. Is there any particular way to intercept the images when using a USB camera?

Yes. Since the SmartDashboard is in Java, it is easily decompileable with tools like JD-GUI. I looked at how the USB camera widget worked and made a public static function in it that allowed me to get the latest frame. In another widget, I had it call that function to get the image and then process it with OpenCV. If you have questions throughout the process, don't be afraid to PM me. It's taken about two seasons but I think I've finally got a grasp on how SmartDashboard's extension system works.

To start off, your SmartDashboard will probably be in C:/Users/<username>/wpilib/tools/SmartDashboard.jar. Open it up in JD-GUI and start poking around for something that resembles the USB camera widget. You'll know it when you find it:

public static final NAME = "USB Webcam Viewer";

[ThomasClark]

Originally Posted by Ozuru

Yes. Since the SmartDashboard is in Java, it is easily decompileable with tools like JD-GUI. I looked at how the USB camera widget worked and made a public static function in it that allowed me to get the latest frame. In another widget, I had it call that function to get the image and then process it with OpenCV. If you have questions throughout the process, don't be afraid to PM me. It's taken about two seasons but I think I've finally got a grasp on how SmartDashboard's extension system works.

To start off, your SmartDashboard will probably be in C:/Users/<username>/wpilib/tools/SmartDashboard.jar. Open it up in JD-GUI and start poking around for something that resembles the USB camera widget. You'll know it when you find it:

public static final NAME = "USB Webcam Viewer";

Or just make a TeamForge account and look at the source code? If you decompile the binary you'll lose all of the comments.

Here's another (simplified) reference for the protocol if you don't wanna make an account.

[Ozuru]

Originally Posted by ThomasClark

Or just make a TeamForge account and look at the source code? If you decompile the binary you'll lose all of the comments.

Here's another (simplified) reference for the protocol if you don't wanna make an account.

That's interesting to know, thanks. I didn't know that you could get a TeamForge account and view the source code. Every time I attempted to view it, I was unable to.

[fsilberberg]

You don't even need an account: wpilib.screenstepslive.com/s/4485/m/wpilib_source/l/480945-getting-the-source.