(Note - I originally posted this in the NI LabView forum. I copied and pasted it over here as I would think it would get more visibility in the Programming forum.)
I am trying to help Team 2751 to do vision tracking for the first time this year. We program our robot using Java, and I am curious how much of the NI Image Processing library is really available to us. A few questions to anyone kind enough to answer:
Other than getting the settings from the color thresholding operation, how much of the NI Vision Assistant algorithm prototyping can be transferred to Java? For example, I did some template matching in the Vision Assistant that works well. I can, of course, save the script for inclusion in VI. Is it possible to access VI from Java?
In the Javadoc reference, the package to access NI’s nvision library is edu.wpi.first.wpilibj.image. It only has the following classes referenced in documentation:
BinaryImage
ColorImage
CurveOptions
EllipseDescriptor
EllipseMatch
HSLImage
Image
MonoImage
ParticleAnalysisReport
RegionOfInterest
RGBImage
ShapeDetectionOptions
These classes appear to be there to support the Java sample machine vision projects from last year. Are other NI image processing functions available through wpilibj or is it just based on the people writing wrapper classes for the functions that are needed?
According to the WPI Robotics Library Users Guide - which is ostensibly for Java and C++ - there is reference to the FRC Vision API Specification document, which gets installed with WindRiver. Is it only available in WindRiver? Do I really have to install the IDE for C++ that I don’t need since I am using Java NetBeans?
The snippet of information I see in the Library Users Guide says that the FRC Vision Interface includes high level calls for color tracking through TrackingAPI.cpp. It also says programmers may call into the low level library by using nvision.h. Are the trackingAPI and/or the low level calls available to Java? The Java VM we have doesn’t support JNI - which is the typical way to make calls to C-libraries.
In summary, it looks like for image processing, LabView has the most support, followed by WindRiver C++, with Java bringing up the rear. From reading this forum, however, I see that several of you are using Java. Are the issues I raise here really not that big of a deal? How did you overcome them?
Any answers and guidance to getting started would be most appreciated. Thanks for your help and good luck during the build season.
Those were nearly the exact set of observations we made while using Java this year, before switching to C++. Basically, the Java WPILib does not expose any nivision.h (imaq*) functions, and consequently makes it very difficult to do any effective image processing in Java. One of the options we could have taken was to generate Java wrappers for all those functions, but since I don’t know Java that well and it would have been a pain to expose everything, we decided to switch to C++.
If you know C++, I would recommend switching if image processing is one of your requirements.
I’m not sure if more of the IMAQ functions were wrapped this year. Being open source, if one team does it and submits it, all teams could use it, so I’d think it would have happened by now. Anyway, for the vision I did this year, I only used one function not in the list. I used Color Threshold as well.
I think you’re making some incorrect presumptions here.
For the first week of build season, I spent a lot of time searching the WPILib docs looking for image processing functions. All I found were some thresholding functions, particle analysis reports, and some detect ellipses off to the side. I was looking for a real image processing library, like OpenCV. We wanted to explore techniques like flickering lights on and off to identify the targets. We were using Java. So, I spent literally days trying to figure out how to achieve what I wanted to. Finally, after going through the source code of WPILib in C++, I found functions that started with “imaq” and then found nivision.h. I learned by reading through the source code that the undocumented Image pointers returned by WPILib functions went with these functions. Even then, I wasn’t sure they would work. They could be completely unrelated. This was completely by chance.
Teams should not be expected to have to go through this, or be able to. This is not the standard that teams who are just learning how to program should be able to reach. If we want teams to actually do some real programming, instead of having fun calling a few cute functions to see what happens, then those who actually have control of how this library is made need to see this as simply inadequate.
I would like to ask what specifically you are trying to do. NI makes an Machine Vision Library, that they left FIRST teams use. Many of the functions in the library have C++ and I assume Java wrappers. For more information on what the imaq* functions do I would refer you to the NIVisionCVI.chm file found in the documents folder for WindRiver. I’m not sure where, or if its given with the Java install.
If you give us a better idea of what you are trying to accomplish, we might be able to give you more help.
I would suggest moving image processing to the driver station PC. Connect the camera to the bridge and get pictures from <ip>/axis-cgi/jpg/image.cgi?resolution=320x240. Then you can use what you want.
To send/receive data back to the cRIO, look at Connector.open()], SocketConnection, HttpConnection, DataInputStream, and DataOutputStream (on the cRIO side, that is).
After looking into it more, I would have to agree with parts of your first post. I imagine labview has direct support for all of the imaging functions, and C++ seems to have pretty good support for all of the functions. Some of those functions have wrappers to make their use easier.
After looking at the Java libraries, it would appear that the entire NIvision library is available, but like I said, it takes a little more effort, as there aren’t wrappers for all of the functions. I would direct you to NativeLibrary.getDefaultInstance().getFunction(); for more info on using the other functions in the library.
The NI Vision libraries are commercially available with a C and LV interface and documentation. NI has no Java libraries to donate. When Java was introduced for the cRIO, the plan was for Java classes to wrap the C entry points. This apparently was not completed. The libraries are open source, and as I said, I’d have thought they would be more complete by now.