[pblankenbaker]

It sounds like you are hitting a similar issue to what has been driving us nuts for the past couple of weeks. We had planned to move the image processing to the driver station to keep some of the load off of the cRIO. We had planned on doing this by creating a SmartDashboard extension.

The SmartDashboard project made it very easy to create WPICameraExtension classes to do image processing and display the results directly in the dashboard.

Unfortunately, we have struggled to get any WPICameraExtension to run for any extended period of time. Somewhere in the WPI/javacv/opencv layering of APIs there exists some serious memory management issues that cause major amounts of memory leakage in the native code once you start finding contours. It is extremely frustrating for our developer as the crash seems to be outside of our code.

We ended up creating a trivial extension which does basically nothing to demonstrate the memory leak (it displays a gray image in the dashboard).

package com.techhounds.camera;
import edu.wpi.first.smartdashboard.camera.WPICameraExtension;
import edu.wpi.first.wpijavacv.WPIBinaryImage;
import edu.wpi.first.wpijavacv.WPIColorImage;
import edu.wpi.first.wpijavacv.WPIContour;
import edu.wpi.first.wpijavacv.WPIGrayscaleImage;
import edu.wpi.first.wpijavacv.WPIImage;

/**
 * A "trivial" SmartDashboard camera extension demonstrating the memory
 * leak issue in the
 *
 * @author pkb
 */
public class MemoryLeakExtension extends WPICameraExtension {

    @Override
    public WPIImage processImage(WPIColorImage colorImg) {

        // Pull out one color
        WPIGrayscaleImage grayImg = colorImg.getBlueChannel();

        // Reduce to black and white for contour tracing
        WPIBinaryImage binImg = grayImg.getThreshold(200);

        // Find the contours within the image (unfortunately memory
        // is leaked big time as soon as it starts finding contours)
        WPIContour[] contours = binImg.findContours();

        // Let the dash board display the gray image
        return grayImg;
    }

}

Removal of the binImg.findContours() invocation or setting a threshold such that no contours are found prevents the memory leak from occurring.

Maybe there is something obviously wrong in the above example - but from what I can tell (there aren't a lot of examples) the code above should be legal and should not cause the JVM to crash (native errors in Java are not fun to track down).

We've reported this as a bug to the SmartDashboard project (artf1466) a few days ago but haven't heard anything yet (I'm sure they are pretty busy).

We aren't entirely certain if the issue is with the WPI wrapper around javacv, the javacv wrapper around opencv or some other dependency.

We have had better luck by avoiding the WPI image processing classes, but it involves a lot of guess work as we aren't familiar with the underlying opencv technology and the javacv and WPI layers don't have a lot of examples or docs to go by.

So, if you don't have a lot of time committed to pursuing image processing via the Java SmartDashboard extension this season, I'd recommend avoiding it.

That being said, we are still pursing doing the image processing in Java on the driver station, but are trying to figure out how to do it outside of the SmartDashboard environment and with minimal (if any) use of the current WPI image processing classes.

Here's one of the methods we've been working on that works directly with the javacv API for the purpose of locating contours and the the polygons within the contours. It's a work in progress and a lot of trial and error, but it's at least been stable so far.

    /**
     * Looks for polygons within a image and adds them to the collection.
     *
     * @param img Typically a binary or gray scale image.
     *
     * @return Number of polygons found.
     */
    public int findPolygons(IplImage iimg) {
        int fnd = 0;

        // javacv crud
        CvMemStorage buf = CvMemStorage.create();
        CvSeq contours = new CvSeq();

        // From findContours() in WPIBinaryImage.java
        //int total = cvFindContours(iimg, buf, contours,
        //        256, CV_RETR_LIST, CV_CHAIN_APPROX_TC89_KCOS);

        int total = cvFindContours(iimg, buf, contours,
                Loader.sizeof(CvContour.class), 0, CV_CHAIN_APPROX_SIMPLE);

        if (total > 0) {
            while ((contours != null) && !contours.isNull() && (fnd < maxFind)) {

                if (contours.elem_size() > 0) {
                    CvSeq poly = null;

                    CvRect rect = cvBoundingRect(contours, 0);
                    int width = rect.width();
                    int height = rect.height();

                    CvMemStorage pbuf = CvMemStorage.create();
                    if (checkDimensions(width, height)) {
                        poly = cvApproxPoly(contours, contours.header_size(),
                                pbuf, CV_POLY_APPROX_DP, this.percentAccuracy, 0);
                    }

                    if (poly != null) {
                        int n = poly.total();
                        if ((n >= minPoints) && (n <= maxPoints)) {
                            ppolys.add(Points.fromCvPoints(poly));
                            boundingBoxes.add(new Dimension(width, height));
                            fnd++;
                        }
                    }
                }
                contours = contours.h_next();
            }
        }

        // This should be done automatically when storage objects are finalized
        //buf.release();
        return fnd;
    }

Not sure if I really helped you at all - but at least you know there's someone else out there feeling your pain.