Tracking Rectangles with Java/C++

[shuhao]

Originally Posted by vinnie

My team is going to be putting a small Atom powered computer on our robot to do vision processing and other high-level functions. We are going to be using the javacv library to utilize OpenCV in Java. We experimented with vision processing on the cRIO last year, but we found that it was very slow and often lagged the rest of the robot functions. You can easily (I wrote a demo program in ~10 minutes) detect edges/contours with OpenCV and from there decide whether or not the contours make the rectangle you're looking for or not.

How exactly are you guys accomplishing that? Power connection, networking? Not sure how that would be done as we're also thinking of doing that.

[JacobGH]

I'm working on this problem in Java. Am I crazy or do they only make a method available for detcting ellipses and nothing else? Can we, at least, access the values for individual pixels? That way, if they give us nothing else, we could at-least write our own image processing algorithms.

[shuhao]

I believe opencv and the axiscamera's JPEG is your best shot

[scottbot95]

basicxman, I tried the code you suggested and I found that we were finding a ridiculous amount of rectangles(around 42 million). Do you have any idea why this is happening? Also, is there any documentation for the imaqDetectRectangles function?

[RufflesRidge]

Originally Posted by scottbot95

basicxman, I tried the code you suggested and I found that we were finding a ridiculous amount of rectangles(around 42 million). Do you have any idea why this is happening? Also, is there any documentation for the imaqDetectRectangles function?

You probably want to filter by brightness (luminance) or color (probably in HSL or HSV space) before trying to detect rectangles.

[scottbot95]

I have a sheet of printer paper with the target(with correct proportions) printed on it except where the retro-reflective tape would be, we just printed green. I am then extracting the green plane and going from there.

[Ross3098]

Our team is also trying to figure out how to track the rectangles. Ive been spending a few hours or so looking over nivision.h as well as a few white papers and i seem to have gotten this far:

	
m_ModifiedImage = m_HSLImage->ThresholdHSL(80,125,45,60,115,130);
	
ImaqImage = m_ModifiedImage->GetImaqImage();

The white paper about the vision targets talks about applying a convex hull operation to really help those rectangles pop out. The main problem I have at the moment is that I have no idea how to apply said operation in C++. Ive found the imaqConvexHull() operation but have no clue how to start it.

[PriyankP]

Originally Posted by Ross3098

I have no idea how to apply said operation in C++. Ive found the imaqConvexHull() operation but have no clue how to start it.

This should help you get started! I'll be more helpful more once I see what the code I wrote does when I get it to run on a robot.

[basicxman]

Originally Posted by scottbot95

basicxman, I tried the code you suggested and I found that we were finding a ridiculous amount of rectangles(around 42 million). Do you have any idea why this is happening? Also, is there any documentation for the imaqDetectRectangles function?

Originally Posted by RufflesRidge

You probably want to filter by brightness (luminance) or color (probably in HSL or HSV space) before trying to detect rectangles.

Aye, if you generate C code from Vision Assistant it will call imaqThreshold too - something I forgot about in my original snippet.

[rudun]

Would this be how to use the detectRectangles function in java. We looked at the ellipseDetect and I am thinking that hey do the same thing just based on different descriptors.

    private static final BlockingFunction imaqDetectRectanglesFn =
            NativeLibrary.getDefaultInstance().getBlockingFunction("imaqDetectRectangles");
    static { imaqDetectRectanglesFn.setTaskExecutor(NIVision.taskExecutor); }
    private static Pointer numberOfRectanglesDetected = new Pointer(4);
    
     public static RectangleMatch[] detectRectangles(MonoImage image, RectangleDescriptor rectangleDescriptor,
            CurveOptions curveOptions, ShapeDetectionOptions shapeDetectionOptions,
            RegionOfInterest roi) throws NIVisionException {

        int curveOptionsPointer = 0;
        if (curveOptions != null)
            curveOptionsPointer = curveOptions.getPointer().address().toUWord().toPrimitive();
        int shapeDetectionOptionsPointer = 0;
        if (shapeDetectionOptions != null)
            shapeDetectionOptionsPointer = shapeDetectionOptions.getPointer().address().toUWord().toPrimitive();
        int roiPointer = 0;
        if (roi != null)
            roiPointer = roi.getPointer().address().toUWord().toPrimitive();

        int returnedAddress =
                imaqDetectRectanglesFn.call6(
                image.image.address().toUWord().toPrimitive(),
                rectangleDescriptor.getPointer().address().toUWord().toPrimitive(),
                curveOptionsPointer, shapeDetectionOptionsPointer,
                roiPointer,
                numberOfRectanglesDetected.address().toUWord().toPrimitive());

        try {
            NIVision.assertCleanStatus(returnedAddress);
        } catch (NIVisionException ex) {
            if (!ex.getMessage().equals("No error."))
                throw ex;
        }

        RectanglesMatch[] matches = RectanglesMatch.getMatchesFromMemory(returnedAddress, numberOfRectanglesDetected.getInt(0));
        NIVision.dispose(new Pointer(returnedAddress,0));
        return matches;
    }

[Ross3098]

What is imaqConvexHull() returning to? I have it returning to an integer but what value does the integer have?

[Greg McKaskle]

From the CVI documentation,

Return Value
Type

Description


int On success, this function returns a non-zero value. On failure, this function returns 0. To get extended error information, call imaqGetLastError().


Greg McKaskle

[dvanvoorst]

Originally Posted by rudun

Would this be how to use the detectRectangles function in java. We looked at the ellipseDetect and I am thinking that hey do the same thing just based on different descriptors.

    private static final BlockingFunction imaqDetectRectanglesFn =
            NativeLibrary.getDefaultInstance().getBlockingFunction("imaqDetectRectangles");
    static { imaqDetectRectanglesFn.setTaskExecutor(NIVision.taskExecutor); }
    private static Pointer numberOfRectanglesDetected = new Pointer(4);
    
     public static RectangleMatch[] detectRectangles(MonoImage image, RectangleDescriptor rectangleDescriptor,
            CurveOptions curveOptions, ShapeDetectionOptions shapeDetectionOptions,
            RegionOfInterest roi) throws NIVisionException {

        int curveOptionsPointer = 0;
        if (curveOptions != null)
            curveOptionsPointer = curveOptions.getPointer().address().toUWord().toPrimitive();
        int shapeDetectionOptionsPointer = 0;
        if (shapeDetectionOptions != null)
            shapeDetectionOptionsPointer = shapeDetectionOptions.getPointer().address().toUWord().toPrimitive();
        int roiPointer = 0;
        if (roi != null)
            roiPointer = roi.getPointer().address().toUWord().toPrimitive();

        int returnedAddress =
                imaqDetectRectanglesFn.call6(
                image.image.address().toUWord().toPrimitive(),
                rectangleDescriptor.getPointer().address().toUWord().toPrimitive(),
                curveOptionsPointer, shapeDetectionOptionsPointer,
                roiPointer,
                numberOfRectanglesDetected.address().toUWord().toPrimitive());

        try {
            NIVision.assertCleanStatus(returnedAddress);
        } catch (NIVisionException ex) {
            if (!ex.getMessage().equals("No error."))
                throw ex;
        }

        RectanglesMatch[] matches = RectanglesMatch.getMatchesFromMemory(returnedAddress, numberOfRectanglesDetected.getInt(0));
        NIVision.dispose(new Pointer(returnedAddress,0));
        return matches;
    }

Where are RectangleMatch and RectangleDescriptor defined? I'd love to use this code to learn more about wrapping the IMAQ functions, but I'm running stuck getting it to compile.

[basicxman]

Originally Posted by dvanvoorst

Where are RectangleMatch and RectangleDescriptor defined? I'd love to use this code to learn more about wrapping the IMAQ functions, but I'm running stuck getting it to compile.

Check out nivision.h and the CVI documentation mentioned earlier.

RectangleMatch: http://mmrambotics.ca/wpilib/struct_...h__struct.html

RectangleDescriptor:
http://mmrambotics.ca/wpilib/struct_...r__struct.html

[Ross3098]

Originally Posted by PriyankP

This should help you get started! I'll be more helpful more once I see what the code I wrote does when I get it to run on a robot.

Correct me if I am wrong but does this mean that the integer that the imaqConvexHull() operation returns to is the score? And also I am wondering if the destination image is actually modified within the operation.