Go to Post Only a few more days till i get to see her again! This is sad you would think I was talking about a chick... no wonder my gf gets jealous. XD - sportzkrazzy [more]
Home
Go Back   Chief Delphi > Technical > Programming > Java
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
Thread Tools Rating: Thread Rating: 4 votes, 5.00 average. Display Modes
  #1   Spotlight this post!  
Unread 02-02-2012, 20:19
coo coo is offline
Registered User
AKA: Christopher
FRC #0503 (Frog Force)
Team Role: Programmer
 
Join Date: Jan 2010
Rookie Year: 2009
Location: Novi
Posts: 3
coo is an unknown quantity at this point
SmartDashboard Vision Processing

Has anyone managed to get the square tracking code that's provided or any javaCV code working using the SmartDashboard? We've been trying to get it working writing our own methods using JavaCV or the square tracking code and getting consistent EXCEPTION_ACCESS_VIOLATION errors with java, causing the dashboard to crash. The lack of documentation has made it really frustrating to try to even fix any of these problems, because there's no documentation at all.
Reply With Quote
  #2   Spotlight this post!  
Unread 02-02-2012, 21:34
pblankenbaker pblankenbaker is offline
Registered User
FRC #0868
 
Join Date: Feb 2012
Location: Carmel, IN, USA
Posts: 108
pblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of light
Re: SmartDashboard Vision Processing

It sounds like you are hitting a similar issue to what has been driving us nuts for the past couple of weeks. We had planned to move the image processing to the driver station to keep some of the load off of the cRIO. We had planned on doing this by creating a SmartDashboard extension.

The SmartDashboard project made it very easy to create WPICameraExtension classes to do image processing and display the results directly in the dashboard.

Unfortunately, we have struggled to get any WPICameraExtension to run for any extended period of time. Somewhere in the WPI/javacv/opencv layering of APIs there exists some serious memory management issues that cause major amounts of memory leakage in the native code once you start finding contours. It is extremely frustrating for our developer as the crash seems to be outside of our code.

We ended up creating a trivial extension which does basically nothing to demonstrate the memory leak (it displays a gray image in the dashboard).

Code:
package com.techhounds.camera;
import edu.wpi.first.smartdashboard.camera.WPICameraExtension;
import edu.wpi.first.wpijavacv.WPIBinaryImage;
import edu.wpi.first.wpijavacv.WPIColorImage;
import edu.wpi.first.wpijavacv.WPIContour;
import edu.wpi.first.wpijavacv.WPIGrayscaleImage;
import edu.wpi.first.wpijavacv.WPIImage;

/**
 * A "trivial" SmartDashboard camera extension demonstrating the memory
 * leak issue in the
 *
 * @author pkb
 */
public class MemoryLeakExtension extends WPICameraExtension {

    @Override
    public WPIImage processImage(WPIColorImage colorImg) {

        // Pull out one color
        WPIGrayscaleImage grayImg = colorImg.getBlueChannel();

        // Reduce to black and white for contour tracing
        WPIBinaryImage binImg = grayImg.getThreshold(200);

        // Find the contours within the image (unfortunately memory
        // is leaked big time as soon as it starts finding contours)
        WPIContour[] contours = binImg.findContours();

        // Let the dash board display the gray image
        return grayImg;
    }

}
Removal of the binImg.findContours() invocation or setting a threshold such that no contours are found prevents the memory leak from occurring.

Maybe there is something obviously wrong in the above example - but from what I can tell (there aren't a lot of examples) the code above should be legal and should not cause the JVM to crash (native errors in Java are not fun to track down).

We've reported this as a bug to the SmartDashboard project (artf1466) a few days ago but haven't heard anything yet (I'm sure they are pretty busy).

We aren't entirely certain if the issue is with the WPI wrapper around javacv, the javacv wrapper around opencv or some other dependency.

We have had better luck by avoiding the WPI image processing classes, but it involves a lot of guess work as we aren't familiar with the underlying opencv technology and the javacv and WPI layers don't have a lot of examples or docs to go by.

So, if you don't have a lot of time committed to pursuing image processing via the Java SmartDashboard extension this season, I'd recommend avoiding it.

That being said, we are still pursing doing the image processing in Java on the driver station, but are trying to figure out how to do it outside of the SmartDashboard environment and with minimal (if any) use of the current WPI image processing classes.

Here's one of the methods we've been working on that works directly with the javacv API for the purpose of locating contours and the the polygons within the contours. It's a work in progress and a lot of trial and error, but it's at least been stable so far.

Code:
    /**
     * Looks for polygons within a image and adds them to the collection.
     *
     * @param img Typically a binary or gray scale image.
     *
     * @return Number of polygons found.
     */
    public int findPolygons(IplImage iimg) {
        int fnd = 0;

        // javacv crud
        CvMemStorage buf = CvMemStorage.create();
        CvSeq contours = new CvSeq();

        // From findContours() in WPIBinaryImage.java
        //int total = cvFindContours(iimg, buf, contours,
        //        256, CV_RETR_LIST, CV_CHAIN_APPROX_TC89_KCOS);

        int total = cvFindContours(iimg, buf, contours,
                Loader.sizeof(CvContour.class), 0, CV_CHAIN_APPROX_SIMPLE);

        if (total > 0) {
            while ((contours != null) && !contours.isNull() && (fnd < maxFind)) {

                if (contours.elem_size() > 0) {
                    CvSeq poly = null;

                    CvRect rect = cvBoundingRect(contours, 0);
                    int width = rect.width();
                    int height = rect.height();

                    CvMemStorage pbuf = CvMemStorage.create();
                    if (checkDimensions(width, height)) {
                        poly = cvApproxPoly(contours, contours.header_size(),
                                pbuf, CV_POLY_APPROX_DP, this.percentAccuracy, 0);
                    }

                    if (poly != null) {
                        int n = poly.total();
                        if ((n >= minPoints) && (n <= maxPoints)) {
                            ppolys.add(Points.fromCvPoints(poly));
                            boundingBoxes.add(new Dimension(width, height));
                            fnd++;
                        }
                    }
                }
                contours = contours.h_next();
            }
        }

        // This should be done automatically when storage objects are finalized
        //buf.release();
        return fnd;
    }
Not sure if I really helped you at all - but at least you know there's someone else out there feeling your pain.
Reply With Quote
  #3   Spotlight this post!  
Unread 05-02-2012, 17:17
dominique dominique is offline
Registered User
FTC #0211 (MK211)
Team Role: Programmer
 
Join Date: Jan 2011
Rookie Year: 2011
Location: Rochester,NY
Posts: 9
dominique is an unknown quantity at this point
Re: SmartDashboard Vision Processing

Message to pblankenbaker - thanks for this post. I am interested in the second part (finding polygons within an image) and I have several questions:

a- Can you share what are your imports ?

b- Do you have a way to exchange between javacv data and WPI data ? In other words can you have in the same method both a WPIColorImage object and an IplImage object and manipulate them ? How do you do that ?

Thanks.
Reply With Quote
  #4   Spotlight this post!  
Unread 06-02-2012, 11:30
pblankenbaker pblankenbaker is offline
Registered User
FRC #0868
 
Join Date: Feb 2012
Location: Carmel, IN, USA
Posts: 108
pblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of light
Re: SmartDashboard Vision Processing

No problem dominique.

a - I've attached the PolygonFinder.java source file (and Points.java helper class) that we are/were using to access the javacv stuff outside of the WPI API.

b - I've attached the WPIImageWrapper.java class which we are using to get access to the underlying IplImage object associated with each WPIImage object. To use this wrapper class, you construct a instance passing in a WPIImage object to the constructor. You can then use the getIplImage() method to gain access to the underlying IplImage.

NOTE: There appears to be a fix to the WPI image processing classes in the SmartDashboard that takes care of the memory leak. See this post.

The PolygonFinder.java file attached has two entry points (one that takes a WPIBinaryImage and one that takes a IplImage).

Hope that helps,
Paul
Attached Files
File Type: java PolygonFinder.java (12.0 KB, 133 views)
File Type: java Points.java (2.9 KB, 75 views)
File Type: java WPIImageWrapper.java (418 Bytes, 130 views)
Reply With Quote
  #5   Spotlight this post!  
Unread 06-02-2012, 14:13
dominique dominique is offline
Registered User
FTC #0211 (MK211)
Team Role: Programmer
 
Join Date: Jan 2011
Rookie Year: 2011
Location: Rochester,NY
Posts: 9
dominique is an unknown quantity at this point
Re: SmartDashboard Vision Processing

Paul, you are not helping me...you are saving me...

Thanks.
Reply With Quote
  #6   Spotlight this post!  
Unread 06-02-2012, 14:33
Jared Russell's Avatar
Jared Russell Jared Russell is offline
Taking a year (mostly) off
FRC #0254 (The Cheesy Poofs), FRC #0341 (Miss Daisy)
Team Role: Engineer
 
Join Date: Nov 2002
Rookie Year: 2001
Location: San Francisco, CA
Posts: 3,080
Jared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond repute
Re: SmartDashboard Vision Processing

Paul,

Good work finding that leak! We had not tested our code for a sufficient length of time to notice the problem (yet) - you no doubt have saved us a huge headache.
Reply With Quote
  #7   Spotlight this post!  
Unread 13-02-2012, 23:06
detruby detruby is offline
Dave Truby
AKA: Dave Truby
FRC #4145 (Wolf Pack)
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2012
Location: Columbus
Posts: 11
detruby is an unknown quantity at this point
Re: SmartDashboard Vision Processing

I really need the fix for bug artf1446 - memory leak around findContours. The dashboard crashes after a minute - which makes it unusable for the competition. Is anyone working on the fix? I don't see anyone assigned.
Reply With Quote
  #8   Spotlight this post!  
Unread 14-02-2012, 10:31
pblankenbaker pblankenbaker is offline
Registered User
FRC #0868
 
Join Date: Feb 2012
Location: Carmel, IN, USA
Posts: 108
pblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of lightpblankenbaker is a glorious beacon of light
Re: SmartDashboard Vision Processing

See:

http://www.chiefdelphi.com/forums/sh...67&postcount=6

It contains instructions and a replacement JAR file that appears to fix the memory leak issue (NOTE: it is not a official build - just something we checked out and built ourselves).
Reply With Quote
  #9   Spotlight this post!  
Unread 15-02-2012, 16:23
detruby detruby is offline
Dave Truby
AKA: Dave Truby
FRC #4145 (Wolf Pack)
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2012
Location: Columbus
Posts: 11
detruby is an unknown quantity at this point
Re: SmartDashboard Vision Processing

I found the bug in edu.wpi.first.wpijavacv.WPIBinaryImage.java in project WPIJavaCV. The findContours() method was doing

final CvMemStorage storage = CvMemStorage.create();

without calling deallocate()

I moved the "storage" variable to an instance variable and added method

protected void disposed() {
super.disposed();
this.storage.deallocate();
}

all better now.
Reply With Quote
  #10   Spotlight this post!  
Unread 16-02-2012, 08:09
Jared Russell's Avatar
Jared Russell Jared Russell is offline
Taking a year (mostly) off
FRC #0254 (The Cheesy Poofs), FRC #0341 (Miss Daisy)
Team Role: Engineer
 
Join Date: Nov 2002
Rookie Year: 2001
Location: San Francisco, CA
Posts: 3,080
Jared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond reputeJared Russell has a reputation beyond repute
Re: SmartDashboard Vision Processing

I was under the impression that the JavaCV XXX.create(...) methods were properly garbage collected...?
Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 12:54.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi