Java "OutOfMemoryError" with OpenCV's Imgproc.findContours()

Technical Java
1

Hey folks! Amidst work on this year’s vision code, I’m running into a problem. After 2+ hours of searching, I’m stumped.

We have this code running:

Mat source = new Mat();
Mat processingImg = new Mat();

while (!Thread.interrupted()) {
  // Get frame from camera
  cvSink.grabFrame(source);
  if (source.empty()) {
    System.out.println("Cannot get camera frame");
    continue;
  }

  // Convert BGR to HSV (easier to process hue than rgb)
  Imgproc.cvtColor(source, processingImg, Imgproc.COLOR_BGR2HSV);

  // Turn green (within range) white, all else black
  Core.inRange(
    processingImg,
    new Scalar(50, 50, 40),
    new Scalar(90, 150, 245),
    processingImg
  );

  // Blur black + white image (remove outlier pixels)
  Imgproc.blur(processingImg, processingImg, new Size(10, 10));

  // Find countours (white objects) in image
  ArrayList<MatOfPoint> contours = new ArrayList<MatOfPoint>();
  ///////////////// THE CULPRIT LINE: /////////////////
  Imgproc.findContours(processingImg, contours, new Mat(), Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);

  // Send vision feed to dashboard
  outputStream.putFrame(source);
}

The code works for about 60 seconds before the following error appears:

 OpenCV(3.4.4) Error: Insufficient memory (Failed to allocate 921600 bytes) in OutOfMemoryError, file /var/lib/jenkins/workspace/OpenCV/OpenCV - Release/arm/opencv/modules/core/src/alloc.cpp, line 55 
 Exception in thread "Thread-0" CvException [org.opencv.core.CvException: cv::Exception: OpenCV(3.4.4) /var/lib/jenkins/workspace/OpenCV/OpenCV - Release/arm/opencv/modules/core/src/alloc.cpp:55: error: (-4:Insufficient memory) Failed to allocate 921600 bytes in function 'OutOfMemoryError' 
 ] 
 	at org.opencv.imgproc.Imgproc.cvtColor_1(Native Method) 
 	at org.opencv.imgproc.Imgproc.cvtColor(Imgproc.java:2223) 
 	at frc.robot.subsystems.Camera.run(Camera.java:74) 
 	at java.base/java.lang.Thread.run(Unknown Source)

After the error, the majority of the system continues to operate, but the camera thread freezes completely.

The error takes places on the cvtColor line, but that doesn’t actually seem to be the issue. Instead, removing the Imgproc.findContours() line solves the problem (the robot never runs out of memory), but finding the contours is crucial to the vision code’s success.

Any ideas?

1 Like
2

Is this on the RoboRIO? Either way, try downscaling the image before processing. For the record - running vision on the RIO can be a bit tricky - it’s pretty slow. Consider using a coprocessor like a Raspberry Pi or a JeVois camera?

3

This would probably help, but would only delay the inevitable, right? If data isn’t being garbage collected correctly (my naive assumption), it’ll still cause problems, just at a slower rate…

(A coprocessor is something I’ve been thinking about, but haven’t pushed for, mostly because the team was successful in past years without one. I’ll take your advice into consideration.)

4

This is odd… I decreased the camera resolution in the code from 640x480 to 320x240:

camera = CameraServer.getInstance().startAutomaticCapture();
camera.setResolution(320, 240);

But when I do, the cvSink.grabFrame(source); line stops working. source.empty() is always true, so the message “Cannot get camera frame” is continually logged (see code above).

The webcam we’re using supports up to 1080p. Is it possible that it won’t even allow grabbing a lower resolution frame than 640x480? That would be odd, but it’s my only guess. :confused:

5

Reuse preallocated objects instead of creating new ones in the loop. This especially applies to Mat and it’s various specializations, but is also important for any OpenCV type since the Java garbage collector only sees their size in the JVM (small) and not the size of the native data (potentially very large)

In particular, the new ArrayList<MatOfPoint>() call, new Mat() for the hierarchy parameter, and the new size and scalar objects should all be created before the loop

3 Likes
6

That made a huge impact. At low resolutions, the code is now able to run seemingly indefinitely (many minutes).

Unfortunately, it still crashes when any resolution higher than 160x120 is selected in the dashboard. Looks like a coprocessor may be necessary after all…

1 Like
7