View Single Post
  #22   Spotlight this post!  
Unread 29-01-2013, 09:30
violinuxer violinuxer is offline
Registered User
FRC #2523
 
Join Date: Jan 2012
Location: Vermont
Posts: 13
violinuxer is an unknown quantity at this point
Re: Walkthrough on how to do offboard image processing!

Here is how we did it last year:

Smart dashboard comes with the wrapper to OpenCV. You install SmartDashboard via the installer (available at firstforge.wpi.edu) and add the included library jar files (there are tutorial pdfs available on the web) into a pure java Netbeans project. You then extend WPICameraExtension in which you can process an image, do thresholding, polygon detection, etc. etc. Any calculated values are then sent back to the robot via NetworkTables.

Below is our image processing code from last year. Note the synchronized block, that's how we sent the values back to the robot.

Code:
import edu.wpi.first.smartdashboard.camera.WPICameraExtension;
import edu.wpi.first.smartdashboard.properties.DoubleProperty;
import edu.wpi.first.smartdashboard.properties.IntegerProperty;
import edu.wpi.first.wpijavacv.*;
import edu.wpi.first.wpilibj.networking.NetworkTable;
import java.util.ArrayList;


public class VisionProcessing extends WPICameraExtension {


     public static final String NAME = "Camera Target Tracker";
     public final IntegerProperty threshold = new IntegerProperty(this, "Threshold", 180);
     public final DoubleProperty contourPolygonApproximationPct = new DoubleProperty(this, "Polygon Approximation %", 45);
     NetworkTable table = NetworkTable.getTable("camera");
        WPIColor targetColor = new WPIColor (0,255,0);
        WPIColor contourColor = new WPIColor (17,133,133);

        @Override
        public WPIImage processImage(WPIColorImage rawImage) {
            WPIBinaryImage blue = rawImage.getBlueChannel().getThreshold(threshold.getValue());
            WPIBinaryImage green = rawImage.getGreenChannel().getThreshold(threshold.getValue());
            WPIBinaryImage red = rawImage.getRedChannel().getThreshold(threshold.getValue());
            // Does the thresholding;

            WPIBinaryImage colorsCombined = blue.getAnd(red).getAnd(green);
            //Mixes the paint;

            colorsCombined.erode(2);
            colorsCombined.dilate(6);
            //Gets rid of small stuff and fills holes;

            WPIContour[] contours = colorsCombined.findContours();
            rawImage.drawContours(contours, contourColor, 3);
            ArrayList<WPIPolygon> polygons = new ArrayList<WPIPolygon>();

            for (WPIContour c : contours) {
                double ratio = ((double)c.getHeight())/ ((double)c.getWidth());
                //if (ratio < 1.5 && ratio > .75)
                if (1==1) {
                    polygons.add(c.approxPolygon(contourPolygonApproximationPct.getValue()));
                    //Makes polygons around edges;
                }
            }
            ArrayList<WPIPolygon> possiblePolygons = new ArrayList<WPIPolygon>();
            for (WPIPolygon p : polygons) {
                if (p.isConvex() && p.getNumVertices() == 4) {
                    possiblePolygons.add(p);
                } else {
                    rawImage.drawPolygon(p, WPIColor.MAGENTA, 1);
                }
            }
            WPIPolygon square = null;

            int squareArea = 0;
            double centerX = 0;
            double heightRatio = 0;

            for (WPIPolygon p : possiblePolygons) {
                rawImage.drawPolygon(p, WPIColor.BLUE, 5);
                for (WPIPolygon q : possiblePolygons) {
                    if (p == q)
                        continue;
                    int pCenterX = (p.getX() + (p.getWidth()/2));
                    int qCenterX = (q.getX() + (q.getWidth()/2));
                    int pCenterY = (p.getY() + (p.getHeight()/2));
                    int qCenterY = (q.getY() + (q.getHeight()/2));
//                    rawImage.drawPoint(new WPIPoint(pCenterX, pCenterY), targetColor, 5);
//                    rawImage.drawPoint(new WPIPoint(qCenterX, qCenterY), targetColor, 5);
                    if (Math.abs(pCenterX - qCenterX) < 20 && Math.abs(pCenterY - qCenterY) < 20) {
                         int pArea = Math.abs(p.getArea());
                         int qArea = Math.abs(q.getArea());
                         if (square != null) {
                             if (square.getY() < p.getY())
                                 continue;
                                 // if we already have a square, and it is lower, doesn't add anything
                         }
                         if (pArea>qArea) {
                             square = p;
                             squareArea = pArea;
                             centerX = (double)pCenterX/rawImage.getWidth()-.5;
                         } else {
                             square = q;
                             squareArea = qArea;
                             centerX = (double)qCenterX/rawImage.getWidth()-.5;
                         }
                         WPIPoint [] v = square.getPoints();
                         int x1 = Math.abs(v[1].getX() - v[0].getX());
                         int y1 = Math.abs(v[1].getY() - v[0].getY());
                         int y2 = Math.abs(v[2].getY() - v[1].getY());
                         int y3 = Math.abs(v[3].getY() - v[2].getY());
                         int y4 = Math.abs(v[0].getY() - v[3].getY());
                         if (y1 > x1) { // first segment isvertical
                             heightRatio = (double)y1 / y3;
                         } else {
                             heightRatio = (double)y2 / y4;
                         }
                         break;
                    }
                }
            }

            if (square != null) {
                int sCenterX = (square.getX() + (square.getWidth()/2));
                int sCenterY = (square.getY() + (square.getHeight()/2));
                rawImage.drawPoint(new WPIPoint(sCenterX, sCenterY), targetColor, 5);
            }

            synchronized (table) {
                table.beginTransaction();
                if (square != null)
                {
                    double distance = (1564.4/Math.sqrt(squareArea) - 0.5719) * 12;
                    table.putBoolean("found", true);
                    table.putDouble("area", squareArea);
                    table.putDouble("distance", distance);
                    table.putDouble("xoffset", -centerX);
                    table.putDouble("heightratio", heightRatio);
                }
                else
                {
                    table.putBoolean("found", false);
                }
                table.endTransaction();
            }
            return rawImage;
    }
}
Anyhow, I'll see if I can find the docs on getting WPIJavaCV set up and will post later.

violinuxer
__________________
May the source be with you!!

(Linux FTW!)