Grip

Hi! This is team’s 8087 lead programmer in need of desperate need of assistance with GRIP image processor and java for the 2020 FRC season.

Our team is trying to use a single USB camera with GRIP to move our robot during AUTONOMOUS, so far i have the GRIP(camera input, resize, HSV[values], erode, mask, find_lines) these are in order inside of GRIP. I’ve tried generating code to our code folder though i do not know where to put it, and how to import a bit of code into the robot.java code to process images. I know at least they get piped in using network tables but I don’t know how to code this nor do i know how to add the camera stream into our code. If anyone could take a little bit of time to look at this and help us out we would be very grateful. if you have any questions contact me anytime directly at {contact info removed at OP request}

Here are the basics on how to get the Camera stream to the dashboard.

Now, that link references source code has the following section:

while(!Thread.interrupted()) {
        if (cvSink.grabFrame(source) == 0) {
          continue;
        }
        Imgproc.cvtColor(source, output, Imgproc.COLOR_BGR2GRAY);
        outputStream.putFrame(output);
      }

That is the guts of where you will put in the processing done in your GRIP pipeline.

Here is a link to code that we used in 2018’s game. Last two years we invested in the Limelight, so we don’t have anything more current.

Let me know if you have more questions.

Also I forgot this link that gives you source as to how to integrate it into the robot structure.

Hello, thanks for the reply the link for the 2018 game is dead 404 on github.

I didnt realize i should have noted the structure of our folder it being \FIRST CODE\src\main\java\frc\robot i put the grip file in both the robot file and frc folders just to be sure

though when i import the file i get an error on the us.org and the team190 i switch the line to:
frc.robot.GripPipeline.myvisionpipeline;
Java throws errors on the entire line.
in a basic sense I would like to know how to generate the code from GRIP application into a java format, export the code into the correct path and get the network tables into the code so i can run it from there. if that helps with saving our teams code let me know if not feel free to ask questions, i know im not very clear. Thanks again your awesome.

[quote=“team_8087, post:5, topic:374545, full:true”]
Hello, thanks for the reply the link for the 2018 game is dead 404 on github.[/quote]

I fixed the repo so it’s not private anymore.

Make sure that you follow the steps here to generate the code.

From that page:

Java Users: the generated class lacks a package declaration, 
so a declaration should be added to match the directory 
where the file was saved.

So make sure that you add a package declaration that matches your source tree, which if you’re using the generated templates is probably frc.robot.

If you follow the documentation on those examples it will work, but you have to read all the sections to see what you are doing. Once you finish the Code Generation section I linked to above, you click the Next button on the documentation on how to use the generated code..

That link shows you how to use the code to find the center of your filtered contours in autonomousPeriodic but you could use it the same way for Teleop.

so i generated the java code with this config:
lan-java
implement WPILib Visionpipeline - checked
pipeline class name - GripPipeline
save location- \ROBOTICS CODE FRC\First code\src\main\java\frc\robot

package name - gripmaybe

when i try to import the package into my code on this line specifically

import org.usfirst.frc.team190.grip.MyVisionPipeline;

I’ve tried following the steps you’ve provided though to no avail
would it be possible you contact me on the phone to talk it through?
Thanks again friend!

Probably can’t talk via phone in the near future but if you put your code on Github where I can look at it I can probably help you get it working.

Your package name should be set to frc.robot since that’s the directory into which you’re telling GRIP to place the generated file. You import it using import frc.robot.GripPipeline

Sorry we have been busy with the robot, this is my code thus far i get an error when trying to

package frc.robot;
import frc.robot.GripPipeline;
import frc.robot.GripPipeline.MyVisionPipeline;
import edu.wpi.first.wpilibj.TimedRobot;
import org.opencv.core.Rect;
import org.opencv.imgproc.Imgproc;
import edu.wpi.cscore.UsbCamera;
import edu.wpi.first.cameraserver.CameraServer;

public class Robot extends TimedRobot {
private static final int IMG_WIDTH = 320;
private static final int IMG_HEIGHT = 240;
private double centerX = 0.0;
private final Object imgLock = new Object();
@Override
public void robotInit() {
UsbCamera camera = CameraServer.getInstance().startAutomaticCapture();
camera.setResolution(IMG_WIDTH, IMG_HEIGHT);

THIS LINE |+++++++++++++++++++++++++HERE__V+++++++++++++++++++++++
visionThread = new VisionThread(camera, new .visionpipeline(), pipeline -> {
if (!pipeline.filterContoursOutput().isEmpty()) {
Rect r = Imgproc.boundingRect(pipeline.filterContoursOutput().get(0));
synchronized (imgLock) {
centerX = r.x + (r.width / 2);}}});
visionThread.start();}

On that line any form of me trying to rename it gives me an error for “unknown type” , Me and my team dont know where to go from here, any help would be amazing, thanks friend

I don’t see an import of VisionThread in the top of that file. That’s why it is an unknown type.

From that link:

Package edu.wpi.first.vision

Is what you should be importing.

new .visionpipeline() isn’t valid Java. Did you miss something when you pasted the code?

no i tried new GripPipeline.Myvisionpipeline and anything else i could think of i was trying to just bring attention to where i needed help

i forgot to add that back in before pasting it here, even with that it still does not work

Please share the contents of your GripPipeline file

package frc.robot;

import java.io.File;

import java.io.FileWriter;

import java.io.IOException;

import java.util.ArrayList;

import java.util.List;

import java.util.Map;

import java.util.stream.Collectors;

import java.util.HashMap;

import edu.wpi.first.wpilibj.vision.VisionPipeline;

import org.opencv.core.*;

import org.opencv.core.Core.*;

import org.opencv.features2d.FeatureDetector;

import org.opencv.imgcodecs.Imgcodecs;

import org.opencv.imgproc.*;

import org.opencv.objdetect.*;

/**

  • GripPipeline class.

  • An OpenCV pipeline generated by GRIP.

  • @author GRIP

*/

public class GripPipeline implements VisionPipeline {

//Outputs

private Mat cvResizeOutput = new Mat();

private Mat hsvThresholdOutput = new Mat();

private Mat cvErodeOutput = new Mat();

private Mat maskOutput = new Mat();

private ArrayList<Line> findLinesOutput = new ArrayList<Line>();

static {

    System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

}

/**

 * This is the primary method that runs the entire pipeline and updates the outputs.

 */

@Override   public void process(Mat source0) {

    // Step CV_resize0:

    Mat cvResizeSrc = source0;

    Size cvResizeDsize = new Size(0, 0);

    double cvResizeFx = 0.25;

    double cvResizeFy = 0.25;

    int cvResizeInterpolation = Imgproc.INTER_LINEAR;

    cvResize(cvResizeSrc, cvResizeDsize, cvResizeFx, cvResizeFy, cvResizeInterpolation, cvResizeOutput);

    // Step HSV_Threshold0:

    Mat hsvThresholdInput = cvResizeOutput;

    double[] hsvThresholdHue = {0.0, 32.55972696245734};

    double[] hsvThresholdSaturation = {43.57014388489208, 109.22354948805462};

    double[] hsvThresholdValue = {169.69424460431654, 239.76962457337885};

    hsvThreshold(hsvThresholdInput, hsvThresholdHue, hsvThresholdSaturation, hsvThresholdValue, hsvThresholdOutput);

    // Step CV_erode0:

    Mat cvErodeSrc = hsvThresholdOutput;

    Mat cvErodeKernel = new Mat();

    Point cvErodeAnchor = new Point(-1, -1);

    double cvErodeIterations = 1.0;

    int cvErodeBordertype = Core.BORDER_CONSTANT;

    Scalar cvErodeBordervalue = new Scalar(-1);

    cvErode(cvErodeSrc, cvErodeKernel, cvErodeAnchor, cvErodeIterations, cvErodeBordertype, cvErodeBordervalue, cvErodeOutput);

    // Step Mask0:

    Mat maskInput = cvResizeOutput;

    Mat maskMask = cvErodeOutput;

    mask(maskInput, maskMask, maskOutput);

    // Step Find_Lines0:

    Mat findLinesInput = maskOutput;

    findLines(findLinesInput, findLinesOutput);

}

/**

 * This method is a generated getter for the output of a CV_resize.

 * @return Mat output from CV_resize.

 */

public Mat cvResizeOutput() {

    return cvResizeOutput;

}

/**

 * This method is a generated getter for the output of a HSV_Threshold.

 * @return Mat output from HSV_Threshold.

 */

public Mat hsvThresholdOutput() {

    return hsvThresholdOutput;

}

/**

 * This method is a generated getter for the output of a CV_erode.

 * @return Mat output from CV_erode.

 */

public Mat cvErodeOutput() {

    return cvErodeOutput;

}

/**

 * This method is a generated getter for the output of a Mask.

 * @return Mat output from Mask.

 */

public Mat maskOutput() {

    return maskOutput;

}

/**

 * This method is a generated getter for the output of a Find_Lines.

 * @return ArrayList<Line> output from Find_Lines.

 */

public ArrayList<Line> findLinesOutput() {

    return findLinesOutput;

}

/**

 * Resizes an image.

 * @param src The image to resize.

 * @param dSize size to set the image.

 * @param fx scale factor along X axis.

 * @param fy scale factor along Y axis.

 * @param interpolation type of interpolation to use.

 * @param dst output image.

 */

private void cvResize(Mat src, Size dSize, double fx, double fy, int interpolation,

    Mat dst) {

    if (dSize==null) {

        dSize = new Size(0,0);

    }

    Imgproc.resize(src, dst, dSize, fx, fy, interpolation);

}

/**

 * Segment an image based on hue, saturation, and value ranges.

 *

 * @param input The image on which to perform the HSL threshold.

 * @param hue The min and max hue

 * @param sat The min and max saturation

 * @param val The min and max value

 * @param output The image in which to store the output.

 */

private void hsvThreshold(Mat input, double[] hue, double[] sat, double[] val,

    Mat out) {

    Imgproc.cvtColor(input, out, Imgproc.COLOR_BGR2HSV);

    Core.inRange(out, new Scalar(hue[0], sat[0], val[0]),

        new Scalar(hue[1], sat[1], val[1]), out);

}

/**

 * Expands area of lower value in an image.

 * @param src the Image to erode.

 * @param kernel the kernel for erosion.

 * @param anchor the center of the kernel.

 * @param iterations the number of times to perform the erosion.

 * @param borderType pixel extrapolation method.

 * @param borderValue value to be used for a constant border.

 * @param dst Output Image.

 */

private void cvErode(Mat src, Mat kernel, Point anchor, double iterations,

    int borderType, Scalar borderValue, Mat dst) {

    if (kernel == null) {

        kernel = new Mat();

    }

    if (anchor == null) {

        anchor = new Point(-1,-1);

    }

    if (borderValue == null) {

        borderValue = new Scalar(-1);

    }

    Imgproc.erode(src, dst, kernel, anchor, (int)iterations, borderType, borderValue);

}

/**

 * Filter out an area of an image using a binary mask.

 * @param input The image on which the mask filters.

 * @param mask The binary image that is used to filter.

 * @param output The image in which to store the output.

 */

private void mask(Mat input, Mat mask, Mat output) {

    mask.convertTo(mask, CvType.CV_8UC1);

    Core.bitwise_xor(output, output, output);

    input.copyTo(output, mask);

}

public static class Line {

    public final double x1, y1, x2, y2;

    public Line(double x1, double y1, double x2, double y2) {

        this.x1 = x1;

        this.y1 = y1;

        this.x2 = x2;

        this.y2 = y2; }

    public double lengthSquared() {

        return Math.pow(x2 - x1, 2) + Math.pow(y2 - y1, 2); }

    public double length() {

        return Math.sqrt(lengthSquared());

    }

    public double angle() {

        return Math.toDegrees(Math.atan2(y2 - y1, x2 - x1));}}

/**
 * Finds all line segments in an image.
 * @param input The image on which to perform the find lines.
 * @param lineList The output where the lines are stored.
 */

private void findLines(Mat input, ArrayList<Line> lineList) {

    final LineSegmentDetector lsd = Imgproc.createLineSegmentDetector();

    final Mat lines = new Mat();

    lineList.clear();

    if (input.channels() == 1) {

        lsd.detect(input, lines);

    } else {

        final Mat tmp = new Mat();

        Imgproc.cvtColor(input, tmp, Imgproc.COLOR_BGR2GRAY);

        lsd.detect(tmp, lines);

    }

    if (!lines.empty()) {

        for (int i = 0; i < lines.rows(); i++) {

            lineList.add(new Line(lines.get(i, 0)[0], lines.get(i, 0)[1],

                lines.get(i, 0)[2], lines.get(i, 0)[3]));}}}}

Looks like that class is correct. Make sure it’s in src/main/java/frc/robot/GripPipeline.java. Your imports in your main Robot class should be import frc.robot.GripPipeline - there is no MyGripPipeline class, so that import should be removed

Alright i changed that, though im still getting an error on the line starting a new vision thread

visionThread = new VisionThread(camera, *HERE*, pipeline -> {

anything i place here will not work

When you compile the class with the build command, what is the specific error message you get? You’ll probably need to scroll up in the vscode terminal to see it

Executing task: gradlew build -Dorg.gradle.java.home=“C:\Users\Public\wpilib\2020\jdk” <

Starting a Gradle Daemon, 1 incompatible and 1 stopped Daemons could not be reused, use --status for details

Task :compileJava
C:\Users\genericname\ROBOTICS CODE FRC\First code\src\main\java\frc\robot\Robot.java:29: error: expected
visionThread = new VisionThread(camera, new , pipeline → {
^
C:\Users\genericname\ROBOTICS CODE FRC\First code\src\main\java\frc\robot\Robot.java:29: error: ‘(’ or ‘[’ expected
visionThread = new VisionThread(camera, new , pipeline → {
^
2 errors

Task :compileJava FAILED
Compilation Error!
GradleRIO detected this build failed due to a Compile Error (compileJava).
Check that all your files are saved, then scroll up in this log for more information.

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ‘:compileJava’.

Compilation failed; see the compiler error output for details.

Assuming visionThread is declared earlier, this line should be

visionThread = new VisionThread(camera, new GripPipeline(), pipeline -> {