Usb Camera w/ Java vision tracking on the roboRIO

We had to switch from an Ethernet camera to a USB camera between competitions because our Axis took a dump on us and replacements are hella expensive.

Because the GRIP program doesn’t appear to support taking USB camera streams coming from the roboRIO, we’ve had to generate code from GRIP and put it in the robot program. However, finding the vision code’s outputs is much more difficult this way than the simple NetworkTables that the GRIP program would output. The Java code as shown on the ScreenSteps tutorial is all we have to work with here and it’s a little bit over my head…

We’re trying to take the centerX of the first two detected contours from the Pipeline. We’re putting the number of contours and their center x values on a NetworkTable for debugging. But, the values we see on NetworkTables never update after the code starts.

Here’s the relevant parts of our program:


//there are imports up here but I've omitted them for this post

public class Robot extends IterativeRobot {
NetworkTable db; //the place we put our debug vision values
UsbCamera camera;
private VisionThread visionThread;
private double centerX1 = 0.0; //used for pipeline
private double centerX2 = 0.0; //used for pipeline
private int dasize = 0; //used for number of contours found in pipeline
double xx1 = 0;double xx2 = 0;int listsize = 0; //for NetworkTables
private final Object imgLock = new Object(); //synchronization variable

@Override
public void robotInit() {[INDENT]
db = NetworkTable.getTable("SmartDashboard/DB");
UsbCamera othercam  = CameraServer.getInstance().startAutomaticCapture(0); //Other USB camera used for human driver vision.
othercam.setFPS(15);
camera = CameraServer.getInstance().startAutomaticCapture(1); //Our main vision tracking USB cam
camera.setBrightness(1);
camera.setExposureManual(1);
camera.setFPS(15);

//Most of this code was pulled straight from ScreenSteps and is the source of our gnashing of teeth- it doesn't seem to keep running after the robot is initialized
visionThread = new VisionThread(camera, new GripPipeline(), pipeline -> {
[INDENT]if (pipeline.filterContoursOutput().size() != 0) {
[INDENT]//Get center x of first and second contours
Rect r = Imgproc.boundingRect(pipeline.filterContoursOutput().get(0));
Rect s = Imgproc.boundingRect(pipeline.filterContoursOutput().get(1));
synchronized (imgLock) {
[INDENT]centerX1 = r.x + (r.width / 2);
centerX2 = s.x + (s.width / 2);
dasize = pipeline.filterContoursOutput().size(); //mostly for debugging
}
[/INDENT]}
[/INDENT]});
visionThread.start();
[/INDENT]
}
@Override
public void autonomousPeriodic() {
double xx1 = -1;
double xx2 = -1;
int listsize = -1;
			
synchronized (imgLock) {
[INDENT]xx1 = this.centerX1;
xx2 = this.centerX2;
listsize = this.dasize;
}
			
db.putNumber("CenterX 1", xx1);
db.putNumber("CenterX 2", xx2);
db.putNumber("# of Contours", listsize);[/INDENT]
}
[/INDENT]

Is there anything that looks straight-up wrong with this? We don’t get any error messages of any kind, but we’ve had a plethora of issues from the end of the program where values returned on the NetworkTables don’t update properly. Would you have any idea why this is happening?

I need to figure out what’s happening before we unbag the robot to finish our Robot Access Period. We already burnt 2 of our 6 hours on this alone. If I can’t figure this out when we unbag tomorrow, we’re going to have to scrap vision processing altogether for the season.

Thank you for your time- it is much appreciated!

Assuming GRIP is running on your driver station it should be able to use the stream from the USB camera easily enough. Just add it as an IP camera and use the URL it’s streaming over (http://roborio-6420-frc.local:1181/?action=stream for my team)

Because the GRIP program doesn’t appear to support taking USB camera streams coming from the roboRIO…

See this thread:
/t/grip-on-driver-stattion-connecting-to-usb-cam-on-roborio/156609/1

Thanks, that thread is helpful but how does this MJPEG steam come to be in Java? What code do I have to add? Do I even need any more code or is the URL already out there for me to find.


CameraServer.getInstance().startAutomaticCapture(...);

Will do everything for you, including setting up an MJPEG stream that you can access with GRIP or a browser

I’ve got it now, thanks!