Get image from USBCamera into an OpenCV Mat

[rpappa]

I've successfully put opencv on the Roborio and have tested to make sure it works (will do a write up on that soon™). However, I have run into the roadblock of the fact that the USBCamera class has no apparent way to get a straight up Java Image or BufferedImage or something easy to pass to Mat.put().

Here's my code which crashes:

        USBCamera cam = new USBCamera("cam0");
        cam.openCamera();
    	cam.startCapture();
    	ByteBuffer buf = ByteBuffer.allocate(76800);
    	cam.getImageData(buf);
    	matOriginal = new Mat();
    	matOriginal.put(0, 0, buf.array());
    	Imgcodecs.imwrite("output.png", matOriginal);

The line that crashes it is probably

ByteBuffer buf = ByteBuffer.allocate(76800);

But I'm not sure if that matters as I don't believe I am approaching this problem the right way. Any suggestions?

[pblankenbaker]

Have you treid using the VideoCapture class from the OpenCV Java libs to open the USB camera instead of the WPI USBCamera class?

Maybe something like:

VideoCapture vc = new VideoCapture();
vc.open(0); // Assuming USB camera is /dev/video0
if (vc.isOpened()) {
    vc.set(Highgui.CV_CAP_PROP_FRAME_WIDTH, width);
    vc.set(Highgui.CV_CAP_PROP_FRAME_HEIGHT, height);
}

If you are able to open the camera this way, you should be able to grab your OpenCV Mat objects via:

if (vc.grab()) {
    Mat img = new Mat();
    if (vc.retrieve(img)) {
        // Do what you want with the image data in the img Mat object
    }
}

This is basically how we grab our OpenCV images on the driver station side (except we open the IP camera URL instead of the USB camera). If you compiled the OpenCV streaming support in your roboRIO build, I'm assuming it should work there as well.

If you then need to take the Mat and go back to a BufferedImage, here is a snippet of what we use:

	public BufferedImage toBufferedImage(Mat img) {
		if (img.type() == CvType.CV_8UC1) {
			return Conversion.cvGrayToBufferedImage(img);
		} else if (img.type() == CvType.CV_8UC3) {
			return Conversion.cvRgbToBufferedImage(img);
		}
                return null;
	}

[rpappa]

That looks pretty good I'll check it out tonight.

[rpappa]

Worked beautifully. Writing a white paper now.