OpenCV USBCamera convert to Mat

[JML]

Our team is trying to do Driver Station image processing from a Microsoft lifecam HD-3000 HD USB camera which is plugged into the robot. We began working with the camera connected to the laptop, converting to and from a BufferedImage with

public static Mat bufferedImageToMat(BufferedImage i){
byte[] pixels = ((DataBufferByte) i.getRaster().getDataBuffer()).getData();
Mat img = new Mat(i.getWidth(),i.getHeight(), CvType.CV_8UC3);
img.put(0, 0, pixels);

return img;
}
public static BufferedImage matToBufferedImage(Mat matrix) {
int cols = matrix.cols();
int rows = matrix.rows();
int elemSize = (int)matrix.elemSize();

byte[] data = new byte[cols * rows * elemSize];
int type;
matrix.get(0, 0, data);

type = BufferedImage.TYPE_3BYTE_BGR;
// bgr to rgb
byte b;

for(int i=0; i<data.length; i=i+3) {

if(i < data.length - 3){
b = data[i];
data[i] = data[i+2];
data[i+2] = b;
}

}

BufferedImage image2 = new BufferedImage(cols, rows, type);
try{
image2.getRaster().setDataElements(0, 0, cols, rows, data);
}catch(Exception e){

}
return image2;
}

The BufferedImage is of type BufferedImage.TYPE_3BYTE_BGR and the code works perfectly when the camera is plugged into the laptop. The problem arises when we are using the camera plugged into the robot. The image is acquired using a modified version of the Smart Dashboard USB Camera Viewer Class included in the java wpilib. When we try to convert the buffered image to a mat, and back to a buffered image (to simulate "Processing") the image has lines throughout it, and is distorted and unusable.

Any assistance would be greatly appreciated.

[JML]

Sorry for the reply to my own thread. Does anyones team use opencv, or know a better way to retrieve images from the usb camera then using the code from SmartDashboard's USB Camera viewer class

[Adam8234]

If you're running OpenCV on the RRio itself create a VideoCapture (http://docs.opencv.org/java/org/opencv/highgui/VideoCapture.html) object then call the read/retrieve method to put it into a Mat.

[JML]

Thanks for the reply, we are currently using OpenCV on the Driver Station, and would prefer to do processing there in order to save RoboRIO CPU for max output and reaction speed. We are using the function from the Smart Dashboard camera viewer in order to get the image from the camera. This works well and gives us a BufferedImage of type TYPE_3BYTE_BGR. We can display the image perfectly fine through a simple gui. The problem is with our bufferedImageToMat(Mat img) function:

public static Mat bufferedImageToMat(BufferedImage i){
byte[] pixels = ((DataBufferByte) i.getRaster().getDataBuffer()).getData();
Mat img = new Mat(i.getWidth(),i.getHeight(), CvType.CV_8UC3);
img.put(0, 0, pixels);

return img;
}

However this is exactly the same as many of the online examples, if anyone has any ideas that would be appreciated. Also, what does everyone think about using OpenCV on the roboRio

[Adam8234]

Thanks for the reply, we are currently using OpenCV on the Driver Station, and would prefer to do processing there in order to save RoboRIO CPU for max output and reaction speed. We are using the function from the Smart Dashboard camera viewer in order to get the image from the camera. This works well and gives us a BufferedImage of type TYPE_3BYTE_BGR. We can display the image perfectly fine through a simple gui. The problem is with our bufferedImageToMat(Mat img) function:

public static Mat bufferedImageToMat(BufferedImage i){
byte[] pixels = ((DataBufferByte) i.getRaster().getDataBuffer()).getData();
Mat img = new Mat(i.getWidth(),i.getHeight(), CvType.CV_8UC3);
img.put(0, 0, pixels);

return img;
}

However this is exactly the same as many of the online examples, if anyone has any ideas that would be appreciated. Also, what does everyone think about using OpenCV on the roboRio

This seems odd. Can you provide some screenshots?

[MatthewC529]

I don't believe this is the typical way of converting a Mat to a BufferedImage. From my experience it is much similar to use MatOfByte:


// Lots of code...

Mat mat = getSomeImageMat();
MatOfByte bytes = new MatOfByte();

Highgui.imencode(someFormat, mat, bytes);

byte[] arr = bytes.toArray();
ByteArrayInputStream bis = new ByteArrayInputStream(arr);

BufferedImage img = ImageIO.read(bis);

// Even more code...
// Exceptions and Stream Closing omitted for brevity


This is what I have seen typically used. It's much simpler.

(I noticed you used this algorithm, but I will leave it here for future reference)
http://stackoverflow.com/questions/14958643/converting-bufferedimage-to-mat-in-opencv

Is there some limitation where you can't use MatOfByte?

[JML]

I attached a screenshot of how the images come out with my current method after converting from BufferedImage to Mat, im going to try with the MatOfBte Method and see if that works, thanks for the help

[JML]

Is there some limitation where you can't use MatOfByte?

No, there is no limitation where we cant use MatOfByte, but i don't believe the issue lies in that part of the code, i can convert a Mat to a BufferedImage without any problems... the problem comes in when I try to take the image from the usb camera, which comes in as a BufferedImage, and convert it to a Mat.

[JML]

For whatever reason, switching the Mat Declaration to the one shown seemed to fix the problem, for anyone who finds this in the future.

public static Mat bufferedImageToMat(BufferedImage i) {

byte[] pixels = ((DataBufferByte) i.getRaster().getDataBuffer()).getData();
Mat img = new Mat();
img = new Mat(new Size(i.getWidth(), i.getHeight()), CvType.CV_8UC3);
img.put(0, 0, pixels);

return img;
}