Our team is trying to do Driver Station image processing from a Microsoft lifecam HD-3000 HD USB camera which is plugged into the robot. We began working with the camera connected to the laptop, converting to and from a BufferedImage with
public static Mat bufferedImageToMat(BufferedImage i){
byte] pixels = ((DataBufferByte) i.getRaster().getDataBuffer()).getData();
Mat img = new Mat(i.getWidth(),i.getHeight(), CvType.CV_8UC3);
img.put(0, 0, pixels);
return img;
}
public static BufferedImage matToBufferedImage(Mat matrix) {
int cols = matrix.cols();
int rows = matrix.rows();
int elemSize = (int)matrix.elemSize();
byte] data = new byte[cols * rows * elemSize];
int type;
matrix.get(0, 0, data);
type = BufferedImage.TYPE_3BYTE_BGR;
// bgr to rgb
byte b;
for(int i=0; i<data.length; i=i+3) {
if(i < data.length - 3){
b = data*;
data* = data*;
data* = b;
}
}
BufferedImage image2 = new BufferedImage(cols, rows, type);
try{
image2.getRaster().setDataElements(0, 0, cols, rows, data);
}catch(Exception e){
}
return image2;
}
The BufferedImage is of type BufferedImage.TYPE_3BYTE_BGR*** and the code works perfectly when the camera is plugged into the laptop. The problem arises when we are using the camera plugged into the robot. The image is acquired using a modified version of the Smart Dashboard USB Camera Viewer Class included in the java wpilib. When we try to convert the buffered image to a mat, and back to a buffered image (to simulate “Processing”) the image has lines throughout it, and is distorted and unusable.
Sorry for the reply to my own thread. Does anyones team use opencv, or know a better way to retrieve images from the usb camera then using the code from SmartDashboard’s USB Camera viewer class
Thanks for the reply, we are currently using OpenCV on the Driver Station, and would prefer to do processing there in order to save RoboRIO CPU for max output and reaction speed. We are using the function from the Smart Dashboard camera viewer in order to get the image from the camera. This works well and gives us a BufferedImage of type TYPE_3BYTE_BGR. We can display the image perfectly fine through a simple gui. The problem is with our bufferedImageToMat(Mat img) function:
public static Mat bufferedImageToMat(BufferedImage i){
byte] pixels = ((DataBufferByte) i.getRaster().getDataBuffer()).getData();
Mat img = new Mat(i.getWidth(),i.getHeight(), CvType.CV_8UC3);
img.put(0, 0, pixels);
return img;
}
However this is exactly the same as many of the online examples, if anyone has any ideas that would be appreciated. Also, what does everyone think about using OpenCV on the roboRio
I don’t believe this is the typical way of converting a Mat to a BufferedImage. From my experience it is much similar to use MatOfByte:
// Lots of code...
Mat mat = getSomeImageMat();
MatOfByte bytes = new MatOfByte();
Highgui.imencode(someFormat, mat, bytes);
byte] arr = bytes.toArray();
ByteArrayInputStream bis = new ByteArrayInputStream(arr);
BufferedImage img = ImageIO.read(bis);
// Even more code...
// Exceptions and Stream Closing omitted for brevity
This is what I have seen typically used. It’s much simpler.
(I noticed you used this algorithm, but I will leave it here for future reference)
Is there some limitation where you can’t use MatOfByte?
I attached a screenshot of how the images come out with my current method after converting from BufferedImage to Mat, im going to try with the MatOfBte Method and see if that works, thanks for the help
No, there is no limitation where we cant use MatOfByte, but i don’t believe the issue lies in that part of the code, i can convert a Mat to a BufferedImage without any problems… the problem comes in when I try to take the image from the usb camera, which comes in as a BufferedImage, and convert it to a Mat.
