Am I missing something huge? The ColorImage class in Java doesn’t seem to do much image processing on it’s own - you have to use its method to extract a MonoImage. But then, a MonoImage can only do one thing - find ellipses with the build in method. Is there any way to access individual pixel values? It would be nice to have methods, like blob detection (which labview has) but at least we could develop things like that if we had access to pixel data.
Any help would be appreciated.
It looks like Java currently exposes a very limited set of the NI Vision library through JNA. I’m really hoping that FIRST is planning on giving us an update that wraps some of the functions necessary to find rectangles (like the ones in the Vision whitepaper), I don’t think they would leave all the Java teams to do that on their own. If they don’t have something up by mid next week, I will probably try and get some of them wrapped and will certainly post the modified NIVision.java if I’m successful.
Alternatively you can use the SmartDashboard Installer and program a SmartDashboard extension to do vision processing on the laptop with OpenCV. I’m hoping to get both running to compare framerates and latency.
I was thinking about trying to wrap more of the functions myself. How exactly would you do that in the code?
And how do I program a smartdashboard extension to do vision processing? I had no idea you could do that. Are there instructions?
I was planning on trying to follow the pattern of existing wrapped functions and using the C++ versions of the files to figure out the parameters.
The SmartDashboard Vision stuff looks new and the docs don’t look done yet, but you can see what is there starting on page 85 of the WPILib Cookbook (Can be found in the Documents section of the WPILib project at firstforge.wpi.edu)
To get the fun functions, you need to threshold into a binary image first.
I’ve been doing some work with the wrappers, and have done a few myself. I posted the code for the convex hull here:
It is possible to do it. The Image class has a pointer to the jpeg image in memory named “Image”. It is a public field and did not catch that it was there the first 2 years. You just decode that data into raw bytes and then you can access as you want to as “pixels”.
This is far, far easier said than done. A jpeg image is by its nature compressed. To access the data, you would need to decompress it, which would take a very long time to do. Instead of only accessing the pixels you need, you would have to access all of them. You can read a little more about it here
And that is what I was going with. It is not as easy as it sounds.
If all you can access is the compressed image, then no. You can not access the image data. I ported a JPEG decoder to the cRIO last year, and it did not go well (it worked, but very slowly). I used to think it might work better if done in C++, but now that I know the cRIO has an onboard FPGA, I would bet that it does the decoding.
It’s worth mentioning that we’ve found that calling certain NI C/C++ functions from Java is very slow. I find it surprising that it hasn’t yet been mentioned that the image classes, namely BinaryImage has a very useful set of methods beyond finding ellipses - the particle report methods. Each “blob” in a binary image is considered a particle, and the particle report allows you to access important properties of each particle, like its position, size, number of pixels in the particle, etc.
The jpeg decoder on the cRIO is part of IMAQ. It is pretty standard consortium code and does not run on the FPGA, but runs on theFreescale processor.
Greg Mckaskle
Which functions in particular are you having slow results from? I got slow results from the getOrderedParticleAnalysisReports, but the slowdown was on the java end.
I tossed together a quick sample that retrieved the image from the camera, did a threshold, a convex hull operation, and got the particles, and sorted them by area. The first three of those were done with the BlockingFunction protoyped methods. The whole operation was clocking in at around 400ms with nothing else running on the robot.
What size image was this, and did it include decoding the image?
Greg McKaskle