Tracking Rectangles with Java/C++

[Derek012]

If your team isnt 100% against using labview at all you could do the image processing from the Classmates Dashboard by connecting the camera through the bridge instead of the cRio.

[vinnie]

My team is going to be putting a small Atom powered computer on our robot to do vision processing and other high-level functions. We are going to be using the javacv library to utilize OpenCV in Java. We experimented with vision processing on the cRIO last year, but we found that it was very slow and often lagged the rest of the robot functions. You can easily (I wrote a demo program in ~10 minutes) detect edges/contours with OpenCV and from there decide whether or not the contours make the rectangle you're looking for or not.

[davidthefat]

Originally Posted by vinnie

My team is going to be putting a small Atom powered computer on our robot to do vision processing and other high-level functions. We are going to be using the javacv library to utilize OpenCV in Java. We experimented with vision processing on the cRIO last year, but we found that it was very slow and often lagged the rest of the robot functions. You can easily (I wrote a demo program in ~10 minutes) detect edges/contours with OpenCV and from there decide whether or not the contours make the rectangle you're looking for or not.

Good luck doing that.

May I suggest using linux without xserver and just going with C++ to bypass the JVM.

[cesaros]

My Question now is:
Will OpenCV work normally on cRIO, and, will it take a lot of processing from it ?

We often write some huge programs to avoid errors, and it already take a little of cRIO, but, if exits anything that we can use to process this images :/

Using any kind of notebook,netbooks, whatever... Are out of our mind, since our money isn't enough for it.

Thanks.

[abrightwell]

Have you read through this whitepaper?

http://firstforge.wpi.edu/sf/docman/...n.root/doc1302

While it doesn't talk specifically to functions or specific library calls, it does talk about different techniques specifically related to this years game.

[Greg McKaskle]

Just so there is no misunderstanding, the vision libraries that LabVIEW uses are equally accessible from C/C++. The NI product for C development is called LabWindows CVI, so the vision documentation with the CVI suffix is all about the C/C++ entry points.

C:\Program Files\National Instruments\Vision\Documentation contains general documentation about vision processing and LV and C/C++ specific documents and others specific to Vision Assistant. It also has a calibration grid file if you need to correct the images for lens distortion.

The libraries are installed on the cRIO and if you have installed Vision Assistant, I believe you have them on the laptop as well.

It has been a while since I've looked at the WPI wrappers, and at least initially, they tended to hide the imaq entry point rather than simplify them. Perhaps the Examples for C based image processing will be helpful. They are located at C:\Program Files\National Instruments\Vision\Examples\MSVC.

NI doesn't have Java wrappers for the C libraries or for much else. I know some have been added to WPILib, but it is far from complete. If you are more familiar with OpenCV, that is certainly a good option. I wouldn't expect it to be that much difference in performance or capabilities, but both libraries have their specializations and benefits. If you search online, you can probably find some comparisons.

Once again, I'd also encourage you to take advantage of vision assistant, its code generation features and the vision concept manual.

Greg McKaskle

[basicxman]

Originally Posted by Greg McKaskle

Just so there is no misunderstanding, the vision libraries that LabVIEW uses are equally accessible from C/C++. The NI product for C development is called LabWindows CVI, so the vision documentation with the CVI suffix is all about the C/C++ entry points.

So I've spent the morning looking around nivision.h, the "NI Vision for LabWindows/CVI Function Reference Help", and the old ellipsis tracking code to figure out tracking rectangles in C++. Here's a snippet of [entirely untested as I don't currently have access to a cRio] code.

void Camera::FindRectangles() {
    HSLImage  *rawImage  = camera.GetImage();
    MonoImage *monoImage = rawImage->GetLuminancePlane();
    Image     *image     = monoImage->GetImaqImage();
    workingImageWidth  = monoImage->GetWidth();
    workingImageHeight = monoImage->GetHeight();
    delete monoImage;
    delete rawImage;

    RectangleDescriptor *rectangleDescriptor = new RectangleDescriptor;
    rectangleDescriptor->minWidth  = 0;
    rectangleDescriptor->minHeight = 0;
    rectangleDescriptor->maxWidth  = workingImageWidth;
    rectangleDescriptor->maxHeight = workingImageHeight;

    int numCurrentMatches;
    RectangleMatch *temp;
    temp = imaqDetectRectangles(image,
                                rectangleDescriptor,
                                NULL, // Default curve options as per manual.
                                NULL, // Default shape detection options.
                                NULL, // (ROI) Whole image should be searched.
                                &numCurrentMatches);
    matches->erase(matches->begin(), matches->end());
    matches = new vector<RectangleMatch>;
    for (int i = 0; i < numCurrentMatches; i++)
        matches->push_back(temp[i]);
    
    imaqDispose(temp);
}

EDIT: CVI manual located at C:\Program Files (x86)\National Instruments\Vision\Documentation\VDM_CVI_User_Manu al.pdf

[scottbot95]

Would I be correct in assuming that in your example code, Camera is a class you made?

[basicxman]

Originally Posted by scottbot95

Would I be correct in assuming that in your example code, Camera is a class you made?

Yes, once we have our tracking code working completely and performing as we'd like, I'll likely post the full source code and an accompanying whitepaper.

[basicxman]

Originally Posted by basicxman

Here's a snippet of [entirely untested as I don't currently have access to a cRio] code.

Do not use this snippet! After I finally got to experiment with the camera for a while today, I've fixed some flaws in that script and found a better way of doing it entirely.

[JamesBrown]

Originally Posted by vinnie

My team is going to be putting a small Atom powered computer on our robot to do vision processing and other high-level functions. We are going to be using the javacv library to utilize OpenCV in Java. We experimented with vision processing on the cRIO last year, but we found that it was very slow and often lagged the rest of the robot functions. You can easily (I wrote a demo program in ~10 minutes) detect edges/contours with OpenCV and from there decide whether or not the contours make the rectangle you're looking for or not.

I won't say that this is a bad idea but I have seen quite a few teams try something like this and have it cause them more trouble than it is worth. The cRIO is powerful enough to handle vision and motor control, there are thousands of industrial manipulators doing both on the same hardware FIRST uses.

Originally Posted by davidthefat

Good luck doing that.

May I suggest using linux without xserver and just going with C++ to bypass the JVM.

While it will reduce the complexity of setting up the board (computer) they use to some degree it is probably far easier to spend an extra hour configuring their OS of choice than it would be to switch from JAVA to C++.

Originally Posted by cesaros

My Question now is:
Will OpenCV work normally on cRIO, and, will it take a lot of processing from it ?

We often write some huge programs to avoid errors, and it already take a little of cRIO, but, if exits anything that we can use to process this images :/

Using any kind of notebook,netbooks, whatever... Are out of our mind, since our money isn't enough for it.

Thanks.

Look at the files mentioned in Greg's posts they will give you a good idea of what is available for you to use on the cRIO.

Derek mentioned above that you could try to do the vision processing on the classmate or whatever computer you use for an OI. This may be an option for offloading some of the processing, however I'd imagine it would not work for Autonomous.

[shuhao]

Originally Posted by vinnie

My team is going to be putting a small Atom powered computer on our robot to do vision processing and other high-level functions. We are going to be using the javacv library to utilize OpenCV in Java. We experimented with vision processing on the cRIO last year, but we found that it was very slow and often lagged the rest of the robot functions. You can easily (I wrote a demo program in ~10 minutes) detect edges/contours with OpenCV and from there decide whether or not the contours make the rectangle you're looking for or not.

How exactly are you guys accomplishing that? Power connection, networking? Not sure how that would be done as we're also thinking of doing that.

[JacobGH]

I'm working on this problem in Java. Am I crazy or do they only make a method available for detcting ellipses and nothing else? Can we, at least, access the values for individual pixels? That way, if they give us nothing else, we could at-least write our own image processing algorithms.