GRIP on an NVIDIA Jetson TK1

[Dinokaiz2]

Can someone explain or link me to me the steps I should take to

1. Deploy a GRIP pipeline to an NVIDIA Jetson TK1

2. Use a camera (preferably USB) plugged into the roboRIO as an input

3. Access the published values from a roboRIO running Java

Is this even possible?

[vScourge]

Thanks for posting this, I'm also very interested in the Jetson+GRIP scenario, and eager to hear others' experiences/insight there.

It seems like this is possible, but I'm not sure all the connective tissue has developed yet.

This thread on GRIP's Github relates to getting GRIP running on Raspberry Pi 2, which is the same ARM architecture as Jetson.

Last night I installed java 8 on the Jetson, and tried to run the Linux ARM build of the GRIP .jar file provided here (via that Pi thread by user "multiplemonomials"). This started without any architecture/platform errors, but threw an exception about missing some libs, such as NetworkTables. A build of those libs was posted here, which might fix that issue, but I have not had a chance to try them.

I haven't done much yet with an actual camera. I have an older Logitech that the Jetson recognizes as a USB device, but that's as far as I've gone. I have a newer Logitech C920 on order that's confirmed to work on Jetson, but I don't yet know if that will still require installing drivers, etc.

If anyone has more experience with GRIP on the Jetson TK1, please share!

[fovea1959]

will using GRIP on a Jetson take advantage of the Tegra hardware, or will it just use the ARM processor (making it an expensive and large rPi)?

[vScourge]

The hope is it will use the hardware, but I think that remains an open question.

[marshall]

Originally Posted by vScourge

The hope is it will use the hardware, but I think that remains an open question.

It will not use the GPU unless it is coded specifically for it and I do not believe GRIP is currently coded for GPUs though I'm not certain. I have not had a chance to play with GRIP yet but I can help answer questions about using GPUs for vision.

[Tom Bottiglieri]

Originally Posted by marshall

It will not use the GPU unless it is coded specifically for it and I do not believe GRIP is currently coded for GPUs though I'm not certain. I have not had a chance to play with GRIP yet but I can help answer questions about using GPUs for vision.

The Jetson TK1 comes prepackaged (once you run JetPack) with some neat libraries like VisionWorks and OpenCV4Tegra, which both take advantage of GPU acceleration. OpenCV4Tegra is particularly cool as Nvidia has put all of the (normally explicit) GPU functions "under the hood" of the normal OpenCV api, so you can run your normal OpenCV code and get the acceleration bonus.

I'm not super familiar with the GRIP deploy, but I believe they are packaging some pre compiled OpenCV libraries into their Jar, which means you certainly would not get the benefits of hardware acceleration.

[marshall]

Originally Posted by Tom Bottiglieri

The Jetson TK1 comes prepackaged (once you run JetPack) with some neat libraries like VisionWorks and OpenCV4Tegra, which both take advantage of GPU acceleration. OpenCV4Tegra is particularly cool as Nvidia has put all of the (normally explicit) GPU functions "under the hood" of the normal OpenCV api, so you can run your normal OpenCV code and get the acceleration bonus.

I'm not saying you are wrong but I'm not sure where you are getting that the GPU functions are "under the hood" with OpenCV4Tegra. It's not nearly as automatic as you are making it seem from our experience working with it and moving images from the CPU to GPU memory is a chore, though definitely doable.*

Check out this webcast/video about porting OpenCV code over to the Jetson: http://devblogs.nvidia.com/parallelf...etson-tk1-gpu/

As far as I know, OpenCV4Tegra is a pre-compiled and ARM/Tegra/CUDA optimized version of the OpenCV library because building it yourself is a pain though definitely doable.

Also, OpenCV4Tegra is being augmented/replaced with VisionWorks now on the TX1.

*Please note that I could be wrong because I haven't dug into Nvidia's code... just our code that is somewhat based on it/uses it.

[fsilberberg]

GRIP does not take advantage of the GPU: it uses vanilla OpenCV and JavaCV so it does not know how to use it. What you might be able to do is use GRIP to prototype and fine-tune your vision algorithms (with realtime visual feedback and easy modification) and then move that into a program that takes advantage of the GPU.

[Jared Russell]

Originally Posted by marshall

It's not nearly as automatic as you are making it seem from our experience working with it and moving images from the CPU to GPU memory is a chore, though definitely doable.*

It's tricky because the CPU and GPU do not/fundamentally cannot share address spaces. You need to explicitly upload your images to the GPU and download the results, which carries some performance overhead. If you are doing this multiple times per image (or even just once per image in some cases), it is possible that the overhead of all this memory copying more than nullifies the benefits of running a particular operation in parallelized fashion on the GPU.

FWIW, GPU implementations of many FRC-useful OpenCV functions are available (ex. Color space conversion, thresholding, resizing, binary morphological operations, edge and corner finding, feature finding, filtering, line and circle detection, etc.), and better yet a large number of them support streams (they're non-CPU blocking). However, other bread and butter FRC tools like line segment finding, contour finding, and convex hulls must be done on the CPU (unless you roll your own CUDA-powered version). I would be impressed by any team that finds a way to do the entire goal detection task reliably entirely on the GPU.

[Tom Bottiglieri]

Originally Posted by marshall

I'm not saying you are wrong but I'm not sure where you are getting that the GPU functions are "under the hood" with OpenCV4Tegra. It's not nearly as automatic as you are making it seem from our experience working with it and moving images from the CPU to GPU memory is a chore, though definitely doable.*

Check out this webcast/video about porting OpenCV code over to the Jetson: http://devblogs.nvidia.com/parallelf...etson-tk1-gpu/

As far as I know, OpenCV4Tegra is a pre-compiled and ARM/Tegra/CUDA optimized version of the OpenCV library because building it yourself is a pain though definitely doable.

Also, OpenCV4Tegra is being augmented/replaced with VisionWorks now on the TX1.

*Please note that I could be wrong because I haven't dug into Nvidia's code... just our code that is somewhat based on it/uses it.

I misread the docs, you are totally right. Still, it seems using the tegra version gets you a few cpu based optimizations. And you can use the GPU module out of the box, which seems nice.