Definitive Guide to Using the Jetson TK1/TX1?
 

Title explains itself, has anyone written a guide for how to set up, program, and use the Jetson boards in FRC? If not, this could be a good resource for teams who want to use a good coprocessor, but are turned away by the complexity of the project.





(I may or may not be in this group :D)

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

I'm pretty interested in this too - we have a TX1 but haven't set it up yet. We would love to see an FRC-specific guide for it to help us get it running!

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

It hasn't been done yet that I know of but I am far from an authoritative on the TX1. That being said, we released some white papers over the summer and at the end of last season discussing how we used it successfully.

What I will also tell you is that there are lots of groups working on resources to better use the TX1 and ultimately what you are trying to do should determine what hardware and software you are using.

If your goal is to get an awesome vision system running on an FRC robot then have you looked at GRIP yet? Or the PIXY cam?

If your goal is to put a TX1 on your robot and use the GPU then do you realize that OpenCV programming in C/C++ is required because there is no python support for the GPU at present? This also means you are responsible for memory management between the CPU and GPU (it's not done automagically unfortunately). There are tons of great examples for OpenCV though so don't let that scare you but it's something to be aware of.

Also something to consider is size and power requirements for the TX1. Do you know how large the TX1 carrier board is and are you prepared to sacrifice that much space on the robot to hold it? Does your CAD/design team know they need to take that space into account?

Do you know how to power the TX1? It's a bit finicky but with the suggestion from one of the 254 mentors we were able to get ours going last year. Check out the white papers we published, they'll give you a good starting point but there is really no definitive guide.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

I was meaning more of a guide on how to set it up software-wise, like how to install the OS, openCV, etc. I ask because we already have a TK1 that is unused. We used GRIP last year with some issues due to the latency.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

Do you know of any example code for the PIXY on an FRC robot?

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

What issues did you have with grip? We used grip and it ran on our driver station. Our shot was fairly fast as well.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

If you have some time in the offseason, I'd suggest trying out this method of running vision without a coprocessor alongside GRIP. Like RC said, GRIP worked fantastically for us this season and made fast vision tracking a breeze - if you did experience some latency, maybe coupling it with this method could help speed things up a bit.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

The issues we had with GRIP were the framerate and latency, the latter of which was not too horrendous, actually. However, the framerate was not too amazing. Since we already have a Jetson TK1 (from FIRST Choice last year IIRC, but I might be wrong) and an excess of USB cameras, we wanted to try to do some development with that in the hopes that it would improve our vision, which it would. Just thought I'd drop on and see if anyone had already created a guide for this, as I have no experience in the coprocessor department

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

So how were you running GRIP? Was it on the driver station? It's possible to run GRIP on a raspberry pi and then narrow the gap to communicate back to the robot.

The TK1 is what came via FIRST Choice last year, it's a bit smaller than the TX1 physically (at least the board is). It's also a slightly different architecture than the TX1.

At any rate, to your original question, there is no definitive guide for how to use a TX1/TK1 for FRC. These boards are meant more as development environments for companies seeking to build products using these chips and not as turn-key hobbyist robot solutions, even though many people are using them for that.

Some resources for you:

Awesome presentation by Greg McKaskle on vision in FRC:
http://wp.wpi.edu/wpilib/files/2016/...n-in-FIRST.pdf

Wiki for Jetson TX1 info (not always current but useful):
http://elinux.org/Jetson_TX1

As close to a getting started guide as I could find for the TX1:
http://www.jetsonhacks.com/2016/08/1...tson-dev-kits/
http://www.jetsonhacks.com/2016/08/1...tson-dev-kits/
http://www.jetsonhacks.com/2016/08/1...tson-dev-kits/
http://www.jetsonhacks.com/2016/08/1...tson-dev-kits/

One more highly over-looked camera for FRC is the Sony Playstation Eye:
http://www.jetsonhacks.com/2016/09/2...ia-jetson-tx1/

If you are just trying to improve framerate and latency... try to find out what the smallest possible frame size you can use is and still get back the data necessary to perform the task.

Someone else above asked about FRC code for the pixy:
https://github.com/Round-Table-Robot...Virtual_Screen

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

Depends on what you mean by etc, but to install the latest OS on the board use JetPack - https://developer.nvidia.com/embedded/jetpack. That will give you the latest OS, OpenCV, CUDA, and so on. At that point it is a Ubuntu desktop machine, so "sudo apt-get install" will get you a ton of other stuff if you need it.

And for fixing latency, try maxing out the clocks on board when you start up : http://elinux.org/Jetson/Performance

We've had good luck this year using python for prototyping vision code. See the tutorials at http://opencv24-python-tutorials.rea...tutorials.html. Sure, you're not going to get GPU acceleration from this, but realistically you should be able to get goal detection going at a reasonable speed just using the CPU.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

I just did this a couple of months ago. Next time we'll try to document it better.

You need a system running Ubuntu 14.04 to install the Jetpack on. A vm might work but I haven't tried it. I did manage to get it work from 16.04, but I would stick to 14.04 if you can.

The current Jetpack documentation is here:http://docs.nvidia.com/jetpack-l4t/i...4t_install.htm

Make sure you install OpenCV for Tegra and CUDA toolkit. You might not want to install some of the samples or machine learning packages.

The install will walk you through putting the board in recovery mode.

Last year we used two builds of our code. One with a GUI to view the processing for calibration and one without the GUI. We would ssh with X forwarding into the TK1 to use the GUI version.

Our code with here: https://github.com/FRC-Team-4143/vision-stronghold
It documents most of the tweaks to the standard configuration we used. I need to clean up a couple of symbolic links in /etc that git didn't follow.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

So you do need a system running Ubuntu to use the Jetson?

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

It makes updating it easier because of the jetpack software and yes a VM should work without any issues as that is how I did it. Granted, I was using workstation pro on a windows host so your mileage may vary.

Once you have it updated, you can do development locally.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

Nice. Definitely "borrowing" the CUDA inRange implementation for next year.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

I was looking for a "Getting started with the Jetson TX1 for Dummies" type of guide and came up empty handed. Our unit from FIRST Choice arrived today, and I'm already kind of regretting getting it. I understand what you're supposed to be able to do with it, and it seems like it has a lot of potential, but actually using this thing seems way over my head. Other than Team 900, I haven't really found any accounts of a FIRST team successfully using it. Does anyone have any kind of FIRST-specific easy guide for this thing? If not, maybe someone wants to trade me for something simple that I can figure out, like a bunch of hex shaft collars....

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

I am no expert with the Jetson TX1 but I do have some experience from last year when the team I was on got one. I will say it certainly requires some time to be put in use but it is quite powerful and as a student on the team at the time I know I learned a lot from it. As to how to set up, use Jetpack with a guide mentioned above in the thread to get OpenCV installed and start small.

We started late due to focusing on other things with programming since we were switching to C++ that year but that was definitely a mistake on not getting started earlier. As well, we looked at going very ambitious right away with it which meant a lot of development time to get something made but starting small and continually ramping up would have been the much better way to go about it to ensure we had something ready in time for our competitions.

Now the above was generic approach advice but that is mostly because other than the resources Marshall already recommended above, there aren't a ton of things I can specifically recommend you look into since I don't really know your knowledge with networking or working with Linux systems. To try to give a little specific help, for networking an option that may be easier for you to integrate would be Network Tables although its not hard to make an argument for pursuing a custom option with TCP or UDP. As well, something I have found advised is using something like the NavX for timestamps so that you know exactly how long it was from when the picture was first taken to when you are about to act on that data so you can account for robot movement in that time.

Now beyond that I would be happy to give answer any specific questions you have about how to get started with the TX1 with my limited experience. If you are more so wondering on how to design your vision program, we did eventually get a successful vision program made so I can certainly give you a general overview of what I would recommend to do on that end as well.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

I'm actually about to go pick up our TX1 that came in with our first choice shipment. This is going to be a bit of a pet project for me, with the plan being to use it for the robot this year if practical. We spent time this off season exploring other vision options so this may get delayed if something like pixy cam is sufficient for this year.

I will do my best to document as I go, and I will try to be active on here to discuss and answer questions when I can. I don't think there is a single source for reference at this point, but I am happy to contribute. I don't think there will be a definitive reference for this season, but perhaps the best solution is to have a go to thread for discussion and help, and then in the offseason we can work to compile a guide.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

This is exactly how I feel about it: that it might be a little too complex for some people to figure out without a step by step guide, and I'm definitely in that group.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

One BIG thing for software updates:::
**DO NOT UPDATE YOUR UBUNTU O/S to 16.04 on host or on the Jetson!!!!**
(more on this in a moment)

I've been hoping to write such a guide this fall before the kickoff, but work (coincidentally at Nvidia) sort of overwhelmed me between Thanksgiving and New Years, and I don't have anything useful to report. I have written an initial guide for doing the software install, but have not yet gotten into the nuts and bolts of using the new Grip-generated pipeline class. It requires writing a wrapper around the generated OpenCV classes, and hooking into that from a camera source (can't use the new WPILIB cameraServer because that is only ported to work on the rio), and hooking it to a NetworkTables driver to send final target data to the Rio. My research thus far indicates the latencies should be fine if you are just getting your distance-to-setpoint (in degrees) from the coprocessor, and then using a gyro for fast feedback on your PID loop to get to the setpoint.

There is some conflicting info "out there" about what languages will support GPU acceleration in openCV. I'll be asking the local developers within the company for the DEFINITIVE answer w.r.t. Jetsons, and will put that in whatever docs I generate over the next couple weeks (Before I'm utterly buried by build season).

Back to the O/S upgrade comment: the systems currently require Ubuntu 14.04, and upgrading to 16.04 will render the system unusable (it disables all the USB ports, for one thing... so you can't actually log in and type anything to recover the system). You have to reflash, which turns out to be difficult given the requirements of having an Ubuntu14.04 host machine to do the flashing. I have a working recipe for developing a VirtualBox VM of the required Ubuntu image for re-flashing, and making the necessary passthroughs so a physical usb can be driven from the VM to the Jetson. It's not straightforward, and is not officially supported (The local team didn't even think it was possible...). I have a virtual disk image now, but it's surprisingly large (25G of content w/ all the jetpack installation stuff) and I have not yet succesfully transferred it from my work laptop to my personal laptop without errors. Still working on that.

16.04 support requires CUDA9 for some debians or somesuch, which isn't production (or even alpha) ready for Jetsons, so don't rely on having it for this competition build season. Thus, stay with 14.04 and don't ever let the onboard O/S do an update (like I did)!!!!!

That's about all I can say for now, and will (with any luck) get some better data/docs posted to this thread soon-ish. Sorry it's not ready for kickoff.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

Wow! Thank you so much! We were about to get started on our TX1 as well, and you may have just saved us a lot of headache!

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

I can clarify this. With OpenCV 3.1, only C++ has support for GPU acceleration (CUDA or OpenCL). This is the version bundled with the 2017 image, and is the version that GRIP-generated code targets.

OpenCV 3.2, released two weeks ago, added Python support for the TAPI, which allows the library to decide to run certain algorithms on the CPU or GPU via OpenCL. Unfortunately, Jetsons (and other Tegra-based Nvidia boards) don't support OpenCL. So you'd still be stuck with C++ if you want GPU acceleration.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

For another point of view, we've been running a stock Jetpack 2.3 OS install on all of our TX1 boards throughout pre-season and have had no significant issues. We're using CUDA (version 8 is the one provided) both via OpenCV and in stand-alone CUDA code and it works fine. Not sure if s-taylor is using a TK1 or TX1 - if it is a TX1 his experience was much worse than ours.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

This post is terrible and contains very large unfriendly text as well as bad info. It's just spreading FUD (Fear, Uncertainty, and Doubt) without actually helping anyone.

Read the Nvidia docs. You need a 14.04 OS to run the JetPack installer (It says so in the docs). The JetPack installer will then install 16.04 onto the Jetson and all of the OpenCV goodness.

Some Googling will tell you that only C/C++ is supported for the OpenCV CUDA accelerated libraries. Python is not supported for this (I've been telling people this on CD for at least 2 seasons now and no one listens).

While this board is complicated to use. It's not that complicated and doesn't deserve a bad reputation with massive red text.

EDIT: YOU'RE WRONG.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

While a lot of what he says seems to be quite restrictive (only Ubuntu 14.04), technically, that is what the Nvidia docs requires for the GUI. However, there are instructions which require some shell commands here. You could probably flash the TX1 using any GNU/Linux OS (probably BSD too).

I used Ubuntu 15.10 with these instructions, and it worked like a charm. The GUI essentially just does this, but the GUI was extremely messed up on my VM (screen size caused weird adjusting).

In any case, you only need a shell, `tar`, and likely `dd` (I'm not sure what is used internally) to install the L4T suite. This comes with: https://developer.nvidia.com/embedded/linux-tegra-r242

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

OP specifically said upgrading from 14.04 to 16.04 on the Jetson rendered the USB ports unusable and that is simply untrue.

EDIT: Also, you don't need any of those command line tools. Use JetPack.

PLEASE STOP and READ THE DOCUMENTATION.

https://developer.nvidia.com/embedded/jetpack-notes

Note the requirement of a host OS running v14.04 of Ubuntu. Specifically the 64 bit version. This is required for JetPack. JetPack will install 16.04 on the Jetson and it will work afterwards. I've done this about 2 dozen times and it works. I've also recompiled the Kernel for Jetson.

900 is going to be working on a guide for this that doesn't contain a bunch of nonsense. Stay tuned.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

Huh, my mom was right. Not everything on the internet is true.

Large text is fun

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

Wow, with this reception I'm going to bow out of this thread.

I happened to be working directly with the Jetson developers internally, and was reading the documentation being referenced. They were aware of the issues and and the incompatibilities I mentioned. Yes, there appear to be people out there who are getting things to run properly. My TX1 (to answer that question) apparently had a worse time than theirs, and I don't yet know why. Glad there are folks out there who are getting things to work with the Jetsons.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

We've been working with Dustin Franklin and some of the other folks at Nvidia for over a year now on the TX1 and before that we were using the TK1. I don't work for Nvidia and I'm not in direct contact with their internal development team.

What I do know is that you came into a thread and posted gigantic red text about not being able to update the Jetson TX1 to 16.04 and I can tell you I've used JetPack (and the command line utilities the other posted mentioned above) to do exactly this about a dozen times.

There are issues with using 16.04 as a host system. It's why the documentation says the requirement is a 14.04 system. I suppose you could say that it is incompatible but when the requirement states use 14.04... who is at fault?

You stated you were using VirtualBox, which you also stated is unsupported, and having issues transferring the VM to another system which probably points to issues with your host truthfully.

Personally, I've used VMWare Workstation, VMware Fusion, and a dedicated 14.04 host system to run JetPack to run these installs. All have worked fine. I suspect that WorkStation and Fusion use is unsupported but they worked and I've never had a major issue with USB ports not functioning or otherwise.

Also, from a professional standpoint, if you do work for Nvidia then you should take a more positive approach to discussing your company's products on a public forum. Your statements (particularly in big bold red text) could be taken as an official statement and lead to much more complicated trouble for you.

Look man, I care what you want to tell other teams about this thing because Nvidia has been a valuable partner for our team and an amazing resource for FRC. I've had the pleasure of meeting with the CEO to discuss what we do and he's a nice guy who is sponsoring something great with his company. I do not want to see that put in jeopardy because someone is going off half cocked and posting things on CD that aren't true.

Want to get an official statement from the dev folks you are talking to out to teams? Great! Awesome! Do it! Until then, 900 will be working on a guide teams can use. We've had a lot of success with this board.

Re: Definitive Guide to Using the Jetson TK1/TX1?
 

Thanks for your clarifications, Marshall. It's a bit late for me to downgrade the red text since it has been referenced in quite a few replies now, but there are apparently some subtleties here that are not universally understood. (Hopefully "universally" doesn't mean "just me"!).