Hello all, my team just got a Jetson Tk1 from FIRST Choice, and have no clue on how to use it. A cursory search for documentation yielded next to nothing, so we need help on getting this thing up and running. I have a few specific questions:
Is this board meant to be used as a coprocessor strictly for vision, or can you run sensors to it as well?
How do we interface with it? Is it part of the robot, or the DS? Additionally, how should it be wired?
What sort of programming will we need to do on this? I understand that it runs a version of Ubuntu 14.04, do we need to run scripts on boot in order to use it in a competition?
Vision processing has been at a minimum in years past for us, so we are all unfamiliar with things such as OpenCV. Help and/or documentation for related vision processing software/libraries would also be greatly appreciated. If I come up with any more questions, I’ll comment on this thread with them.
It is a full Linux computer so it has a lot of different uses, but it has a powerful GPU which makes it well suited for computer vision. For most sensors (besides the camera), you are probably better off using the roboRIO because it has better support for standard sensor protocols.
You would have to make custom software to interface with it, probably over ethernet. I don’t really know about wiring it, besides connecting its ethernet port to the D-Link.
You can program it in any language that is supported on Linux, but if you are doing computer vision you probably want to use C/C++ for maximum performance.
It’s built for computer vision and computer graphics sensors is kind of a waste. You would put it on the robot.
Easiest interface is via RS-232 though this year using GPIO would be sufficient. If you were to use GPIO between this board and the Rio note that the Jetson has 1.8V logic level, thus you need a logic level converter (Rio is 5/3.3V).
You can get a barrel jack and wire it to the 12V 2A port on the VRM for power. DO NOT connect it directly to the PDB, it needs a very specific 12V. Yes you do need to run scripts on boot in order to use it in a competition. You also need to have it shutdown on command before the end of the match so you aren’t doing hard shutdowns all the time.
Just put a USB camera on it. The Logitech C920 works out of the box and is arguably the best option for us. No networking, no separate power cable, and it’s smaller than the axis. If your only using one it’s literally plug and play.
The point of this thing is that it has a GPU which is good for computing several elements in a matrix all at once. In image being a matrix of pixels can thus have math preformed on groups of pixels all at once as opposed to one at a time as a traditional processor would. To use the GPU you need to program in C++ with the Open CV CUDA libraries. If you want to use the 2.32 quad core processor on the thing (faster than anything else you could put on the robot at that size and cost) you can use any language supported by Open CV (C++, Python, Java, etc.).
I’m going to go in reverse order… since vision processing has been minimal for you in years past… the Jetson is the equivalent of jumping into the deep end to learn how to swim. That’s not bad… it’s just a warning.
It is an embedded computer. You can use it how you like but the “sweet spot” for this board is with the Nvidia GPU that can, in theory, be used for better vision processing. It takes special coding to make that work though.
You can interface it with the driver station or the robot. It’s probably safer to keep it on the driver station side but you could place it on the robot side. I recommend wiring it to the VRM via the 12v/2A port BUT you don’t have to do that. In fact, this board can in theory pull more than 2A. It is considered a custom circuit so you could wire it up to a 40A breaker per R37.
To take full advantage of this board you will want to do C++ programming with OpenCV. You will probably need to run custom scripts.
If anyone has other questions then please feel free to ask. We’ve been playing with these boards for about 9 months now. We now own 4 or 5 of them between the team and a few mentors.
Also, it’s worth mentioning that not all CUDA cores are created equal. The CUDA implementation for Nvidia’s high end graphics cards is not equivalent to the CUDA implementation for these boards, despite what Nvidia’s marketing department would like you to believe.
Thanks for the input. I realize that this is quite a leap, and I’m willing to work through that. As for looking around on CD, I did look at that thread but was really looking for more tutorial-esque style help (thus this thread). As for interfacing with the DS or robot, which would you consider the most efficient? I believe that keeping it on the robot would be simpler, though I may be wrong.
The only tutorials you’ll find are on the eLinux Wiki page (Jetson TK1 - eLinux.org) and the Nvidia developer website which is where they tell you how to set it up. I started with the wiki which has all the steps and links to the dev site. Other than that the tutorials you need are for using Linux Ubuntu, developing with OpenCV, using the OpenCV CUDA library, and how RS-232 works.
Robot is simpler. Plug in USB camera, plug in DB9 cable, plug in power, done. Also regulating the voltage going to the Jetson is a good idea (what else are you putting in to the 12V 2A VRM port?).
There aren’t many tutorials specifically for FRC just yet. I suspect we’ll see a few show up starting this year and into next.
The elinux site linked above has a few tutorials but I won’t claim any of them are great and most involve knowing the internals of linux to some degree. Key words to Google for are Jetson, OpenCV, Nvidia, TK1, Robotics, etc…
Believe me, I would love to point you at more specific resources but they just aren’t out there at the moment that I’m aware of (not that I’m the arbiter of all things Jetson and FRC… trust me, I’m not even close).
As for efficiency. It all depends. Assuming you mount the camera on the robot and not on a pole over the driver station, the most efficient use of the Jetson is to put it on the robot and then connect to it from the RoboRIO for automated control. This way you never send data back over the field wireless to the Jetson and all of your processing is done on board the robot.
If instead, you were to opt for putting a camera above your driver station… then putting the Jetson in the driver station is probably your best bet.
I’m also hoping that Nvidia will release a new version of this board with their latest and greatest within the year but we shall see… Imagination and Intel are both making solid progress in the mobile GPU and GPGPU fields so I expect to have more options in the years to come… I suspect Nvidia ended up making these boards because they had a surplus of the chips for them but that’s a whole different conspiracy theory.
If you run into specific problems then feel free to PM me and I’ll get you help as best as I can. I do my best to monitor CD but these last few weeks are always killer.