Unofficial Guide to the Jetson TK1

[oshamaa]

I have been looking around to see if anyone has posted a Guide or tutorial on using the Jetson TK1, because I personally find it perfect for FRC. I am apart of team 5572, and despite having never worked with ubuntu for any developing with Ubuntu last year at the beginning of the season, I decided to learn how to use the Jetson TK1, and program with OpenCV. Now I will try to teach you what I learned.
Required:
*Ubuntu Host System
*Ubuntu 14.04
*USB FTDI to Null Modem Serial UART


First off, Setup:
1. Sign up for a NVIDA Developer account, so you can access the resources and factory flash it so you can repair the Jetson if it gets messed up,
-Register Link
-Downloadable Resources
2. Flash the Jetson TK1 with the Grinch kernel
-Here is the install tutorial
-Here is the install script(NOTE: Remember to add executable permissions to the installGrinch*.sh by using "chmod a+x ./installGrinch*.sh")
3. Install OpenCV
-Press Control+Alt+T, to open a terminal
-type "sudo add-apt-repository universe", to ad the universe repository
-Run this install script (NOTE: Like the previous install script it needs to be executable by using "chmod a+x install-opencv.sh")

Now its time for the hard part(Using the Jetson via Command Line Interface):

After you make a OpenCV program, preferably in C++ or python, you have 2 options to communicate to the RoboRio
* Piping the output of the program to a dev file which communicate out of a port, which makes it output only.
* Creating a function to communicate, which allows for bi-directional communication.

To communicate out of the Serial DB9 Port, you will need a Null Modem to USB cable. When you plug the cable into the Jetson, no dev files will be created, this is because the Serial Port's dev file is named "ttyS0" and is always active. To put an output through the Serial Port escalate your privileges to root by using "su" in terminal, and run 'echo "Hello World" > /dev/ttyS0'. To read from the Serial port you would need to escalate your privileges as well, and run "sudo cat /dev/ttyS0". If you want to test the RoboRio reading/writing, you would need to ssh into the roborio with putty or a native ssh application. If you have a computer with the driver station to tether your computer to the RoboRio via USB, this link may also work.
HostName: roboRIO-[TEAM NUMBER]-FRC.local
Username: admin
Password: <none>
It is not over yet, you need to make sure that the USB can actually recognize the format the Jetson is outputting. To do this you will need to make sure that the usb is recognized, to do this unplug the usb that connects the Jetson and the roborio, and run this "ls /dev/ttyUSB*", anything returned is not the Jetson. Plug the cable back in an run "ls /dev/ttyUSB*", and there should be a new file there, that new file is the file you would use to communicate to the Jetson TK1 the exact same way you would communicate as if you were on the Jetson, except no root required. Run this to configure the USB port on the roborio while SSH'd as admin.

sudo stty -F /dev/ttyUSB0 115200 intr ^C quit ^\\ erase ^? kill ^U eof ^D start ^Q stop ^S susp ^Z rprnt ^R werase ^W flush ^O min 1 time 0 -parenb -parodd cs8 hupcl -cstopb cread clocal -crtscts -ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl ixon ixoff -iuclc -ixany -imaxbel -iutf8 opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0 isig -icanon -iexten -echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke

(REMEMBER: The USB file may not always be ttyUSB0, but the line above only works for ttyUB0)

Probably the hardest part for most teams that attempt this and get their program working, is deploying it so it automatically start it, my team has made what I consider a really good method of deployment. What we did is we modified the /etc/rc.local file to mount /dev/mmcblkp1 so the SDCARD would be accessible. From there we made it run a "setup" shell script, and recompile the program on the SDCARD, and run that compiled program. This solution I think was the best possible solution, because it allowed for changes to the vision processing program without a monitor/keyboard so we could deploy quick changes incase of anything.
SDCard Directory

If you do not want to use this way however, there is another way. The RS-232 port on the roborio works on 3.3v and 5v base ports, while the UART PINs on the Jetson TK1 works 1.8v, to make these communicate you would need a level shifter.

Using these ports is the same as serial, you would just need to "cat" or "echo" into the specified dev file. In this case I would recommend using /dev/ttyTHS0 and a level shifter in between the connections from the Jetson to the roborio.

I apologize if you get confused at any point, and please reply with any questions/clarifications. I hope this is helpful for the most part.
Have fun experimenting!!

[oshamaa]

Sorry for the improper grammar, I forgot to double check it.

[AMendenhall]

Very informative. Thanks for the post.

Why not use the ethernet port on the Jetson to put data directly into NetworkTables where it can be accessed by both the RoboRIO and the DS? Also, why not use Java; that's what is recommended by WPIlib?

[oshamaa]

I think, not quite sure, that using if you want the fasted vision processing you would want to use OpenCV, and OpenCV does not have a native API for Java, Java would just slow it down. I would not use NetworkTables, because that would also slow the process down, and risks the max-bandwidth being met in matches. With this method we had the max frames per second last year, specifically it only used 25% of our Cores.

[oshamaa]

fastest*

[SamCarlberg]

Originally Posted by AMendenhall

Very informative. Thanks for the post.

Why not use the ethernet port on the Jetson to put data directly into NetworkTables where it can be accessed by both the RoboRIO and the DS? Also, why not use Java; that's what is recommended by WPIlib?

The Java bindings don't support CUDA. You have to use C++ to get GPU acceleration, which is the entire point of the Jetson.

Originally Posted by oshamaa

I think, not quite sure, that using if you want the fasted vision processing you would want to use OpenCV, and OpenCV does not have a native API for Java, Java would just slow it down. I would not use NetworkTables, because that would also slow the process down, and risks the max-bandwidth being met in matches. With this method we had the max frames per second last year, specifically it only used 25% of our Cores.

OpenCV does have an official Java API. It just doesn't include the CUDA functions. NetworkTables is asynchronous; it wouldn't slow down the vision processing. There'd be a bit of latency (up to 10ms-100ms depending on how you configure it), but uses very little bandwidth, so your final statement there is just plain wrong.

[oshamaa]

Thank you for correcting me, I was not aware of that.

[TheGuyWhoCodes]

Originally Posted by AMendenhall

Very informative. Thanks for the post.

Why not use the ethernet port on the Jetson to put data directly into NetworkTables where it can be accessed by both the RoboRIO and the DS? Also, why not use Java; that's what is recommended by WPIlib?

Our team tried to use Network Tables with the Jetson last year. It was working fine before regionals, but when we got there it totally quit working after flashing the router for competition. Both the Roborio and the Jetson wouldn't communicate at all, and we spent an entire day trying to get it working. In the end, we got it so we could run our vision code on our driver station and sent data to the rio that way.

[Bo8_87]

Originally Posted by SamCarlberg

The Java bindings don't support CUDA. You have to use C++ to get GPU acceleration, which is the entire point of the Jetson.
OpenCV does have an official Java API. It just doesn't include the CUDA functions.

Does the python api for OpenCV support CUDA?

[SamCarlberg]

Originally Posted by Bo8_87

Does the python api for OpenCV support CUDA?

No. Only the C++ API has CUDA support.

[marshall]

Originally Posted by SamCarlberg

No. Only the C++ API has CUDA support.

This is accurate.

Further. We used NetworkTables when we did this with the TK1 a couple years ago:

https://m.youtube.com/watch?v=4KlYdCBdjEg

We've moved on to using ZeroMQ these days.

[AMendenhall]

I loved your team's cascade classifier - that was super cool.
Why do you use ZeroMQ instead of NetworkTables? Did you have issues similar to TheGuyWhoCodes?

[marshall]

Originally Posted by AMendenhall

I loved your team's cascade classifier - that was super cool.
Why do you use ZeroMQ instead of NetworkTables? Did you have issues similar to TheGuyWhoCodes?

Thanks! We try to do cool stuff (note: I didn't say it always works).

Well, last year we made the switch before the NetworkTables rewrite was complete and we haven't looked back yet. NetworkTables has evolved a lot and it works nicely. ZeroMQ is easy to get moving with though. I'll also add that we're investigating ROS but I can't say it's viable for our usage yet.