Hi all!
Our team just got around to using our Jetson, and we wanted to use it for OpenCV vision tracking. I installed the default Nvidia package on it, along with CUDA and Jetpack. I understand how to train OpenCV and all that, what I can’t seem to get a clear answer for is where should I do the training and software development? Some sites say to do it on the jetson locally, others say to do it on a Linux system and then transfer the files over, which one is the easiest (for a semi-advanced programmer)?
Thanks in advance!!
Can you define what you mean by “training?” Are you referring to an OpenCV cascade classifier, or do you just mean writing vision scripts? We do all of our development on our laptops, then transfer over everything when we’re ready to test and integrate.
As for cascade classifiers, it’s probably going to be faster to train those on a laptop. What are you thinking of using them for (if I interpreted your question correctly)?
The more system resources you have the faster it will go. The Jetson is powerful relatively speaking but compared to a work station type desktop, not so much. So how much “training” will you do and how many objects will you “train”, and how much time do you have? Those would be the questions to ask yourself.
So we have used both the TK1 and TX1, and have had great success with both. There are a few terms in your message which may mean other things, but here is what I would recommend:
- When talking about development for a TX1/TK1, the Roborio, or another headless platform (one without a monitor/keyboard) there are two primary ways to approach: Cross-compile (Do the development on a completely separate machine, write all the code, create the binary, and then transfer it over to the target for use.) (This is exactly how every team develops for the RoboRio, and you can do it similarly for the TK1/TX1). This what my team does and supports our multiple person development team, allows us to source control our software, and we can write our software to run on a desktop when the TK1/TX1 is not available.
The second way to do it is try and develop directly on the embedded device itself. Sometimes this isn’t possible, but for the TK1/TX1 it is.
Cross-compiling is a bit more complicated, you need to understand a lot more about tool-chain development and cross compiling, and it will be a lot more difficult for a beginner. This is the approach we use on my team, but we have a lot of knowledge in that field, and dedicated controls mentors to support. There isn’t a lot of good information on cross compiling for TK1/TX1, and while we are hoping to produce it, we haven’t been able to just yet.
Developing directly on the TK1/TX1 is a lot easier to start off with, but also has its issues. There are two primary ways to do this type of development:
- Plug a keyboard/mouse and monitor into the TK1 and use it like a computer running Ubuntu
- Connect to it over Ethernet using SSH.
When considering that the TK1/TX1 will be installed in a Robot, the SSH route may be a lot easier to handle vice having to plug in montitors/keyboards to debug/troubleshoot. You can use putty on windows or any Linux based computer to SSH into the TX1/TK1 and develop remotely. One of the biggest issues most teams have with developing directly on the TX1/TK1 is the lack of code tracking because the TX1/TK1 is usually never on the internet and github is unavailable (where as your development computer in cross compiling can be). The other is that if you only have 1 device like we do, it makes it hard for multiple people to collaborate. Some teams use this method and have multiple devices (we don’t). These are not major issues, but for developing, it can be a bit annoying, depending on how your development team is structured.
I typically instruct teams to follow a tiered approach:
- Use keyboard/mouse and monitor to start development, so you can spend more time focusing on development and less with all the issues of communication with tool chains etc.
- Once you are comfortable with developing/running code on TX1/TK1, try to graduate to using SSH, don’t use keyboard/monitors anymore, but use another machine to remote into the TX1/TK1 to get your tasks done, modify code and run though a debug process.
- If you are really comfortable with that, graduate to cross-compiling if there is a need to do so.
Make sure to set a static IP for your TK1/TX1 in all cases so you don’t rely on DHCP. use a 10.TE.AM.XX address. I also have my home network, and a lot of my developers also have their home network internal DHCP set to 10.TE.AM.xx so they can plug in at home and work as well - its a very hand little trick. This allows you to set the IP once on the TX1/TK1, and you can plug it into your robot, or your home network (with internet) and it will just work.
Deepending on your comfort level with embedded devices you can start at #2.
Now on to your use of “learning”. This word triggers a lot of people to discuss deep learning principles. And while it is a very cool topic (I run a bunch of nets on Amazon AWS for autonomous car learning), I will always hold the stand that for FRC it is not required to solve the types of computer vision tasks we have.
While not knowing your own abilities, I highly recommend before attempting to do any deep learning, you first develop a more modest computer vision algorithm that uses the basics of OpenCV to do color thresholding, object detection etc. There are a lot of hurdles to overcome and if you want a challenge it will still be plenty of challenging for anyone to go this route. The computer vision part is just 1/2 of the problem. Once the object is detected, the other half is the automated control system side to make your robot do something with this information. So if you want a challenge to keep you busy, you can still get it even without going the deep learning route.
At the end of the day, your trying to create a solution to give you team a competitive edge in competition, that is your primary goal. Once you do that, you can focus on doing so in more challenging ways like Deep Learning if you so desire.
So I recommend making the simplest end to end system now, then slowly increasing the complexity as required to meet your goals, requirements, and the competitive edge you wish to have. You always want to start as simple as possible and grow in scope or complexity. Some people will try to just start at the most complex solution first, because the simple solution won’t do all of what they want and then realize the issues in the details of developing that complex system prevent them from meeting the goal in a timely manner.
Again, your experience and knowledge base trumps the direction, the above assumes you don’t have a background in either computer vision or deep learning. However If you are more comfortable with deep learning principles as applied to object recognition because you or your teams mentors have a lot of experience in that field, you may want to focus on that. You always want to play to your strengths, your mentors strengths, and your teams strengths first.
Hope this helps,
Kevin